This guide will show you how to set up an A/B split test in Google Ads or Facebook to measure the effectiveness of weather targeting on your campaigns. Creating a split test can give you concrete metrics on the impact that the weather targeting has had on your campaign performance. We recommend letting your A/B split test for 10-15 days as this will allow enough time for the test to produce conclusive results.

Setting up a split test in Google Ads is very straightforward with Google Ads Experiments. There are a number of variables you can test, but for the purposes of this guide we’ll show you how to set up a simple test to measure effect of weather targeting on campaign performance (IE, we'll be comparing a weather targeted vs. non-weather targeted variant of the same campaign). Simply follow the steps below:

1. Log into your Google Ads dashboard & click on the campaign you wish to run an A/B split test on.

2. Click on ‘drafts’ in the upper right hand corner, select ‘new’, and name your draft.

3. Hit ‘Create’ and then select ‘Run an experiment’

4. Name your experiment and allocate a traffic split between the original and the test variant. (if you’re unsure if weather targeting will make a difference, try a 50-50 split). If your product is highly weather sensitive and you want the bulk of your impressions to be weather targeted, you can choose a higher percentage in favor of weather targeting, e.g. 80-20).

5. Navigate back to WeatherAds and import one of variants in for weather targeting (we recommend importing the test variant). Create your weather targeting rules and set live.

Google Ads effectively treats the variants as independent campaigns, complete with their own metrics. However, the two variants will not compete against each another, instead sharing traffic and budget.

Crucially, the weather targeting that runs on one variant will not affect the other, as you can see from the screenshots below:

Weather targeted variant - you can see the original geo (UK) has been automatically broken down into more granular locations by WeatherAds, and weather based bid modification are being actioned.

Original variant (non-weather targeted) The original geo (UK) has not been broken down into more granular locations, hence this campaign variant is not being weather targeted.

6. Once your experiment has some history, Google Ads will show you a report that compares the results from both campaigns, so that you can easily identify the best performer. It will look something like this:

NOTE: You can run a split test on either a fresh campaign or a campaign that has some history. However, we do not recommend running a split test on a campaign that has already been running in WeatherAds.

NOTE: Any manual changes you make to your original campaign will not be reflected in the variant campaign. We do not recommend making manual optimizations to your campaign during the test period as this could void the experiment. However, If you absolutely must make changes, remember to make them across both variants.

In the above example we’ve used the most obvious test – weather targeted v.s non weather targeted variant. However, you can also use this method to test other variables, for example:

Like the Experiments feature in Google Ads, Facebook also has the ability to create a split test. Here's their help article on it. Below is a step by step guide to setting up your A/B Split Test in Facebook to determine the value of running weather targeting on your campaign:

1. Log into your Facebook Ads dashboard, and click on the green ‘Create’ button.

2. Pick your marketing objective, and then tick ‘create split test’ checkbox

3. Under the ‘What do you want to test?’ dropdown, select ‘creative’. Since there is no option to test a 'custom' variable (E.G. Weather targeting), we will have to use a workaround. The idea here is we will be testing two identical creative, but one will have a WeatherAds automation applied to it, whilst the other won't.

4. Fill in your audience & budget as you normally would. Pick a start and end date for your split test. We recommend running it for around 10 days.

5. Upload your creative into Ad Set 'A' and create your ad. When you’re happy with it click ‘continue to Ad B’

6. Your ad should be copied over into Ad Set B. Now you have to make a minor change to your second ad because when you're split testing creative, Facebook won’t allow you to test identical ads. However, the most minor change between the two creative variants is enough to approve the test. We recommend putting an extra space at the end of your headline - which is enough for Facebook to see that the ads are different, yet still imperceptible to the viewer). The aim here is to make a change which is so innocuous it will not impact the results. When you’ve done that click ‘confirm’.

7. Voila! Your split test has been created. You can navigate to your ad sets and rename them so you know which one is the weather targeted variant.

8. Head over to your WeatherAds dashboard, go to import campaigns, and import in one of your newly created ad sets for weather targeting. Do not import the other variant, which will be your control group. Create your weather targeting rules for your ad set and set it live. Congratulations! Your test has been set up. Now all you have to do it sit back and wait for the results of your test to roll in.

As you can see, the weather targeting will be applied to one of your ad sets but not the other:

Weather targeted variant - you can see the locations are being added and removed based on localized weather.

Control Variant (non-weather targeted) Locations have not been broken down by WeatherAds.

9. Let your split test run and then check the results to see which is the winning variant – the weather targeted variant or the generic variant. You can also run your results through an A/B significance test to determine if they are valid.

NOTE: Whilst an alternative method to run an A/B split test in Facebook is to simply duplicate the ad set, and add weather targeting to one but not the other – you may run into issues (for example internal competition between the 2 ad sets) caused by both of your ad sets targeting exactly the same audience and exactly the same locations. With the A/B split test method described above, Facebook splits your audience into 2 groups, ensuring no overlap, whilst your geo targets remain identical across your 2 variants. This is a fairer method of running a weather targeting split test.