The Absolute Beginner’s Guide to A/B Testing

In this day and age, uncertainty often is the main element that drives businesses to adopt a series of operational measures as diverse as product differentiation, benchmarking, risk management and competitive intelligence tracking.

The point is to reduce—if not eliminate—potential losses that could arise if you make the wrong decisions.

This is where A/B testing comes in, helping you evaluate different scenarios before making a final decision. If performed properly, A/B testing can help enhance your business’ reputation, attract customers, generate a long-term positive return on investment, and polish your corporate reputation.

First—The Basics
A/B testing is performed exactly as it sounds: evaluating whether Option A is better or worse than Option B. You choose what Option A and Option B are. In other words, you have complete control over the variables, and the same is true for the time period during which you want to run your experiments.

Variables can be anything of operational interest to your business, be it email marketing, website design, seasonal promotion or discount deal packaging. The idea, though, is to pick one variable and stick to it, adopting the same analytical approach for the time period as well.

For example, you can decide to gauge the public-relations effectiveness of an email marketing campaign. Create a newsletter, which will become the control version, and send it to, say, the 500 email addresses in your database.

One week afterwards, do the same with a second newsletter, Option B, and then compare the response rates for both newsletters.

Second—How Do You Perform A/B Testing?
A/B testing initially came from the world of statistics, but you don’t have to be an Einstein or an arithmetic whiz to perform the testing. The important thing is consistency, consistency, consistency.

Whatever variable you decide to assess, make sure it is in sync with your operational goals and that it fits nicely in your analytical approach.

Then, choose the attributes you want to test on the variable. List them, either by operational or chronological order, and set a time frame for testing—which could be a specific time, say, Monday at 2:30 a.m., or during a specific period, such as Monday and Tuesday.

Finally, choose the “test singleton,” which is the variable you want to change when evaluating both options.

You’re thinking this is a bit convoluted, and I agree with you. So, let’s break it down further.

Let’s go back to our newsletter example.

– Variables: Layout, font size, font color, content, product discount of 5%
– Test singleton: Newsletter layout
– Option A: formal, professionally looking layout
– Option B: casual, excitement-prone, “show-biz”-like layout
– Test period: Option A: Friday at midnight; Option B: one week after the first test, at the same time

After running both tests, you may find that Option A had a response rate of 25%, whereas Option B had 18%. Consequently, you can infer that Option A was the most effective newsletter, and it is the one you should adopt in your broader marketing campaign.

Third—What Are the Pros and Cons of A/B Testing?
The positives of A/B testing include convenience, ease of implementation and affordability—no need to possess an advanced degree in Stochastic Theory to perform the testing, and no need to hire someone with that academic pedigree, either.

The main negative is that you can only test one variable at a time in order to get an effective technique.

So it can be costly if you have, say, 10 variables and need to have a full picture of those variables before making a final decision.

One way to mitigate this negative is to use online tools to speed up the process and take the guesswork out of all testing procedures.

The Bottom Line
A/B testing is straightforward and can be performed manually, but don’t hesitate to use online tools if needed. All you need is a list of variables as well as a specific time period and a test singleton to modify in Option A and Option B.

Image Credit: We Liqid