Is my A/B test done yet?!

Published on
February 3, 2015
Contributors
Kiri Tamte-Horan
kiri@vicimediainc.com

Kiri Tamte-Horan is the Director of Digital Operations at Vici, and oversees the stellar Philadelphia Operations team through the development, implementation, and reporting of all digital campaigns. Kiri has managed hundreds of campaigns spanning Display, Video, and Social platforms, and has generated countless calls and conversions for clients across the country. Kiri is certified in Google Tag Manager and Acquisio, as well as Google AdWords Fundamentals and Display Advertising. Additionally, Kiri has a certification from Disney Institute’s Leadership Excellence. Kiri was the campaign manager for an award winning Digital Campaign as awarded by the Maryland Tourism Council in 2015.

Subscribe to our newsletter
Read about our privacy policy.
Thank you! Your submission has been received!
Oops! Something went wrong while submitting the form.

I can relate to the excitement of getting definitive answers, especially when those answers will directly affect your business. Unfortunately, A/B testing is something that depends on waiting until the right moment. So how do you know when that right moment is

A test stopped too early can yield inaccurate results. These results can range from merely inaccurate to completely erroneous. In an example done by Conversion XL,we are shown a test two days after unveiling



0 percent

According to these results, there was a 0% chance that the variation would beat the control.

Well, let’s reevaluate after ten days of launch and see if the results have changed

95 percent



Now the results are showing that the variation has a 95% chance of beating the control. What a difference eight days can make

Okay, now we know the damages of stopping the test too early, but how do we really know when it’s just right? No doubt about it, this is a tough call, but there are a few rules to go by with A/B testing, according to Neil Patel in The Definitive Guide to Conversion Optimization:

  • It should run for at least 7 days to ensure validity.
  • The test should be conducted until there is a 95% chance of beating the original variable.

However, there is no precise formula for running the perfect A/B test. Context is key when it comes to A/B testing. If the traffic being drawn to your page is higher, it is more likely you will have a shorter time span of the trial than a website that’s bringing a smaller amount of views a day. The larger the audience for your site, the faster relevant results will be delivered.

Related Posts

Tips, guides, industry best practices, and news.
Category Link
Class Dismissed, The Digital Campaign Mastered It!
A local college was looking to enroll new students for the 2023-2024 school year. The campaign highlighted the college’s Artificial Intelligence and Data Science master degree programs.
Vici Team
June 30, 2025
Category Link
YouTube+: The Branding and Awareness Machine
In today’s digital landscape, attention is everything and YouTube is where that attention lives. With billions of users, laser-sharp targeting, flexible budgets, and measurable performance, advertising on YouTube isn’t just a trend it’s a strategic advantage.
Dan Charon
June 25, 2025
Category Link
From Pitch to Performance: Selling Smarter with Digital Media
The best digital salespeople aren’t just selling—they’re guiding. With the right materials, mindset and momentum, your pitch becomes a performance your client won’t forget.
Tracey Geary
June 9, 2025