Get a free demo

Discover Your Growth Strategy

See Segmentify in Action

✓ Valid number ✕ Invalid number

Shortly after you submit the form, one of our team will contact you to organise a time for your demo. Thank you!

What is Multivariate Testing and Learn How to Test Your Pages?

Multivariate testing is a technique that benefits approaches of engineering and statistics to choose between different options, or variates as commonly known. Remember the days when you made your toys fight with each other to select the strongest one. Now think that things got elaborated and your toys have removable parts and you try different combinations that have different arms, tails, or feet. This is basically multivariate testing for the winner toy. Marketers apply multivariate experiment to optimize their sales.

Multivariate test vs A/B testing

Over the years A/B testing have become the standard in the market. It is mainly because it is easy to analyze. You have two alternatives. Put them at display and let the visitors decide.

A simple example of A/B testing.
A simple example of A/B testing.

Remember the toy example? A/B testing in that situation would be selecting two complete toys that you think is the strongest. So one of them would be the winner. Back to the digital world…

You are trying to decide between two different headlines for your charity funding page. One saying “Donate now to help save children’s lives” and the other one saying “Show your support for world’s children”.

UNICEF AB Hypothesis Test
UNICEF have done the test

A/B testing would show 50% of the visitors the alternative 1. The other half would see the second alternative. Observing the conversion rates and donation amounts, the one with better results is chosen.

So how does multivariate testing (MVT) change the game?

Assume besides having 2 slogans, you also couldn’t decide where to place it. Or even which picture to put on the background. This is where multivariate testing becomes the solution. Instead of having 2 alternatives, we need 8 (a multiplier of 2 for each case). Multivariate testing creates 8 different pages to show to visitors. Each having 12.5% of the general traffic. By this way 3 different variables can be tested with only one test and come up with optimized combination.

A/B Testing vs Multivariate Test Graphic
It is a trade-off!

What are variable and variates?

Before we continue knowing the difference between variate and variable is crucial. Variables are the properties we are testing. The headline, font, background color, and button display are each separate examples of variables. Variates are the alternatives we show to our customers.These are formed by different combinations of selections in variables.

What are the types of multivariate testing?

There are three types of MVT.

1.Full factorial testing
2.Fractional factorial testing
3.Adaptive multivariate testing
Let’s understand the differences of each elements.

Full factorial testing: This is basically testing every variant you have. This is the most popular technique as it has no assumptions but all-covering numerical data. Biggest drawback is required traffic. As the number of variates increase, more traffic is needed to make meaningful results about different variates. Think of 20 variates each having 5% of the traffic.

Fractional factorial testing: As the name suggest this technique include some of the possible combinations. Most known technique belongs to famous Japanese engineer and statistician Taguchi. His methods enables us to test high number of variables with fairly low numbers of variates yet having enough information about every variable.

This method almost eliminates traffic requirement problem but leans on some some statistical assumptions and errors. The very same methods will show you the variation of parameters used in MVT.
Adaptive multivariate testing: The newest method of MVT uses real-time data to come up with the winner combination.

Why should I use MVT?

  • Best thing about multivariate testing is the precision it has. You will end up with clear views on how to arrange your design. This precision is reached by element-by-element testing of MVT.
  • Except some types of MVT, it contains no assumptions. MVT technique deals with each of the changes, which enables it have full numerical data.
  • Interrelation of different features or objects is hard to catch.Not with MVT. Multivariate testing presents information on how thing go well together.
  • Speak with data not with gut feelings as a marketing professional.

What are the drawbacks of MVT?

  • The biggest problem with MVT is its high traffic requirement. If you have 10 different combinations, each having 10% of the traffic, you need large volumes of traffic to have meaningful and trustable outcomes. So the advise is not to use it once you are new in town and don’t have enough traffic.
  • As this technique is mainly used for correlation between different aspects of a page and aims small but high impact changes, MVT is not an option to begin a webpage. A/B testing will be your welcome gift. Same thing goes for if you think of a whole redesign.
Webpage Design by Multivariate Testing
  • One last drawback is MVT takes time! Fairly long time. In emergency, don’t use MVT.

Dos and Dont’s of Multivariate testing

There are certain golden rules for doing MVT you must not miss.

What not to do?

Don’t change excessive number of variables. Advised gap is between 3 and 5. As you increase the number of changed variables, shared traffic volume diminish by half.

Traffic Volume in MVT
It gets so tiny so quick!

What should you do?

  • As you many combinations, some are guaranteed to be ridiculous.A combination having a banner showing “Free admission” and a headline saying “50%discount” as a gift will contradict. This will confuse the customer and will reduce your credibility. So check every combination for fallacies.
  • Estimating your traffic and timespan you can spare for the test before starting your multivariate test will clear your vision.These will even help you to decide whether to apply multivariate testing. Then, some more like how many variable to change or how many combinations to put on the test.

Now you can try MVT for the sake of deciding better webpages.

Marketing Know-How

Become a better marketer with Segmentify's weekly newsletter!

You may also like

How to Unsubscribe from Web and Mobile Push Notifications

This guide will walk you through unsubscribing from push notifications on different platforms, enabling you to customise your settings and find the ideal balance between staying informed and maintaini...

24 November 2023
How to Write Effective Email Subject Lines (Best Practices)

Do you read every single email you get? If you are not a highly-driven reading addict, it is quite possible that you don’t. There might be many emails you leave unopened or send straight to the bin....

22 November 2023
Unlocking Creativity: 3 Gamification Marketing Examples We’ve Been Loving

When executed with finesse, gamification has the power to elevate the mundane, injecting a dose of excitement into the everyday. In this captivating exploration, we unravel the stories behind three ex...

15 November 2023
10 Essential Email Marketing KPIs You Should Be Tracking

What happens to the email campaigns you send? Do they get lost in the digital abyss?  How do you keep track of them? Or are you keeping track of them? Join us for an adventure exploring the essen...

10 November 2023
Breaking Down Data Silos in eCommerce: Unlocking that Extra 30%

Insights gathered from an SEO campaign are very rarely shared in “real-time” with the email marketing or social selling channels and instead form part of the siloed data for SEO. Similarly, insigh...

10 November 2023
3 Smart Christmas Email Examples to Inspire You

Consumers are expected to spend nearly $900 this Christmas season! For marketers, this really is no surprise. After all, Christmas is the biggest shopping season of the year. “Of the $875 consumers ...

06 November 2023