Searchandising 2.0 is live! Get the details here.

Get a free demo

Discover Your Growth Strategy

See Segmentify in Action

✓ Valid number ✕ Invalid number

Shortly after you submit the form, one of our team will contact you to organise a time for your demo. Thank you!

What is Multivariate Testing and Learn How to Test Your Pages?

Multivariate testing is a technique that benefits approaches of engineering and statistics to choose between different options, or variates as commonly known. Remember the days when you made your toys fight with each other to select the strongest one. Now think that things got elaborated and your toys have removable parts and you try different combinations that have different arms, tails, or feet. This is basically multivariate testing for the winner toy. Marketers apply multivariate experiment to optimize their sales.

Multivariate test vs A/B testing

Over the years A/B testing have become the standard in the market. It is mainly because it is easy to analyze. You have two alternatives. Put them at display and let the visitors decide.

A simple example of A/B testing.
A simple example of A/B testing.

Remember the toy example? A/B testing in that situation would be selecting two complete toys that you think is the strongest. So one of them would be the winner. Back to the digital world…

You are trying to decide between two different headlines for your charity funding page. One saying “Donate now to help save children’s lives” and the other one saying “Show your support for world’s children”.

UNICEF AB Hypothesis Test
UNICEF have done the test

A/B testing would show 50% of the visitors the alternative 1. The other half would see the second alternative. Observing the conversion rates and donation amounts, the one with better results is chosen.

So how does multivariate testing (MVT) change the game?

Assume besides having 2 slogans, you also couldn’t decide where to place it. Or even which picture to put on the background. This is where multivariate testing becomes the solution. Instead of having 2 alternatives, we need 8 (a multiplier of 2 for each case). Multivariate testing creates 8 different pages to show to visitors. Each having 12.5% of the general traffic. By this way 3 different variables can be tested with only one test and come up with optimized combination.

A/B Testing vs Multivariate Test Graphic
It is a trade-off!

What are variable and variates?

Before we continue knowing the difference between variate and variable is crucial. Variables are the properties we are testing. The headline, font, background color, and button display are each separate examples of variables. Variates are the alternatives we show to our customers.These are formed by different combinations of selections in variables.

What are the types of multivariate testing?

There are three types of MVT.

1.Full factorial testing
2.Fractional factorial testing
3.Adaptive multivariate testing
Let’s understand the differences of each elements.

Full factorial testing: This is basically testing every variant you have. This is the most popular technique as it has no assumptions but all-covering numerical data. Biggest drawback is required traffic. As the number of variates increase, more traffic is needed to make meaningful results about different variates. Think of 20 variates each having 5% of the traffic.

Fractional factorial testing: As the name suggest this technique include some of the possible combinations. Most known technique belongs to famous Japanese engineer and statistician Taguchi. His methods enables us to test high number of variables with fairly low numbers of variates yet having enough information about every variable.

This method almost eliminates traffic requirement problem but leans on some some statistical assumptions and errors. The very same methods will show you the variation of parameters used in MVT.
Adaptive multivariate testing: The newest method of MVT uses real-time data to come up with the winner combination.

Why should I use MVT?

  • Best thing about multivariate testing is the precision it has. You will end up with clear views on how to arrange your design. This precision is reached by element-by-element testing of MVT.
  • Except some types of MVT, it contains no assumptions. MVT technique deals with each of the changes, which enables it have full numerical data.
  • Interrelation of different features or objects is hard to catch.Not with MVT. Multivariate testing presents information on how thing go well together.
  • Speak with data not with gut feelings as a marketing professional.

What are the drawbacks of MVT?

  • The biggest problem with MVT is its high traffic requirement. If you have 10 different combinations, each having 10% of the traffic, you need large volumes of traffic to have meaningful and trustable outcomes. So the advise is not to use it once you are new in town and don’t have enough traffic.
  • As this technique is mainly used for correlation between different aspects of a page and aims small but high impact changes, MVT is not an option to begin a webpage. A/B testing will be your welcome gift. Same thing goes for if you think of a whole redesign.
Webpage Design by Multivariate Testing
  • One last drawback is MVT takes time! Fairly long time. In emergency, don’t use MVT.

Dos and Dont’s of Multivariate testing

There are certain golden rules for doing MVT you must not miss.

What not to do?

Don’t change excessive number of variables. Advised gap is between 3 and 5. As you increase the number of changed variables, shared traffic volume diminish by half.

Traffic Volume in MVT
It gets so tiny so quick!

What should you do?

  • As you many combinations, some are guaranteed to be ridiculous.A combination having a banner showing “Free admission” and a headline saying “50%discount” as a gift will contradict. This will confuse the customer and will reduce your credibility. So check every combination for fallacies.
  • Estimating your traffic and timespan you can spare for the test before starting your multivariate test will clear your vision.These will even help you to decide whether to apply multivariate testing. Then, some more like how many variable to change or how many combinations to put on the test.

Now you can try MVT for the sake of deciding better webpages.

Marketing Know-How

Become a better marketer with Segmentify's weekly newsletter!

You may also like

Say Hello to Powerful Search with Searchandising 2.0!

Customers increasingly rely on internal search to navigate your website. But what happens when search results are irrelevant, slow to load, or fail to personalise the experience? Frustration sets in, ...

25 April 2024
Custom Couture: High-End Lessons in Marketing Personalisation from Luxury Brands

Welcome to Marketing Personalisation 201: Lessons from Luxury Brands! But why are we looking to luxury for lessons in marketing personalisation? Because for these high-end brands, personalisation isn...

25 March 2024
Email Subject Line A/B Testing Best Practices

Ahh, the subject line… The first impression, the battle cry that cuts through the noise and determines whether your message gets heard or deleted in a ruthless swipe… While some might find the art...

15 March 2024
Finding That Sweet Sweet Spot: The Ideal Email Marketing Frequency

Much like courting, email marketing thrives on the delicate balance between rhythm and tempo. Send too many love letters, you overwhelm your lover. Send to few, you appear distant and disinterested. S...

11 March 2024
How LEGO Conquered the Adult Market (And Why You Should Care)

LEGO, long known for its colourful bricks and playful creations, has made a remarkable strategic shift in recent years. The company, traditionally associated with children’s toys, has successfully e...

04 March 2024
Predictive Segmentation: How to Supercharge Customer Engagement with Segmentify

Are you struggling with reaching your target audience? You’re spending hefty chunks of your budget on targeted ads but not getting sales in return? Do you wish for a magic mirror to tell you what yo...

28 February 2024