What's the Difference Between Multivariate Testing and Behavioral Targeting?

One of my colleagues recently asked me the question, "We've invested this money in behavioral targeting, but don't we already have multivariate testing software"? What's the difference between the two?"

Tough question, but I used the following analogy to explain it.

Let's start with Multivariate Testing. Consider any city block filled with restaurants. During a given night patrons will populate them based on the level of activity and/or personal food preference or even reputation to name a few. The perceived interest from a population is registered through multivariate testing. Each time a customer walks through the door, a vote is registered for one restaurant over another.

Over time, MVT will understand which restaurant should be recommended to a specific group (a segment of visitors) based on the level of activity and or overall traffic generated. Ultimately, MVT compares the results of this targeted group over another sampled group(s) to determine the overall performance lift.

Based on observing a group "over time", MVT will suggest a specific "champion" restaurant (most often selected) over all others.

In truth, MVT collects a number of data points, including page referrer, keyword, browser, time of day, etc, but it cannot use that information to leverage decisions on its own. It does not maintain that level of sophistication. For lack of a better term, it is dumb.

Eventually MVT will require human intervention from the marketer to further take advantage of usage patterns that come to light in the response data, i.e. creating new segments to optimize testing efforts.

Comparing Behavioral Targeting

We've established that multivariate testing looks at a given population of visitors over a specified length of time. Conversely, behavioral targeting tends to look at individual behavior, comparing it to others that may have performed similarly in the past.

Now back to our restaurant analogy. Behavioral targeting looks at this same issue of preference, but from a slightly different perspective. It may not be enough to put you in any restaurant. It may be more important to put you in the restaurant that's right for YOU at that time.

Behavioral targeting attempts to narrow the list of possibilities by taking into account your previous preferences, the number of people in your party, whether or not you are with friends or you are on a date. It may even consider whether or not you are interested in French or Thai or even if you just want appetizers or a full entre.

And yes, it may even test price sensitivity to determine how much you and others that look like you are willing to spend.

You may be inclined to believe none of this matters, but the truth is that it does. Many of these factors are evaluated by behavioral targeting in real time, without human intervention. It is constantly evaluating all of the possibilities in order to display the creative message that generates the most clickthroughs, conversions, or average order value.

The problem with both of these approaches is that they require thought. Dare I say strategy? Marketers have to become better at tailoring messaging and varying use of creative. Sadly, this is the exception rather than the norm.

The secret to online success is that there is no secret. Work is essential if you want to generate remarkable results. Next I'll talk about how both can be exploited and how the two can be synergized.

Be sure to have a look at my diagram (PDF) which outlines fundamental differences between multivariate testing and behavioral targeting.