We’re often asked by clients how to evaluate the quality of a global segmentation solution. There are many criteria for assessing whether you have a great segmentation, but none of them are concerned with scoring on a “statistic”.
While it’s possible to build a statistically unsound solution, no summary metric of between-segments differentiation or within-segment homogeneity can substitute for a business strategist’s judgment about the usability of the solution.
To help illustrate what we mean, Material has compiled 9 key criteria for judging a global segmentation.
1. Segments have face validity (i.e., you believe they’re real)
The most basic requirement is that all the segments make sense to you. When you are told that a group of people like this exists, do a gut check. Does that seem true on its face, or warrant further investigation?
When we discuss segments in-depth with our clients, as part of a Data Room process of evaluating and selecting solutions, it’s always a great sign when you hear someone say, “oh, this my brother,” or “I’m this one,” or “we talked to this person at the groups in Topeka”. Likewise, it’s a bad sign when you hear, “I just don’t understand them — how could someone be both this and that at the same time?”
The best segmentations are a healthy mix of “I already knew Segment X exists” and “oh, isn’t Segment Y surprising?!”
2. Segments are clearly distinct from each other without too many nuanced distinctions
You need to be able to quickly and easily articulate what sets each segment apart from other segments that share common traits. If there are two highly attractive targets, how are they going to be differentiated from one another?
It’s better that they be very different on one or two wedge issues, accompanied by many attendant small differences rather than being moderately different on many unrelated things.
While our clients almost always choose a 6-7 segment solution, they invariably want to be able to summarize those groups simply, often in a 2-by-2 matrix.
If the segments are driven by too many different things, you won’t be able to provide that snapshot or tell the elevator story to the CMO.
3. No important segments are obviously missing
If there’s a group you expected to find (particularly one you think would be a strong target) and you don’t find it in your solution, it means one of three things:
- You really don’t understand your market (i.e., the group really doesn’t exist),
- The segment was too small to emerge (i.e., it’s there, but not large enough to form one of the handfuls you requested), or
- The model didn’t leverage the key traits that define that group (i.e., the analyst probably didn’t feed relevant metrics into the model).
If you’ve worked in the industry a while — or you and your research partner have done your due diligence — then it’s unlikely that you guessed wrong about the existence of that group, so you should go back to the data to find out if the group is just too small to warrant pulling out on its own, or whether a new statistical analysis focused on its traits could pull it out for you.
You can avoid time-consuming, model-building iteration if you start the research process by identifying all the hypothetical groups you imagine might be helpful to find. This can guide both the survey development and the statistical analysis.
4. Differences between segments are actionable with respect to core business objectives
There’s an infinite number of ways to divide a market in statistically valid and highly differentiated — but only a few such ways can really help you drive your business forward more effectively than the competition.
All of Material’s segmentation solutions are bespoke because every brand within a category has slightly different business issues and strengths/weaknesses. A great segmentation solution is the one that fits you like a glove.
The segments should be differentiated on things that help you identify economically attractive and capturable segments you want to prioritize your marketing or new product development efforts against while also identifying which segments to divert resources away from.
Furthermore, the target segments should be differentiated in ways that spark ideas for how to serve or speak to them differently. If the segmentation is primarily for branding and communications, you’ll likely want different wedge issues than if it is primarily for product development or service model optimization.
5. No country is dominated by a segment (it’s okay if a segment exists mostly in one country)
If you are building a global framework to align your marketing across regions, then the solution needs to work well in every market; you don’t need to necessarily all agree to target exactly the same set of segments, but you want a common framework so you’re talking the same language and building a globally coherent brand.
If a segment exists almost exclusively in one market, that may be just fine, so long as it reflects unique marketplace dynamics that exist there and isn’t an artifact (see #6).
On the other hand, if a segment is an overwhelming portion of a given market — say more than 40% — then you likely need to sub-divide that segment in that market so the marketing team there still gets the benefit of a segmented market lens. Alternatively, you should look for another solution that better divides all the markets.
6. Segments are driven by attitude differences, not scale usage differences
A market-dominating segment can be a red flag; it’s possible this group is so large because of how the survey questions were asked, using scales that are subject to cross-cultural scale usage biases.
If a segment looms large in a particular market, pressure test whether in-market cultural attitudes suggest the group ought to be so large. If not, you may need to revisit the math driving the model.
If your study is in the US only, don’t rest easy though; this problem does not only occur in global studies. Within every market there are people who use rating scales differently, so there is always the risk of identifying segments based on response styles rather than actual attitudes and needs.
Material designs our segmentation surveys to minimize this risk, relying on bias-free methods like bipolar scales and rankings. We have also been known to rescue a project by re-engineering someone else’s model that was built using bias-susceptible metrics such as agree/disagree scales.
7. Short form typing tool is efficient
Many people who lack the strategic clarity to limit their segmentation inputs to a handful of key dimensions (or who lack the practical experience of dealing with overly nuanced solutions and overly long typing tools), think it’s best to use a statistical approach that throws everything in the pot and stirs. They often wind up with segmentation solutions that are not only muddy (see #2 above), but which also are very hard to find in future research.
It’s unlikely that you have room for a 4-5 minute typing tool in your future qualitative recruits, concept test screeners, or over-stuffed brand tracker. Defining your segments on the basis of too many things will translate into the need for a really long typing tool that,
- Increases your costs for years to come,
- Potentially lowers the quality of your findings (i.e., longer surveys lower the representativeness of samples through non-response bias and high break-off rates),
- Reduces the quality of data by engendering fatigue or speeding.
At Material, we target short form typing tools that take less than 2-3 minutes to administer.
8. Short form typing tool is accurate
Some people try to solve the problem of a lengthy typing tool by simply removing the least predictive items, thereby lowering the accuracy of their assignments. This is a risky game to play.
Many people think a 75% accurate tool is good enough — but do you really want 1-in-4 people who you think belong to your target segment to actually be members of some other segment?
The risk increases if you are doing go/no-go concept testing or costly in-home ethnographic research, so we strongly recommend against using a shortened typing tool for those use cases. We also recommend that you make sure the person is a “core” member of the segment (in other words, they have a very high probability of belonging to that segment).
Also, in the case of the qualitative work, we suggest you re-screen them a second time to ensure they still fall within the bullseye. Since Material culls the segmentation input set down to represent only the key wedge issues driving the solution, you can get a typing tool from us that is both more efficient and more accurate.
9. Segments are targetable in media (at least where relevant infrastructure exists)
For decades the Holy Grail of marketing was to find a way to target attitudes- and needs-based segments in media. The growth of addressable media has now made that possible in many markets.
When you begin your segmentation journey, you should talk with your research partner about how targeting capabilities for in-market activation can be baked into the research program, but don’t fall into the trap of defining your segments entirely (or in part) based on demographics — simply since those are relatively easy to target in media.
With the exception of technology and tech-enabled categories, it is the rare category where age plays as big of a role as you think it does. It is better to have a highly strategic framework that drives your thinking and decision-making, with modest targetability, than to have a more targetable framework that tells you little about how to win in the marketplace.
If you’re reading this article and worrying that your segmentation does not meet all these criteria, let’s talk. Even if all is well, you may be due to refresh it anyway.