"Signal stacking" for more comprehensive customer insights
TLDR: instead of over-relying on a single learning tool, use a stack of qual + quant + observational insights. (Also, some examples.)
Even if you avoid major sources of bias, you still won’t achieve 100% confidence with your customer understanding with any single approach. This is because no single source of data can ever give full coverage: analytics show one piece of the picture, conversations show another, and so on.
The solution is “signal stacking,” where you use multiple sources of insight for each customer, providing a more comprehensive and trustworthy view of the picture.
It can take a bit of tooling and setup to prepare for gathering this sort of mix of qualitative, quantitative, and observational insights. But once it’s set up, it’s a delight. For example, I build my books as if they were problem-solving products, with the help of a full learning and iteration stack.
So yesterday, I was running a reader conversation for some of the new stuff I’m working on, in which I was both doing classic listening conversation (to build understanding and empathy) as well as some 1-on-1 teaching (since teaching/coaching is the quickest way to testing a book’s underlying education design, acting like a Concierge MVP for the future book.)
Based on what I was hearing from that qualitative layer, I asked for a small commitment of time by suggesting that if he wanted to spend the 30 minutes reading it, I’d be happy to share the first 7k words of the new draft. He said, “Hell yeah” (which I of course ignored, since I’m waiting to see if he actually does it, not if he just says he wants to do it).
But an hour later, my manuscript dashboard was full of feedback, so I knew that he’d actually invested the time. In the screenshot below, you can see the quantitative layer on the left (i.e., both that he read through the whole thing, as well as the reactions along the way) and more qualitative on the right:
If he’d never opened the document (or quit after five minutes), that would have told me something. But in this case, I know that he spent 30 minutes and reached the end. *This stacks up on top of the earlier layers to offer a more complete picture of where this particular customer is coming from.* At this point it’s all pretty standard — any product manager worth their salt will already be using a stack of conversations, analytics, and user testing.
To tally it up, our current signal stack is:
- [Qualitative] Listening and teaching conversation
- [Time commitment] He spent one hour talking with me about it, plus an extra 30 minutes reading about it
- [Quantitative] He read the entire document, identifying 16 specific bits within the manuscript that were more (or less) useful to him
- (With observational insights arriving momentarily.
But what often gets missed is the observational layer about users’ behavior after leaving your product. In many cases, your product is helping them to do something in their broader life (e.g., get healthier, simplify financial reporting, etc.).
I normally do this with follow-up calls after a couple weeks (“Hey, just wanted to check in on how it’s been going for you, and if you ran into any issues putting it into practice—I’d love to hear about it and help if I’m able.”). A customer community makes those follow-ups way easier (since they’re still hanging out with you and you can just grab them after a couple weeks), but it’s still got a relatively high time cost. **
But in this particular case, the customer ended up sending exactly what I wanted straight to my inbox. Not as explicit feedback (biased!), but as evidence of action (unbiased):
quick question - can i send this book draft to some people on our team? we have a large and growing community of therapists and we can make a ton of practical changes to it based on this.
This is so perfect because 1) it’s a behavior, not an opinion and 2) it’s a leading indicator that the current draft is an effective solution to a desirable problem with a strong recommendation loop, which is exactly what I’m looking for as an early indicator while iterating toward a successful book. Similarly, while building our software (i.e., the beta reading tool shown above), we use another signal stack: custdev conversations for baseline understanding, product analytics for quant, individual onboarding by video call for user testing and extra qual, and a customer community for observation and follow-up. As a result, the level of understanding we’re working with is better than any custdev I’ve ever experienced in my life.
(And that community uses its own signal stack to ensure we’re doing a good job there as well. It’s turtles all the way down🐢🐢🐢.)
Everyone has a comfort tool for customer understanding. Some people like conversations and use them for everything. Others use pitches or landing pages. Some over-rely on analytics. But there’s no one right answer. They all work best when you stack them.