How to Conduct Meaningful Market Research

In this episode of The Consumer Insights Podcast, we speak with Annie Pettit, Chief Research Officer, NA, at E2E Research Services.

The only thing certain is change. From evolving trends to market landscapes, everything is dynamic. The ability to adapt to changing circumstances is a major factor in determining a company's long-term success. And that’s why market research is essential in order to keep up with trends and make more informed decisions.

In this episode of The Consumer Insights Podcast, Thor is joined by Annie Pettit, Chief Research Officer, NA, at E2E Research Services.

They cover:

• What qualifies as an insight
• Practical recommendations for sourcing consumer insights
• Integrating market research insights for better business outcomes

• The advantages of conducting market research

• Common mistakes when conducting market research
• How will different methodologies will impact the future of market research
• Opportunities available to consumer insights professionals
• How consumer insights professionals can challenge the status quo
• Challenges facing the consumer insights industry

If you’re interested in how effective market research can lead to valuable and applicable consumer insights, tune into this episode of The Consumer Insights Podcast.

You can access all episodes of the Consumer Insights Podcast on Apple, Spotify, Google, or use the RSS feed with your favorite player. Below, you'll find a lightly edited transcript of this episode.

Thor Olof Philogène: Hello, everyone, and welcome to The Consumer Insights Podcast. Today, I'm excited to be joined by a market and social research strategist, speaker, and writer, who in her own words: “puts people first and research second”. 

Annie Pettit is the Chief Research Officer for North America at E2E Research Services. Annie's also the chair of the Canadian ISO Standards Committee. Thank you so much for joining me today, Annie.

Annie Pettit: You're very welcome. I'm very happy to be here.

Thor: Annie, for us to kick-off, could you take a couple of minutes to tell us about yourself, your company, and how you got to where you are today?

Annie: Sure. My background is all in psychology. I originally wanted to do psychometry, which is testing: personality testing, intelligence testing, employment testing, that kind of thing. I did start out doing that, and then I realized I'm doing the same statistical tests day after day after day. There is more than Cronbach’s alpha to doing statistics. 

I ended up moving into market research where we got to do literally everything, questionnaires, which are just another form of testing, factor analysis, ANOVA, clustering, everything that I have learned in school, all the statistics were suddenly available to me. 

I switched right into market research, had a great time, and just over the years have evolved into playing with methodology from a marketing research point of view.

Where I am now at E2E Research, we are a full-service research provider offering both research companies and clients either full-service market research where we design the questionnaires, all the way through to writing reports and strategizing and recommendations and that sort of thing. 

For other clients, they look at us for scripting and hosting, so a full range of services, which means I get to understand the full range of where methodology is used in the range of things. I can have an impact on writing better questionnaires, writing better reports, doing better analysis, all sorts of things, so lots of great things to do with E2E Research.

Defining insight

Thor: Thank you for sharing that! And Annie, I had the privilege of meeting you. We talked about this before we got started, pre-COVID. We met pre-COVID. 

Obviously, you've spent a lot of time in this industry and you have such a deep experience, that I would really like to start off with what might sound like a very simple question, but what actually hides an iceberg, which is: how do you define an insight?

Annie: Oh, okay. I've seen, I've read, heard lots of different people describe what they feel an insight is. I think most disagree on what it is not. 

When you get an Excel file and it's got 300 worksheets and it's just reams and reams and reams of tables, this is just data. There is nothing insightful about a set of data, a set of numbers in a report. When you open your PowerPoint report and you see “66% of the people said this, did this”, there's nothing insightful there.

The insightful part comes, from my perspective, when that lightbulb goes off. Perhaps there is a number on the page, 66% of people did this. If it's something that everybody knows, we've seen before, this is normal stuff, that's not insight, that's just data. 

I expect that for any project we do, we're lucky to find one real insight. A real genuine light bulb that has just never occurred to you before. 

When you're reading that and it's like, "Oh, my God, I never thought of that before." It may be basic, may be simple, but if that light bulb just turned on in your brain, that's an insight.

I expect that for any project we do, we're lucky to find one real insight. A real genuine light bulb that has just never occurred to you before. 

Throughout a report there's probably three or four or five neat things, “hadn't thought about it in that way”-s. That seems a little surprising, but just one true, “That just changed how I think about everything, completely brand new to me” - that's how I see an insight, those light bulbs.

Why insights and market research are important

Thor: I really like that definition, Annie, because it really forces you to ask yourself, “Did that really alter the way I see things? Was there a data point that was what I would expect, or was it a data point that was partially interesting and maybe brought something new to the table, or was it a light bulb moment?”

Obviously, that is not something that you will come across on a regular basis easily. It actually is quite tricky to get to one of those. 

Now, I think what's interesting with you is you have this background in psychology, and I think what market research allowed you to do is to effectively allow you to use the full breadth of your knowledge and apply that in making analysis. 

For people that are not necessarily as seasoned in the industry, can you tell us why market and consumer insights are so important? What does conducting, analyzing and integrating research allow businesses to do?

Annie: Sure. Before I answer that, I'll make one comment about psychology. This is one of those cases when a lot of people are coming out from school with their degree in psychology and are qualified for nothing, “I can't do anything with the psychology degree.”

My mind is blown many times. Do you want me to list out all the things that you are now uniquely qualified for with a psychology degree, and market research is right in front? If you want to use a psychology degree, this is the place to go. It's incredibly right on par for what you need to learn, how you need to be skilled.

That aside, why is research important? I look at it like this. Every single person has unique insight into the world. I see things from my perspective, from how I was raised with my siblings, with my parents in my city, at my school, my first jobs, all the things that I did, that's my unique perspective.

We really need to know what other people think. Not just what I think, because my view of what is important or what is true about a brand is absolutely not the same as what you think is important or true about a brand.

From your point of view, it's 100% different. Where you lived, your parents, your school, that's all completely different. I could even look at my immediate sibling, same gender, same school, same parents, same city, 100% completely different perspective. 

We think we can see how other people are thinking and feeling because we're pretty close to a situation, but we absolutely cannot. We are all biased in some specific way.

It's the same for any brand manager. As a brand manager, you are intrinsically close to your brand. You have seen everything, how it was built, how it evolved, how it was conceptualized. You know that brand inside and out, but you don't know my perspective on the brand, my sibling's perspective on the brand, your perspective. Everyone has a completely different perspective on the brand.

This is why we desperately need insights to help us figure out what the perspectives are about a brand. We really need to know what other people think. Not just what I think, because my view of what is important or what is true about a brand is absolutely not the same as what you think is important or true about a brand.

Thor: I think that's really spot on, because it is true that we do think that we have enough to understand from the customer's perspective, but in reality, we are very much limited by our biases. I think that's true for anyone that you work with and for a brand manager of a company that is of course, very well versed in all the details of the brand. That's what they're paid to do, right?

Annie: Yes.

Integrating insights into the business

Thor: Fully agree, but I'm also thinking about the application of insights and consumer insights. Could you maybe, talking about the application and again, leveraging your experience, tell us a bit about someone? 

It doesn't have to be someone you work with, but a case where you've actually highlighted that power of research. Maybe a business example where an insight has been integrated in an interesting way, for instance, to build a better campaign or a project or just an example where that has been applied in a very powerful way.

Annie: Sure. I'll mention two, one very briefly, and then one I love the details about. One thing we've heard for years, decades, I'm not sure how long we've heard, is "Steve Jobs never does market research." Every time I would hear that it just drove me nuts. It's 100% not the case.

Let's say, maybe he didn't do questionnaires, maybe he didn't do focus groups, but market research is vastly more than questionnaires and focus groups. They have reams of transactional databases. All the analytical work around the purchase data, website data, all of that stuff, that's market research. 

Yes, he was doing market research all along to build his business into what it is. Competitive analysis, they were always doing that, too. They were always paying attention to incoming emails and complaints. This is market research, so for a brand that did know market research they did reams of it. Just not what we traditionally would like to call it. 

We really need to open our minds to what market research is and it goes far beyond questionnaires and groups.

We really need to open our minds to what market research is and it goes far beyond questionnaires and groups.

Thor: I'm just going to add something here, because I spoke to Edwin Taborda in our previous episode, who is at Electrolux and he even went as far as saying that Steve Jobs was an insights person. And I think that's super powerful, because I remember being taken by surprise by that, but of course he was. 

He was definitely utilizing that and that's what's made him so successful. He was actually able to build that deep understanding and allow that to shape everything he did with Apple and with the products that they launched. But I didn't mean to interrupt you and cut your flow. That was the first part.

Annie: Yes. It's cool to say "I don't need market research", but you're all doing it whether you think you are or not. The second example I'll mention is a brand that we probably all know and many of us probably love a lot, which is Lego. That company's been around for nearly 100 years making all those tiny little plastic bricks that you can build into whatever you may wish to build it.

As a kid, my brother loved making cars and spaceships. Day after day after day was a new car, a new spaceship, more of this stuff. My partner, the same, spaceships, castles. This, the Lego toy, was loved by boys in so many places around the world.

However, it was barely less than 20 years ago, Lego was trying to increase their profits to compete with companies like Mattel. They just about went bankrupt. They tried making action figures and other toys, but that did not work, and they nearly lost the company.

What they did instead of just innovating, clearly just innovating, making new stuff was not working. They ended up turning to research. They ended up doing things like MRI scans of kids. What were their brains doing while the kids were playing? They realized that kids just love skill-based play. They love the challenge of building something.

This insight led them to making toys of varying complexities. The younger kids had a toy that was suited for the complexity they required, the older kids had complexity for what they required, and the adults had enormous sets with thousands of pieces that cost hundreds of dollars. 

This insight led them to making toys of varying complexities. The younger kids had a toy that was suited for the complexity they required, the older kids had complexity for what they required, and the adults had enormous sets with thousands of pieces and cost hundreds of dollars. 

They're like the Colosseum, a Millennium Falcon, and the Taj Mahal. Now, they figure out a way to get adults interested in paying $1,000,  $800 for a children's toy for themselves.

That was the first insight they came up with through research: complexity suited for the audience. The second part that I love has to do with gender. 

Gender on a term is a whole huge issue. We socialized kids to have a gender in the way that we believe a gender should be. Let's just put that aside and accept that we have socialized a male gender and a female gender and have expectations for what those should be.

What Lego realized is only a very small percentage of girls were playing with Lego toys, even though there's nothing even remotely genderized about the toy, it is just pieces of plastic. They did their research and they tried to figure out what girls want from the toys - instead of just assuming what girls want, let's not just innovate something. Let's actually do the research and figure out what girls want.

At that time, they realized the girls love the vibrant colors, so that's where we see now the pink and purple and bright blue, bright green, bright yellow, all the great colors far more than just black, gray, and yellow before it. 

They also tried to make sure that the things that could be built or things that were of the female stereotype. While my brother was making cars and spaceships, I was making houses, parks, and gardens. So this like "Bang-oh!" hit onto what I was doing as a girl.

They introduced the Friends Lego line, specifically made for girls: colorful, had lots of things to make houses, to make gardens, lots of cute little animals.

Now, when gift-giving season arises, my partner gets the nice Star Wars set, and I get the Friends sets and we're both really happy because the research gave us the Lego that we actually wanted, even as adults.

The consequences of not conducting market research

Thor: That's a fascinating story. I know Lego very well but I did not know how much it had impacted the trajectory of that company.  Because you're right, I remember it being at the brink of bankruptcy. 

It's incredible what they've been able to do since but I think it's also interesting on a personal level. I've got two daughters and both of them are really into the Friends set, and even if I try to build a spaceship every now and then, they're really not interested.

Kids love the challenge in building something, but we all do as an entrepreneur. I love building something too, but the age on the box set is different. I think that's a very good point. 

I think it would be interesting to get your viewpoint, having been in the industry for so long and interacted with clients over all these years.  Because now we've talked about the benefits. We've talked about what Lego was able to do and I think we both have a viewpoint on Apple, that is clear. 

I think we, at least, share the same view here, but what's the probable consequence of not conducting market research? What will happen then?

Annie: Oh, goodness gracious. I would say the one big advantage of conducting market research is not necessarily to create success, find success, but to increase your chances of finding success. You can just go ahead and innovate and make stuff, but that doesn't mean you will be successful. 

Some people will though. I can't remember what the actual statistic is… whatever, 5% of brand new companies are successful, but without market research, you're not going to be among that 5% chance. By conducting research, you will increase that chance of success. 

I would say the one big advantage of conducting market research is not necessarily to create success, find success, but to increase your chances of finding success. You can just go ahead and innovate and make stuff, but that doesn't mean you will be successful. 

You'll have a much greater chance of understanding what the potential drawbacks, weaknesses, worry points are of your business and be able to work around them or avoid them. With research, you'll have a better idea of what your competitors are doing, trying to do, hoping to do. Same thing, you'll be able to either work around that or understand how you are doing better than that.

I think it really is important to respect what market research offers, to take full advantage of it whenever you have the opportunity to do so. I think I talked around your question, but… 

Common mistakes when conducting market research

Thor: No, I think that was a clear perspective, and I think you might also be advising other companies. In guiding them in your job over these years, you must have seen a lot of mistakes also. If you were to highlight one, what is the one mistake you see a lot of businesses making when it comes to conducting market research?

Annie: I'll call this one a pet peeve, I think, stat testing which is very funny because in school, I took every single statistics class I could, every psychometrics measurement design class I could. For me, that was the “funnest” part, if “funnest” is a word.

Now, I've completely reversed trends on that. I 100% percent appreciate the need for statistics. You need to do segmentation, you need to do ANOVAs, you need to do the stats. 

At the end of the day, we sit there, when we look at the p-value like, "Oh, it's 0.06. Well, this was a waste of my time," or "It's 0.04. I knew we had something important here." That p-value I find is the most useless -  I'm going to get in trouble for this- , number out there. I don't care what the p-value is, I can manipulate that p-value into anything I want, just by changing my sample size. It's a pointless number.

If you're looking at a set of data, and your brain says, "I wonder if that's statistically significant," then don't waste your time. If you have to wonder whether it's statistically significant, it's not important enough to pay attention to. 

We really need to spend more time looking at effect sizes. If 30% of people buy this, versus 32% of people buy that, is 30 versus 32 meaningful? I don't care if that's significant or not. I don't care. Would I change my business based on 30 versus 32? That's what matters.

I am at a point where I pretty much never do p-value tests anymore. Statistics, yes, I do that all the time. P-values are pointless. If you're looking at a set of data, and your brain says, "I wonder if that's statistically significant," then don't waste your time. If you have to wonder whether it's statistically significant, it's not important enough to pay attention to. 

Press on, move to where the numbers go, "Holy cow, I cannot believe this is 30 and this is 38." It's the same thing. I don't care if statistically if that p-value is 0.4 or not. If that 30 versus 38 makes me pause, that is far more important. That's when I'm going to pay attention.

How  methodologies will shape the future of market research agencies

Thor: I really like that. It's almost like the test is, do you need to force it? If you're thinking about the statistical significance, that's a warning sign. That's not the right question you should be asking yourself here. I'm with you 100%.

Tell me, there are a lot of different types of methodologies. When you conduct market research and the differences  in these methodologies, how will they shape the role of market research companies in the future in your view?

Annie: Oh, goodness. I feel the analytical tools that we have are going to have a huge impact really on our work. Historically, we really only had questionnaires and focus groups, there was no other way to get information. 

Now, the digital space, oh, my goodness. Every single thing we do everywhere is recorded. Anytime you look at a website, think about what you want to buy, they're not just recording what you added to the basket, the skew, the size, the coupon... 

That's not all they're doing. They're watching your mouse. The mouse went to this part of the page, it went to this part of the page, it dismissed this part. All of that is getting stored somewhere.

The folks who have a background in statistics, background in SQL and SaaS or Python, wherever it is, those analytical people, I think, are going to see the expansion of our industry, hugely. 

So much market research is going to move from our traditional tools into those analytical tools. The way I see it is behavior is the best data you can have. We all know the bias. People will say anything, but it really only comes down to what will they do? "Oh, sure. I would buy that." - “Have you ever bought it?” - “No”.

Thor: It sounds so obvious when you say it, but oh, my God. 

Annie: I really think we're going to see analytical tools really take over the market research space in the future.

How insights professionals can challenge the status quo

Thor: I think that's something we should bring with us, the fact that these analytical tools will play a bigger and bigger role going into the future. 

Again, guiding people, so guiding our listeners, what opportunities do you think that there are for insights professionals? With your perspective on the future, how can they make a true business impact and how can they, what many people want to do, challenge the status quo?

Annie: Right. One of the things I would love to see people do in terms of challenging the status quo comes down to basic things. Look at the research you're doing right now. What's the design you're doing right now and truly ask yourself why am I doing it like this?

For many of us, we have a favorite research design. I will confess I have two, questionnaires and analytics. Whenever I start to ponder a research study, questionnaires it is. I have that as a template and then I have two favorite sample slides and it's 300 and 700. Question that. 

When it comes to that 300, why 300? We choose 300 because it's cheaper, which is a terrible answer. We know that when we choose 300, it's basically impossible to learn anything about marginalized groups of people. 

If you instantly have an answer for here's how I do it, question yourself, why? Why can't I do it differently? How can I do it differently? What am I missing by just doing it the same way every time? It's difficult, it makes your research take longer, but truly it is better in the long run.

We can't learn anything about black indigenous people even though in the US there are something like 40 million black people, but in a typical study of 300, we're lucky to get 30 black people. Then they just get averaged in and dismissed because, well, we couldn't figure out anything about black people. They're just a small group of people in this study so don't pay attention. 

Indigenous people, that's an even smaller group so we can't do anything with that, but they are millions of people. Our status quo has been, “Let's just average it into the rest of people”. 

That's not a good status quo to keep, and something I would love for us to actively change. Not just, "Well, let's get 5 or 10 more people," but I need massively over-sample marginalized people so that we can actually understand who they are, what they need, what they want, instead of just averaging them in overall.

The same thing with any technique we are doing. If you instantly have an answer for here's how I do it, question yourself, why? Why can't I do it differently? How can I do it differently? What am I missing by just doing it the same way every time? It's difficult, it makes your research take longer, but truly it is better in the long run. Absolutely.

The challenges facing insights professionals

Thor: Tell me, Annie, I think that's really good advice, but on the flip side if you think about challenges, what type of challenges do you think that insights professionals but also the wider industry, should expect in the near future?

Annie: One of the challenges we've really seen this in the last couple of years and it always disappoints me is the lack of numeracy, the lack of understanding of research and science in the broader population. 

Many, many people don't understand how science works. They don't understand that data changes, new data arises, conclusions change. They feel like they're being manipulated, or somehow they're not getting the full truth. There's just not sufficient understanding of “science means change”. As we learn and we change.

We've seen this in the polling area as well. Outsiders looking in, they see us doing weighting. Why are you weighting marginalized people? Why are you making Black people count for more? We're not making any person count for more, but we're making sure the data is accurate. There's just not a lot of understanding about sampling and waiting so it makes our industry look bad.

I think if we can do a better job of educating the public about why we do what we do and how our processes work, I think our industry will work out better in the long run.

There are many things that the general population just doesn't really understand about market research. I think we need to do a better job of helping people understand, why am I excluded for research, or why am I screened out of research? 

There are many places along the research journey where we can help people learn, “Here's why you were screened out because other people who look like you have already participated”, simple little things we can do to help.

I think if we can do a better job of educating the public about why we do what we do and how our processes work, I think our industry will work out better in the long run.

Thor: That's interesting. I think it's in some odd way, you can almost tie it to the difficulties we have when we step into this. I mean, the sense that we assume that we understand the consumer, and at the same time, we assume that they understand us, but they don't.

Annie: Right.

Who Annie would love to have lunch with in the world of insights

Thor: It's effectively the other side of the coin, but I think that's a very good point. Hey, Annie, although I really have enjoyed this conversation, we are approaching the end, but I have one last question for you. The world of insight is a big one, but if you had to pick one person, who in the world of insights would you love to have lunch with?

Annie: Okay, there are several people I would love to have lunch with, but I am going to name a person I'm a huge fan of, Whitney Dunlap-Fowler. She is the founder of Insights In Color, and I just love the work she does. 

She is so focused on making sure that marginalized people get their place in the market research space. Whether that's us understanding them as consumers, customers, clients, however, that may be, or from the researcher perspective, making sure that those researchers have more face time in the research world. Definitely, it would be Whitney that I'm gonna have lunch with. 

Summary

Thor: I would love to join you for that lunch. It seems like it's a person that I probably could learn a great deal from also. Annie, this has been such a great and insightful conversation. It's always fantastic to hear from you, somebody that's so passionate. I, at least, have remembered a lot of things from this conversation. 

If I'm just going to play back a couple, I think it's advice for us as insights professionals to really force ourselves to question our own preferred choices in terms of methodology and really challenge the way we approach things. Sure, it has always worked for us, but that's actually a good reason for us to question that.

I also think you reminded me of something that is very easy to forget. That behavior is always the best data that we have. No matter what people tell you they will do, what are they actually doing? I think that's a very, very good thing to be reminded of. 

I know that I've learned a lot from talking to you today and I'm sure our audience has as well. Thank you, thank you, thank you for joining me today and thanks everyone for listening in.

Annie: You're very welcome. I'm glad to join you.