Choosing the Right Data Collection Method

Insights drawn from research form the foundation of nearly all business decisions, and the quality and reliability of those insights rely on the data gathering methods used. An organization that collects data primarily from qualitative interviews, for instance, could create an effective campaign for its target audience – but if it had conducted quantitative research as well, it might have discovered that the audience wasn’t large enough to warrant the investment.

When it comes to gathering data, there is no one-size-fits-all approach. This guide not only explains what data gathering methods are but also discusses the benefits and drawbacks of common techniques and how businesses can select the best approaches for their needs.

What Are Data Collection Methods?

A data collection method is the structured approach used to gather information for analysis with an eye to solving business problems. Deciding which data collection techniques to use, however, entails much more than settling on which tactic is easiest or cheapest. This is because, in addition to determining how data is sourced, each approach influences what form the data takes and how analysts will interpret it. 

Different methods produce different types of findings. Focus groups and one-on-one interviews yield exploratory insights that help brands better understand a market or situation, so they can form hypotheses for further research. Surveys generate descriptive insights that measure and define existing markets and audiences. And usability testing, A/B tests and other types of experimentation produce causal insights; for instance, whether changing a CTA button from red to blue affects response rates or if promoting a product’s health benefits is more effective than touting its price. 

For these reasons, choosing a data collection method is as much a strategic decision as it is a tactical one. Organizations that choose the wrong approaches might waste resources answering questions irrelevant to their goals or missing out on vital insights that could turn failed campaigns into successful ones. 

Common Methods for Data Collection 

The most commonly used data collection methods are popular for good reason: They work. That said, no single approach can answer every relevant question. That’s why organizations typically use several of the methods below to gain a full suite of insights.

Surveys and questionnaires 

Customer satisfaction surveys and brand awareness questionnaires are just two examples of this method, which enables you to efficiently gather standardized data at scale.  

• Structured surveys with close-ended answers for each question enable easy aggregation and comparison, which is important for identifying patterns across populations. 

• Adding open-ended questions can provide qualitative context to quantitative findings. 

• Strengths include speed and scalability. You can reach hundreds or even thousands of people quickly and affordably via email, text or phone. And because the results are structured, they’re easy to analyze. 

• Limitations include response bias, with respondents sometimes answering how they think they should rather than accurately. In addition, survey fatigue can lower response and completion rates. And while surveys and questionnaires might reveal what respondents think or do, they don’t provide in-depth insights into why.  

Interviews 

Many organizations use one-on-one conversations with carefully selected participants early in the research process to gain an understanding of the context and complexity of how people perceive a product or situation.  

• Interviews can be structured, with a fixed set of questions; unstructured, encouraging open conversation; or somewhere in between. More-structured interviews lend themselves to easier comparisons and analysis, while less structured ones often reveal unexpected issues and viewpoints. 

• One-on-one interviews are best for understanding motivations, perceptions and decision drivers, like why customer churn occurs and how buyers evaluate their purchase options. 

• Strengths include the ability to capture nuances and contextual richness unavailable from quantitative methods. Interviewers can also ask follow-up questions and adapt their inquiries as needed. 

• Limitations include the time and effort necessary to schedule, conduct and analyze the interviews, with skilled facilitation and analysis essential to minimize bias. This makes interviews impractical for large-scale insights. 

Focus groups 

Facilitated conversations of typically 6–10 participants, focus groups provide insights from multiple points of view, making them particularly useful for concept testing and message development.  

• Focus groups enable facilitators to observe group dynamics, hear opinions and see reactions, offering insight into how people talk about categories and products. 

• The group dynamic of focus groups can surface unexpected ideas and perspectives. 

• Strengths include the efficiency of hearing from multiple people in one session and the potential for interaction-driven insights. 

• Limitations include the possibility of groupthink, with some participants being less willing to offer dissenting opinions. Also, due to their limited numbers, focus groups aren’t statistically representative. 

Observation 

There’s truth to the saying, “seeing is believing.” This is why observation techniques like shop-alongs or usability testing with screen recordings are trusted data gathering methods. 

• Observation can occur in real-world or controlled settings. It can also be structured, during which the observer records only certain behaviors; or unstructured, with the observer taking open-ended notes. 

• By revealing what people do rather than what they say they do, this method is particularly critical for UX, workflow analysis and service design. 

• Strengths include behavioral accuracy, free from self-report bias.  

• Limitations include the possibility of bias on the part of the observer’s interpretation of actions, and there may be a lack of insight into the intent or motivation behind the actions. This method is also more expensive and time-intensive than others, making it difficult to scale. 

Experiments and testing 

This controlled approach enables organizations to test cause and effect by altering one variable and analyzing subsequent user responses.  

• A/B tests of creative assets, products, pricing and onboarding flows are common experiments. 

• Because testing encourages optimization and validates decisions, it’s typically applied later in the research and development process. 

• Strengths include causal clarity, with defensible and actionable findings. 

• Limitations include the need to carefully design the test to avoid confounding variables that might lead to false conclusions. Also, the settings in which experiments occur might not reflect real-world conditions where actual behavior takes place; this can skew findings. 

Existing or secondary data 

Sometimes information needed to resolve a problem or answer a question already exists. This is secondary data, which was compiled and gathered, often by another organization, for a different purpose. 

• Sales records, industry reports, government statistics and company filings are examples of secondary data. Sources range from internal systems (like a company’s CRM platform) to media coverage or public records. 

• Secondary data is best used for benchmarking, trend analysis and other types of historical context. Organizations often gather secondary data during the exploratory phase to help determine if an initiative warrants further consideration. Companies also use it to help validate their primary data. 

• Strengths include cost and time savings. Because the data already exists, organizations don’t have to go to the expense of conducting their own surveys, tests and interviews. 

• Limitations include a lack of control over the relevance, structure and quality of the data, as well as a lack of context regarding how it was collected. 

How to Choose the Right Data Collection Method

It can be tempting to opt for the fastest or most inexpensive data collection techniques but failing to align your methods with your goals can result in wasted resources. Selecting the optimal approach entails considering a number of factors.

• Decision impact. This is the first thing to consider: What decision or action will the data inform? Determining whether to expand into a new market requires different data than, say, seeking ways to improve website metrics.

• Types of insight required. Quantitative data centers around numbers and measurements —the who, what, where and when of an issue. Qualitative information explores the why and how of those numbers. Many projects require both.

• Audience access. Who do you need to reach, and how will you do so? Can your existing customers provide the necessary insights, or should you look to non-customers, churned customers, or people from demographics different from those of your current audience?

• Scale vs. depth. The larger the scale of the research, the less in-depth the findings. One-on-one interviews can deliver rich contextual findings that delve deep in motivations, but conducting a statistically significant number of them takes far more resources than emailing a survey to thousands of consumers. Again, many initiatives require both in-depth and broad data.

• Bias and data quality risks. Each approach has its own inherent risks regarding bias and data quality. People who fill out questionnaires might have poor recall; participants in focus groups can succumb to peer pressure; unconscious bias can affect an interviewer’s questions or an analyst’s interpretations. Triangulating methods — using multiple approaches and sources — can mitigate risks, as can building blinding into experiments and working with experienced facilitators and survey designers.

• Time and resources. It’s important to be realistic about budget, time frame and internal capacity.

As you can see, effective research programs don’t just happen. They require intentional design and, more often than not, multiple data gathering methods to ensure reliable and actionable findings.

Common Pitfalls with Data Collection Techniques  

All methods and techniques of data collection can fall prey to errors that erode data quality, undermining the value of subsequent insights. Being aware of the most common mistakes is key to avoiding them. 

• Choosing methods based on convenience rather than goals. It’s much easier to send out surveys than to conduct focus groups, but the latter can yield qualitative insights that a quantitative survey can’t.  

• Relying on a single data source or technique. Triangulating methods and sources strengthens the validity of the data, yields richer insights and diminishes the risk of bias.  

• Ignoring bias introduced by the method itself. Each approach has inherent blind spots, whether it’s self-report bias in surveys or observer effect in experiments. Acknowledging them from the start enables you to take steps, such as carefully structuring questions or using multiple observers, to reduce them. 

• Confusing tools and platforms with methods. Open-source survey platforms let you quickly and easily send out thousands of surveys. However, if those surveys aren’t devised by research specialists and the audience isn’t selected with precision, they aren’t a scientifically valid data collection method. The method is the design, not the tools. 

• Failing to validate, clean or contextualize collected data. Raw data always needs to be cleaned and checked for outliers, inconsistencies and misinterpretation before being analyzed.  

Strengthening Research Design with Material

While practical considerations such as budget and deadlines are important when deciding on data collection methods, they’re far from the only elements to consider. The methodology needs to align with the goals of the insights being sought. The approaches used by customer experience consultants to transform customer relationships, for example, will likely differ from those used by strategists seeking to improve ROAS.  

Material has helped numerous organizations by pairing rigorous research methodology with an emphasis on strategy and results. We’ve designed effective research programs, collected high-quality data and translated insights into scalable, successful growth strategies. Contact us today to explore how the right data gathering methods can improve your organization’s decision-making.