Frequently Asked Questions

I treat survey design a bit like journalism—my job is to ask questions without putting words in anyone’s mouth. Bias creeps in subtly: a leading adjective here, an imbalanced scale there. I once audited a survey that asked, “How satisfied are you with our excellent customer service?” That single word—“excellent”—skewed everything. My process starts with stripping questions down to their most neutral form, then pressure-testing them in pilot interviews. I also randomize answer choices where possible and avoid double-barreled questions like “How satisfied are you with price and quality?” because those are rarely the same thing. Another trick I use is reading questions out loud—if they sound like they’re nudging the respondent, they probably are. Ultimately, bias isn’t something you eliminate entirely; it’s something you actively manage and document so your findings remain credible.

Market sizing often starts as an estimate and evolves into a more precise figure. I use a tiered approach—beginning with quick, directional estimates based on secondary data, then refining with primary research where needed. It’s like sketching before painting. I also triangulate multiple sources to validate assumptions. In one project, we combined industry reports, expert interviews, and customer surveys to build a robust estimate. Transparency is key—I always document assumptions and provide confidence ranges so stakeholders understand the level of certainty.

I think of this as a question of perspective versus proximity. In-house teams bring deep institutional knowledge and immediate access to stakeholders, which is invaluable. External partners bring fresh eyes, specialized expertise, and scalability. I’ve seen organizations struggle when they rely too heavily on one or the other. In one case, an internal team was so close to the product that they unintentionally framed questions in a biased way; bringing in an external firm helped reset the approach. Conversely, outsourced teams can miss nuance without strong collaboration. My recommendation is often a hybrid model—keep strategic ownership in-house while leveraging external experts for execution or specialized methods. The key is clarity around roles, expectations, and communication.

Designing good questions is part science, part empathy. I try to step into the respondent’s shoes and ask, “How would I interpret this if I saw it cold?” Clarity is everything—no jargon, no ambiguity. I also focus on making questions actionable. Asking “Do you like our product?” is far less useful than “Which feature influenced your decision most?” I pay close attention to scale design as well; a poorly constructed scale can distort results. In one project, shifting from a 5-point to a 7-point scale revealed meaningful differences we had been missing. I also include options like “not applicable” to avoid forcing artificial answers. Pretesting is where the real magic happens—watching how people interpret questions often uncovers issues I’d never anticipate. Good questions don’t just collect data; they guide better decisions.

Cross-cultural research requires more than translation—it requires interpretation. I work with local experts to ensure questions are culturally relevant and understood as intended. Back-translation helps verify accuracy, but I also pilot studies in each market to catch subtle differences. For example, response styles can vary significantly across cultures, which affects how data should be interpreted. I also adapt examples and references to local contexts. The goal is to ensure that we’re measuring the same concept across different settings, not just using the same words.

I think of reliability and validity as trust and truth. Reliability asks, “Would I get the same result if I ran this again?” Validity asks, “Am I actually measuring what I think I’m measuring?” I’ve encountered surveys that were highly reliable—consistent results every time—but fundamentally invalid because the questions missed the real issue. To guard against that, I lean on validated instruments when possible and pilot everything before launch. I also standardize procedures rigorously—training interviewers, documenting protocols, and minimizing variation in how data is collected. Triangulation is another tool I rely on; if multiple methods point to the same conclusion, my confidence increases. In one healthcare study, we combined patient surveys, clinician interviews, and administrative data to ensure we weren’t just seeing one slice of reality. Reliability and validity aren’t boxes you check—they’re habits you build into every step.

Representativeness is harder than ever in a fragmented digital landscape. I start by defining key demographic and behavioral variables that matter for the study, then use quota sampling to ensure those groups are proportionally represented. I also monitor responses in real time—if I see underrepresentation, I adjust recruitment strategies. Weighting helps correct imbalances, but it’s not a cure-all. In one project, we combined online sampling with targeted outreach to underrepresented communities to improve coverage. I also consider mode effects—some groups respond differently depending on how they’re contacted. Ensuring representativeness is an ongoing process, not a one-time setup.

I approach data ethics with the mindset that I’m borrowing people’s trust, not just their data. That starts with informed consent—being clear about what I’m collecting, why, and how it will be used. I minimize data collection to what’s truly necessary and anonymize wherever possible. In one project involving sensitive financial behaviors, we redesigned the study to avoid collecting personally identifiable information entirely, which increased participation and reduced risk. I also ensure data is securely stored and access is tightly controlled. Compliance with regulations is the baseline, not the goal; ethical research often goes beyond what’s legally required. I regularly ask myself, “Would I be comfortable if my own data were handled this way?” If the answer is no, something needs to change. Ethics isn’t a constraint—it’s what makes the work sustainable and credible.

Speed and quality don’t have to be opposites, but they do require discipline. I start by narrowing the scope—what absolutely needs to be answered now versus later. I rely on prebuilt templates and tools to accelerate setup, and I run processes in parallel whenever possible. In one rapid study, we designed, fielded, and analyzed a survey in under a week by overlapping tasks and using automated data cleaning. That said, I’m careful not to cut corners on fundamentals like sampling and question design. I also communicate clearly about limitations—fast research often comes with trade-offs, and stakeholders need to understand them.

Intangibles like trust and brand value are challenging because they don’t show up directly in spreadsheets, but they absolutely influence behavior. I approach them indirectly—through proxies like customer loyalty, willingness to recommend, and sentiment analysis. I also combine qualitative insights to understand what those metrics actually mean. In one project, we linked customer trust scores to repeat purchase behavior, creating a more tangible measure of impact. It’s not perfect—there’s always some level of interpretation—but by triangulating multiple signals, we can build a credible picture. The key is to be transparent about assumptions and avoid overclaiming precision.

AI is transforming research in the same way spreadsheets once did—it’s speeding up the mechanics so we can focus on thinking. Tasks like transcription, coding open-ended responses, and even initial analysis are now dramatically faster. I’ve used AI tools to process thousands of customer comments in hours instead of weeks. But the real value isn’t just speed; it’s pattern recognition. AI can surface connections that might otherwise go unnoticed. That said, I treat AI as an assistant, not an authority. It can misinterpret nuance or amplify biases in the data. Human judgment is still essential for framing questions, interpreting results, and ensuring ethical use. The firms that succeed will be the ones that integrate AI thoughtfully rather than chasing it as a shortcut.

A/B testing seems simple, but doing it well requires rigor. I always start with a clear hypothesis and ensure proper randomization—without that, the results are questionable. I also make sure the sample size is large enough to detect meaningful differences. One common mistake I see is “peeking” at results too early, which can lead to false conclusions. I also focus on practical significance, not just statistical significance—does the change actually matter in the real world? In one case, a statistically significant improvement had negligible business impact. Good experimentation is as much about discipline as it is about design.

Behavioral tracking can provide powerful insights, but it also raises unique ethical concerns. I prioritize transparency—participants should know exactly what is being tracked and why. Consent must be explicit, especially for passive data collection. I also minimize data collection to what’s necessary and ensure strong anonymization. In one study, we redesigned tracking protocols to reduce intrusiveness while still capturing key behaviors. There’s also a responsibility to consider how the data might be used—especially if it could lead to unintended consequences like discrimination. Ethical tracking is about balancing insight with respect for privacy.

Mixed-methods research is where I get to connect the dots between human stories and hard numbers. I often describe it as using both a microscope and a telescope—you see the fine detail and the big picture. For example, I worked with a nonprofit trying to improve program participation. We began with qualitative interviews and discovered that participants felt intimidated by the onboarding process. That insight shaped a survey we sent to a broader audience, which confirmed the issue at scale and helped quantify its impact. The advantage is not just in using both methods, but in integrating them thoughtfully. I plan those integration points from the beginning—what I want qual to inform, what I want quant to validate, and how the two will speak to each other. When done well, mixed methods produce insights that are both rich and reliable.

If research were cooking, sampling would be your ingredients—no matter how skilled the chef, bad ingredients lead to a bad meal. I’ve seen beautifully designed studies fall apart because the sample didn’t reflect the population they were trying to understand. Early in my career, I worked on a study about “average consumers” that accidentally overrepresented urban professionals; the conclusions were elegant—and completely misleading. Now, I start every project by clearly defining who counts as “in scope” and why. From there, I decide whether I need a probability-based sample for statistical confidence or a more targeted approach for exploratory work. I also pay close attention to nonresponse—who isn’t answering can be just as important as who is. Weighting can help rebalance things, but it’s not a magic fix. In my experience, thoughtful sampling is the difference between insight and illusion.

I see primary and secondary research as original reporting versus archival research. Primary research is when I go directly to the source—conducting surveys, interviews, or observations tailored to a specific question. Secondary research is when I synthesize what’s already been collected—industry reports, academic studies, or public datasets. In practice, I almost always start with secondary research to understand the landscape and avoid reinventing the wheel. For instance, when estimating market size, I’ll pull from existing reports to build a baseline, then use primary research to fill in gaps or validate assumptions. The key is knowing the limitations—secondary data might be outdated or collected for a different purpose. Primary research gives me control and specificity, but at a higher cost. The best approach often blends both, using each where it adds the most value.

I like to think of qualitative and quantitative research as a conversation versus a census. When I sit down with someone in an interview, I’m listening for the “why” behind their behavior—the stories, contradictions, and emotions that don’t fit neatly into a checkbox. That’s qualitative. Quantitative, on the other hand, is where I zoom out and ask, “How many people feel this way?” or “Does this pattern hold at scale?” I once worked with a retail client who thought price was driving churn. Qualitative interviews revealed it was actually confusion about their return policy. We then validated that insight with a survey across thousands of customers. If I start with quant alone, I risk measuring the wrong thing precisely. If I stay in qual, I risk overgeneralizing. The real power comes from knowing when to explore and when to confirm—and often, doing both in sequence.

Open-ended responses are where you often find the most interesting insights, but they can be overwhelming at scale. I use a hybrid approach—AI to quickly cluster and categorize responses, and human analysts to interpret and refine those themes. It’s a bit like using a map and a guide; the map shows you the terrain, but the guide helps you understand it. I also develop a clear coding framework and measure consistency across coders. Pulling representative quotes is important—they bring the data to life and make findings more persuasive. The goal is to balance efficiency with depth.

Executives don’t need more data—they need clarity and direction. When I present findings, I lead with the “so what.” I distill the research into a few key insights and tie each one to a business implication. Visuals help, but only if they simplify rather than overwhelm. I once turned a 60-slide deck into a 10-slide narrative with a clear storyline, and it had far more impact. I also make a point to address uncertainty—what we know, what we don’t, and what assumptions we’re making. Executives appreciate transparency when it’s paired with a path forward. I frame recommendations as choices with trade-offs, not just conclusions. The goal is to move from insight to action as seamlessly as possible.

Online surveys are incredibly useful, but they’re not a universal solution. I avoid them when I need deep, nuanced understanding—people rarely share rich insights through multiple-choice questions. They’re also problematic for hard-to-reach or less digitally engaged populations. I once worked on a study targeting older adults, and an online survey alone would have excluded a significant portion of the audience. We supplemented with phone interviews to get a fuller picture. Another issue is engagement—respondents may rush through or drop off, especially if the survey is long or poorly designed. Online tools are best for structured, scalable data collection, but they work best when paired with other methods.