TU-L0022 - Statistical Research Methods D, Lecture, 2.11.2021-6.4.2022

When you start doing a research study, one of the most important, early questions is whether you should do a qualitative study or a quantitative study. In practice, professional researchers tend to specialize so that they do more qualitative studies or more quantitative studies, or mostly qualitative or mostly quantitative. But more generally, whether you do a qualitative study or a quantitative study, depends on what you're studying. So if there is a person who specializes in quantitative studies, then that person typically specializes also addressing the kinds of questions to which a quantitative approach is better. And similarly, if a person is focusing on qualitative studies, then that guides their choice of research questions.

Why would one want to use a qualitative study or a quantitative study depending on the research question is something that I will address now. Before we go into the choosing between qualitative and quantitative approach, we need to understand how these approaches have been criticized. So many qualitative researchers criticize quantitative research, and many quantitative researchers criticize qualitative research. And then there are researchers who criticize both approaches. The main criticism toward quantitative research is that the numbers are really just numbers. They don't tell us much about the social world. So we don't, based on the numbers, we don't know what is real and what is a person's interpretation. So it's difficult to distinguish between a realist research and interpretive research if you just have numbers. Then the measurement processes, they produce numbers and those numbers might not be as precise as we think. So if you have a regression coefficient of 0.235, then that creates you a false sense of precision when you basically can just say that the effect could be positive. So the numbers give us false sense of precision and accuracy.

For some reason, if we quantify something, people trust us more than if we explain the same finding based on qualitative data. Then the measurement instruments are sometimes not as great as people think that they are. If we ask a person whether the company that they work for is innovative or not, that might be affected by social desirability. For example, it might not accurately reflect the actual level of innovativeness of a company. Then finally, the relationships between two variables presents a static snapshot. So we can only observe correlations or differences between means or differences between numbers. They don't really tell us much about the causal process. So we don't get to observe the actual causal process, we only observe the inputs and the outputs,  but the actual causal process remains a black box for a quantitative researcher.

So qualitative research also has been criticized. And one is that it is too subjective. So you can always come up with a story from a set of interviews, and it's not clear if two people would come up with the same story. So it's entirely possible that from the same data two people would come up with a completely different explanation for the phenomenon. With quantitative research, the procedures are more objective. There is more like a cookbook kind of thing, where you pick your procedures, you apply those procedures. And if two people apply the same procedure, they will end up with the same result. Qualitative research is difficult to replicate. And this relates to the fact that the qualitative data analysis process is subjective. It is very difficult for two people to replicate exact same analysis. The same thing with interviews. If you interview a person, then that person will not tell you the same things from one interview to another in contrast to measuring, for example, the weight of the person which should stay the same over time. So there are lots of things that are undocumented that really can't be documented about social interactions in qualitative research that affect the results. Then there's the problem with generalizability.

If you are studying a single case, you really can't make any strong claims about generalizability. If you study a sample of 100 companies, you are in a much better position to claim generalizability than if you study a single company. And finally, there's the lack of transparency in that, what is actually documented in the methods section and the results section
in qualitative research is not as well established as it is for quantitative research. So researchers make all kinds of decisions, all kinds of inferences during their analysis that don't end up being documented in the actual published paper. So this is a disadvantage of qualitative research.

Now, so both can be criticized but which one should you use based on the research question? This relates to the maturity of the field that you're studying or the maturity of the topic. We have nascent topics, nascent fields where we really don't know much about the topic. And then we have mature fields or mature topics where we already could have a couple of good theories that explain what is going on. When we're focusing on nascent topics and nascent fields, then we need to start with conceptualization.  So we need to understand what kind of concepts we need to make sense of the phenomenon. And then we need to think about how those concepts are related causally, so we need some theory. So in nascent fields or nascent topics, we're focused more on conceptualization and theory building. And for this kind of research, qualitative research is a lot better because it allows you to go and actually see what is going on.

In mature fields, when we have already a few competing theories,  we need to start focusing more on testing on what works, what doesn't work, to what extent different theories explain the phenomenon we're studying. And for this quantitative research that puts numerical values on the strength of different theories is a lot better approach. Also because we already have the concepts we can start building measures. So there are practices that we do theory building with qualitative research, there we have open-ended questions, we have why, we don't really know an answer. And in mature fields we have a pretty good understanding of what the potential answers to our questions could be. So we are, instead of asking why, we are asking, among these alternatives, which of this work, or how well does this theory explain this particular phenomenon?

So the questions in mature fields are much more focused and much more close-ended than in nascent fields. And that steers us toward using quantitative techniques. What kind of problems we encounter if we are using an incorrect research approach or a suboptimal research approach? If we are working on mature fields and we are using qualitative research, then the problem is that we are too easily reinventing the wheel. If there already is like five or seven theories that explain the phenomenon, and we start looking at what explains the phenomenon, we have basically two options. Either we reinvent one of the existing theories maybe with different labels, or we invent a completely new theory. But if we already have seven theories, then what's the point of giving an eighth theory? Instead of coming up with new theory in mature fields, we should be more focused on checking which of those theories actually explain the phenomenon in larger samples. So the problem here is that our qualitative study that produces theory, does not really answer the thing that we want to do in mature field, which is more like focusing on what works instead of producing new explanations.

In contrast, in nascent fields, if we do a quantitative study. So there's a typo in the paper here it should be quantitative, not qualitative. The problems are that if we do a quantitative analysis and we don't really have a strong theory to start with, then how would we construct measures? If we lack good concepts, we can't really measure things. So our measures could be very weak or even completely invalid. Another thing is that if we are focusing on,  if we have a field where we don't have explanations for the phenomenon, and we start looking for statistical associations, then that becomes like a fishing expedition. And it is very likely that we end up reporting correlations for other associations that are either because of chance only or completely spurious.

So the problem with using a quantitative approach in nascent field is that to do quantitative research well, we need concepts and we need theories, and those would not exist in nascent fields. So in practice, we use qualitative research for addressing new questions and then we move toward more quantitative approaches, once we have an existing theory base for a phenomenon. So quite often there is this sequence of first doing qualitative studies to gain initial understanding, and then you apply quantitative studies to see which of those initial ideas actually work. In practice, people tend to focus or specialize either on qualitative research or quantitative research, and that tends to dictate whether they work with nascent questions or mature questions.