TU-L0022 - Statistical Research Methods D, Lecture, 2.11.2021-6.4.2022

The first strategy for making causal claims using quantitative data is a randomized experiment.

The idea of randomized experiment is that we have some population of interest from which we take a sample.

Then we divide the sample into two randomly. One is called the treatment group and another one is the control group.Because we select those two groups randomly there are no differences statistically between these two groups.

Or if there are some differences then it's due to chance only.This relates to back to our example of dividing the men and women-led companies into two groups randomly to see if there's a difference.

We divide these in to two groups with treatment and control. Then we apply some kind of treatment to the treatment group.Typically this example is from medical research so this is applied in medicine and it's easy to understand. One group receives a pill, the other one doesn't. Then after, let's say two days, we assume that the effect takes two days to be realized.

We measure the health of these two groups, we compare if the group that received the pill, the medicine is better than the second group. Then we conclude that there's a causal effect. 

The why this is a valid causal claim is that these groups are perfectly comparable to start with because they are randomly chosen from the same sample.Therefore the only plausible explanation beyond chance for the difference between the groups is that there is an actual effect of the treatment.

This works well under certain conditions. So we need to have a random assignment that's very important. If we have people who get to choose whether they receive the medicine or not, then those people who are more sick will likely to choose to be in the treatment group than the control group. And then comparison here would confound the selection effect of how people chose to be in these groups and the treatment effect.

Then we have a large enough sample. And then some other assumptions that are not as relevant. We have large enough sample that we don't have to worry about chance, we have random assignment here and after that we can compare the difference after receiving the medicine or the treatment as causal effect.

The randomization is important because we want to show that this difference is because of the treatment and not because we chose to assign the groups in a certain way. We want to show that there is the treatment effect instead of a selection effect.

Then we repeat this a couple of times and when the study results have been verified independently or two times, then we can sell our medicine. And that's how randomized experiments work. Of course there are variations to this design like you can compare how health of an individual increases, so that would be a within individual study. This is a between individual study but this is the base case. This is the simplest possible experimental design.

Experimental designs are not always feasible they can be done in business studies but if we study organizations then appliying treatments to organizations could be difficult to organize.

We also have a second best option called Quasi-experiment and the idea of a quasi-experiment is that we have some elements of experimental approach, but we don't have the full experimental control.

For example we could have separate sample pretest and post-test. We have something for example, we know that we have a school and the kids will receive a medicine. Everyone gets the medicine on one day, but we can't influence that.

What we can do is that we randomize the kids, we measure their health for half of the students before the treatment for other half after the treatment. And then we assume that this after the treatment group is otherwise comparable for the before the treatment group except for the treatment.So we assume that there are no time effects. And that would allow us to make a causal claim based on quasi-experimental design.

We can also have experiments where the choice between treatment and control is not random. Either it would look like random we don't have control on the randomization in which case we would assume that these samples behave as if they were random samples. Or we can do some statistical adjustments for this non-random selection. So that's nonequivalent control group design.

Another one is interrupted time series design. So we follow some units or some companies, people over time. Then there is an exogenous shock that happens, so some kind of exogenous event. For example new regulation is implemented in markets independently of these organizations. Then we can analyze what is the effect of that new regulation on company performance. Assuming that the implementation doesn't in any way depend on how these companies are doing.

That's another quasi-experimental design. So the idea of quasi-experimental design is that we have a treatment but we don't have the full randomization. So something happens, something is manipulated but we don't really have quite a full experimental design.

Quasi-experiments are something that people overlook when they think about their designs. There's a great article in Organizational Research Methods about different quasi-experimental designs. I would recommend that you consider these when designing your studies because you can make really strong claims that are perhaps more generalizable than lab experiments, because quasi-experiments typically take place in real-life settings.