TU-L0022 - Statistical Research Methods D, Lecture, 2.11.2021-6.4.2022

Factor analysis results can be used for calculation of reliability indices similar to coefficient alpha. The most commonly used one is the composite reliability index, also known as coefficient Omega and the average variance extracted, or AVE. The AVE is not that useful of an index and it has some problems, but it's useful to understand what it quantifies because it's still fairly commonly used.

Let's take an example. The Yli-Renko article has both composite reliability here, and they explain that composite reliability is similar to coefficient alpha and they interpret these results as reliability indices. So the reliability index here quantifies that if we take a sum of the indicators, then the reliability of the sum is 0.7 and this sum is 0.75. So it's interpreted the exact same way as coefficient alpha. Then they have the average variance extracted statistic. This is often interpreted as the average reliability of an individual indicator in the scale. So it doesn't really measure the reliability of the scale as such, but instead it measures the average reliability of each individual indicator.

So the scale reliability increases the reliability of the sum of the indicators, increasing as a function of the number of indicators and composite reliability. They take that into account where average variance extracted does not. And then they claimed that it indicates internal consistency, which these indices do not. So they measured reliability assuming internal consistency but they do not test internal consistency. The assumptions are basically the same as with alpha.

The composite reliability index and average variance extracted index are presented with these kinds of equations. So the composite reliability is basically the variance due to the factor divided by the total variance of the composite. So the total variance of the sum divided by the variation due to the factor. And then the average variance extracted is approximately the average of the estimated indicator reliability of the scale. So it's an approximation - it's not exact because there are some nuances for how it's calculated.

Let's do an example. So let's check what kind of reliability indices we get from our exploratory factor analysis results. So we have the first indicator - first factor here. We can use the equation from the previous slide to calculate indicator reliabilities. The reliability is the square of factor loading. That's an estimate of reliability of an indicator. We divide by 3 because we have three indicators and the average variance extracted for this factor would be 81%.

Then composite reliability -which is more useful - is calculated with this kind of equation. So we calculate how much variance the factor explains, what is the total variance of the data of the composite, and it's a 93%. So we can do these calculations pretty easily by hand based on factor analysis results. So of these, the composite reliability is useful because it tells us something that is not obvious from the factor analysis results. So it tells us what is the reliability of the sum. It takes into account the individual indicator reliabilities and the number of indicators. Doing this kind of calculation by looking at the factor loadings would be a bit difficult.

The average variance extracted on the other hand doesn't really tell us much beyond the factor analysis results. If we want to know what are the reliabilities of each indicator - is there bad indicator somewhere here- we are much better off by looking at the factor loadings than looking at the other variance extracted statistics. So I recommend that you apply the composite reliability, and it's nowadays more recommended than alpha because it has less assumptions. But I don't think that the average variance extracted statistics is that useful even if it's widely used.

So what's the difference between composite reliability and coefficient alpha? The idea of an alpha is that it tells us what is the reliability of the sum of the scale items. The composite reliability tells the exact same information, but instead of taking the indicator correlations, it uses factor analysis results as input for its calculation.

The difference is that you make less assumptions by doing so. So the composite reliability index is also called congeneric index because it doesn't make the Tau equivalence assumption that every indicator is equally reliable. So it allows the indicators to vary in how reliable they are and it is therefore a more general index than Cronbach's alpha. You will need to do a factor analysis anyway to asses uni-dimensionality to use alpham, and therefore you have the input for the composite reliability index regardless. So using the composite reliability as a matter of routine is a better idea than calculating the alpha, which makes the more constraining assumption that all indicators are equally reliable.

So there's a counter-argument to what I just explained. So this argument for the superiority of composite reliability over coefficient alpha has been made in a number of papers, for example Cho and McNeish, make that kind of argument. But there's also the counter-argument, that Peterson's study has found, that alpha and composite reliability tend to be about the same value. So they went through a number of studies that reported both of these two indices and concluded that they give you almost the same results. So why bother choosing because it doesn't really matter.

Well that can be challenged by stating that if the composite reliability and Cronbach's alpha provide different values, that is evidence against the assumptions of alpha. And because composite reliability is more general, then in that kind of scenario, alpha probably wouldn't be reported. So it's possible that these results of Peterson's are just a selection effect. You are better off using composite reliability because under the alphas assumption, it produces the same result. If alphas assumptions don't hold, then composite reliability can still be applied and it provides you less biased estimates.

One key weakness in composite reliability and alpha is that they both rely on uni-dimensionality assumptions. If your scale has multiple dimensions - for example there are cross loadings or you cannot justify the unit-dimensionality assumption for some other reason - then there are other indices that are better than composite reliability. Also, the fact that you calculate composite reliability doesn't provide you any evidence of dimensionality. That you have to check from your factor analysis.