The Chi-Squared Test

What is the chi-squared test?

The chi-squared test, often written as χ² test, is a statistical hypothesis test used in the analysis of categorical variables to determine whether observed data are different from expectations. Chi-squared tests are a commonly used nonparametric test, meaning that they do not assume the distributions of the data involved (eg. a Normal distribution). Rather the test relies on the Chi-squared distribution, a theoretical distribution of values for a population.  

Types of chi-squared test

There are two main types of the chi-squared test:

1. A chi-squared goodness of fit test, which is used to test whether the observed frequencies (number of observations in each category) of one variable are different from what was expected. In other words, the test determines if a sample distribution matches a population distribution.
2. A chi-squared test for independence, also referred to as a chi-squared test for association, which compares two variables to see whether they differ from each other.

These types of tests use the chi-square test statistic and distribution, and comparisons between the values you observe with the expected values, for different purposes. They are referred to as Pearson’s chi-squared tests. There are various other tests that use the chi-squared test statistic including the chi-squared test for a trend, the McNemar’s test (for analysis of paired data), the test of a single variance and the likelihood ratio test – all of which are beyond the scope of this article.

Here we will focus on the chi-squared test for independence because of its widespread applications across social sciences, medical statistics and econometrics, where it is used to compare the distribution of a categorical variable in one sample with the distribution of a categorical variable in another sample.

When to use the chi-squared test

The chi-squared test for association is applied to a cross-tabulation of frequencies in the two variables you would like to compare. This is called a contingency table, where each combination of a row and column (eg. number of people in a small household having symptomatic COVID-19) are represented by a cell in the table (see Table 1). It is key that both variables are categorical, meaning that they can take on a limited number of possible values in distinct categories. Examples of categorical variables include ethnicity, presence of a disease (yes/no), and age grouped into bands (for example, 0-5 years, 6-10 years, 11+ years).

You can use the chi-squared test to investigate the association between two variables like these with the following hypotheses:

• The null hypothesis (H0) is that there is no association between the two variables
• The alternative hypothesis (H1) is that there is an association of any kind.

A limitation of the chi-squared test is that it requires a sufficiently large sample size to be valid. As a general rule, when expected values are less than 5, we should turn to an exact probability distribution and use Fisher’s exact test instead.

How to perform a chi-squared test

A health researcher might be conducting a study into transmission dynamics of COVID-19 and wishes to investigate whether household size is associated with whether a person presented with symptomatic disease among those who tested positive for the virus. To help answer this research question they may perform a chi-squared test for independence using the following four steps.

Step 1: Present the null and alternative hypotheses

The first step in performing the chi-squared test is to clearly state the hypotheses. In this example the specific hypotheses are as follows:

• The null hypothesis (H0) is that in the population there is no association between household size and symptomatic infection with COVID-19. In other words, the true difference between the proportions who were symptomatic in the two household size groups is zero (π1 – π2 = 0).
• The alternative hypothesis (H1) is that there is an association between household size and symptomatic infection, and that the true difference between the proportions who were symptomatic in the two household groups is not zero (π1 – π2 ≠ 0).

Step 2: Calculate expected values under the null hypothesis

In this step, we use our contingency table (Table 1) to find for each cell the frequency that would be expected if the null hypothesis were true. To do this we use the row and column totals, called the marginal totals, and derive the expected numbers as:

Therefore, if household size had no association with symptom status, we would expect the same proportion of symptomatic infections in the two household groups:

(73 x 126) / 218 = 42.1 in the small household group expected to have symptomatic COVID-19 and (145 x 126) / 218 = 83.8 in the large household group expected to have symptomatic COVID-19. Table 2 shows the expected values for all four cells in the contingency table. It is important to note that the chi-squared calculations are carried out on the frequency values themselves, not on the proportions.

Table 2: Contingency table showing symptom status for COVID-19 by household size among 218 participants in a study that tested positive for COVID-19. Expected numbers are displayed in bold.

In our example we use a 2x2 contingency table given that our variables of interest both have two categories, but if we would like to compare the distribution of categorical variables with more than two categories the chi-squared test can be easily extended using these same four steps.

Step 3: Calculate the chi-squared test statistic

Now we compare the observed and expected frequencies of the two variables. We need to calculate a test statistic that summarises how much they differ and to what extent any discrepancy is due to random variation. The chi-squared (χ²) test statistic is calculated using the following formula:

In this formula, the ∑ symbol denotes taking the sum of (adding together) the succeeding quantities over all four cells in the contingency table. For our example:

This gives us a test statistic of χ² = 12.51.

Step 4: Calculate the p-value and assess the strength of evidence against the null hypothesis.

The larger the χ² test statistic the greater the differences between the observed and expected values. To test the strength of association we can compare the χ² test statistic to its known distribution under the null hypothesis and calculate the p-value. The null χ² value and p-value can be easily calculated by statistical software in practice and using a reference table if calculating by hand. These values will depend on the degrees of freedom, which is equal to 1 for 2x2 tables and will be larger for contingency tables with more categories. They also depend on the significance level (usually α=0.05).

In our example, the χ² yields a small p-value (p<0.001), which means our interpretation from the chi-squared test is that there is very strong evidence against the null hypothesis. We may conclude that there is evidence for a possible association between larger household size and symptomatic COVID-19 infection. 

Elliot McClenaghan is a research fellow in Epidemiology and Medical Statistics at the London School of Hygiene & Tropical Medicine