You have survey results from customers in three regions and one question: which brand each person prefers. The percentages look different. Could sampling noise produce that gap? The chi-square test of homogeneity answers that question, and it is one of the core tools covered in our guide to statistical significance in survey research.
Key takeaways
- The chi-square test of homogeneity tests whether two or more populations share the same distribution of one categorical variable, such as brand preference across regions.
- It compares observed counts with the counts expected if every group shared one distribution, where expected count is row total times column total divided by grand total.
- Degrees of freedom equal (rows − 1)(columns − 1), and a p-value below your significance level, usually 0.05, means the distributions differ.
- Every expected count should be at least 5, and the test needs raw counts of people, not percentages.
- A significant result shows that groups differ, not by how much or why, so check row percentages and an effect size such as Cramér’s V.
What does the chi-square test of homogeneity test?
It tests whether several populations share the same distribution of one categorical variable. You draw a separate sample from each population, such as 200 customers from each of three regions, and ask everyone the same question. The test checks whether the mix of answers is the same in every region.
It works for any number of groups and answer categories. Two regions and a yes/no question is the smallest case. A tracker with five markets and eight brands is the same test with a bigger table.
What are the hypotheses?
- Null hypothesis (H0): the distribution of the categorical variable is the same in every population. In the example below, brand preference is split the same way in every region.
- Alternative hypothesis (H1): at least one population has a different distribution.
The alternative does not say which group differs or how, only that the groups are not all alike.
How do you calculate it, step by step?
- Build a contingency table. Put the groups in rows and the answer categories in columns. Fill each cell with the observed count (O) and add row totals, column totals and the grand total.
- Calculate the expected count for each cell. If every group had the same distribution, the expected count is
row total × column total ÷ grand total. - Calculate the statistic. Compute
(O − E)^2 / Efor every cell and add them up:sum((O − E)^2 / E). - Find the degrees of freedom. With r rows and c columns,
df = (r − 1)(c − 1). - Get the p-value from the chi-square distribution with those degrees of freedom, or compare the statistic with a critical value. If the p-value is below your significance level (usually 0.05), reject H0.
What does a worked example look like?
The numbers below are illustrative, not from a real study. Suppose you asked 200 customers in each of three regions which of three brands they prefer.
| Region | Brand A | Brand B | Brand C | Row total |
|---|---|---|---|---|
| East | 60 | 90 | 50 | 200 |
| Central | 45 | 70 | 85 | 200 |
| West | 55 | 40 | 105 | 200 |
| Column total | 160 | 200 | 240 | 600 |
Step 1: expected counts
Each expected count is the row total times the column total, divided by 600. Every row total is 200, so every row has the same expected counts:
- Brand A: 200 × 160 ÷ 600 = 53.33
- Brand B: 200 × 200 ÷ 600 = 66.67
- Brand C: 200 × 240 ÷ 600 = 80.00
| Region | Brand A | Brand B | Brand C |
|---|---|---|---|
| East | 53.33 | 66.67 | 80.00 |
| Central | 53.33 | 66.67 | 80.00 |
| West | 53.33 | 66.67 | 80.00 |
All nine expected counts are above 5, so the test is safe to use.
Step 2: each cell’s contribution
| Region | Brand A | Brand B | Brand C |
|---|---|---|---|
| East | (60 − 53.33)^2 / 53.33 = 0.83 | (90 − 66.67)^2 / 66.67 = 8.17 | (50 − 80)^2 / 80 = 11.25 |
| Central | (45 − 53.33)^2 / 53.33 = 1.30 | (70 − 66.67)^2 / 66.67 = 0.17 | (85 − 80)^2 / 80 = 0.31 |
| West | (55 − 53.33)^2 / 53.33 = 0.05 | (40 − 66.67)^2 / 66.67 = 10.67 | (105 − 80)^2 / 80 = 7.81 |
Adding all nine contributions gives a chi-square statistic of 40.56.
Step 3: degrees of freedom and p-value
There are 3 rows and 3 columns, so df = (3 − 1)(3 − 1) = 4. The 5% critical value for 4 degrees of freedom is 9.49. Our statistic of 40.56 is far beyond it, and the p-value is about 0.00000003, which is less than 0.001.
Step 4: the conclusion
Reject H0. Brand preference is not distributed the same way across the three regions. That says the regions differ, not by how much or which region is responsible.
How is homogeneity different from the test of independence?
The arithmetic is identical: expected counts, statistic, degrees of freedom and p-value are computed the same way, and software gives the same output for both. What differs is how the data were collected and what you are asking.
| Test of homogeneity | Test of independence | |
|---|---|---|
| Sampling | Separate samples drawn from each group, with group sizes fixed by design | One sample drawn from one population, then classified on two variables |
| Question | Do the groups share one distribution? | Are the two variables related? |
| H0 | Same distribution in every population | The variables are independent |
A quota sample with fixed numbers per region is a homogeneity design. A single random sample cross-tabulated by region and brand is an independence design. Name the one that matches yours when you report it.
What assumptions does the test make?
- Independent observations. Each respondent appears once. Repeated measures on the same people, such as two waves of one panel, need a different test.
- Adequate expected counts. Every expected count should be at least 5. Some textbooks relax this slightly for large tables, but 5 is the usual rule.
- Representative samples within each group. The test cannot repair bias.
- Mutually exclusive categories. Each respondent falls in exactly one column, so a multi-select question breaks the test.
What are the common mistakes?
- Testing percentages. The test needs counts of people. Percentages throw away the sample size.
- Ignoring small expected counts. If a cell expects fewer than 5, merge related categories (two minor brands into “Other”) or collect more data.
- Reading significance as size. With thousands of respondents, a trivial gap can give a tiny p-value. Check the percentages and an effect size too.
What should you do after a significant result?
A significant result tells you something differs. To find what, work in this order.
- Compare row percentages. East prefers Brand B (45%) and Brand C least (25%). West prefers Brand C (52.5%) and Brand B least (20%). Central is close to the overall pattern.
- Find the cells doing the work. Cells with large contributions, or adjusted residuals beyond about ±2 to ±3, drive the result. Here they are East Brand C (fewer than expected), East Brand B (more), West Brand B (fewer) and West Brand C (more).
- Report an effect size. Cramér’s V is
sqrt(χ² / (n × min(r − 1, c − 1))). Here that is sqrt(40.56 / (600 × 2)), about 0.18, a modest association. - Be careful with follow-up tests. Many pairwise comparisons will throw up some false positives, so adjust for multiple comparisons or say which you planned in advance.
- Do not claim a cause. The test shows regions differ in preference, not why.
For a refresher on what a p-value does and does not mean, read P-values and statistical significance: what you should know. If your outcome is a numeric score rather than a category, you need a comparison of means, and analysis of variance (ANOVA) is the next tool to learn.
How do you do this in practice?
Most survey work starts with a crosstab. Halo Reports produces crosstabs and banner tables with weighting and significance testing, so you can see where groups differ without building the table by hand.
Next time a crosstab shows a gap that catches your eye, run the test before you tell the story. Check that every expected count is at least 5, then report the p-value and an effect size together. If preferences shift between waves, Pulse surfaces meaningful movement in your trackers and studies, with the evidence one question away. For brand preference and consideration over time, market and brand intelligence tracks what changed and why.