What is Chi-Square Test?
A chi-square test is a simple statistical test that checks whether two categorical variables are related (for example, language group and answer choice). It tells you if differences you see in a contingency table are likely to be real or due to random chance.
The chi-square test (most commonly the chi-square test of independence) compares observed counts in categories with the counts you would expect if there were no relationship between the variables. You arrange responses into a contingency table (rows and columns of categories), compute a statistic that measures the difference between observed and expected counts, and use that statistic to get a p-value. A small p-value suggests the variables are associated; a large p-value suggests no evidence of association. There is also a chi-square goodness-of-fit test to compare observed counts to a specific expected distribution.
Usage example
You run a survey asking whether parents would attend an evening school meeting (Yes/No) and record the preferred language of the parent (English, Spanish, Somali). Build a 3 x 2 contingency table (language by Yes/No) and run a chi-square test. If the test returns p = 0.01, you conclude there is evidence that attendance intention depends on language; you can then investigate which language groups differ.
Practical application
In Hearo-style surveys, the chi-square test helps you check whether outcomes vary across groups defined by language, location, age bracket or any categorical field. Use it to detect potential participation biases (e.g., underrepresentation of a language group), to see whether responses to a yes/no or multiple-choice question differ by community, and to decide when follow-up outreach or translation changes are needed. Keep in mind its limits: it does not show which specific cells drive a difference, it requires a reasonable sample size in each cell, and it cannot control for multiple variables at once (use regression for that).
FAQ
When should I use a chi-square test?
Use it when you want to test for an association between two categorical variables (for example, language group and response category) and your data are counts of responses in each category. If you have continuous outcomes, or want to control for other variables, consider other tests or models.
What assumptions should I check before using it?
Main assumptions are independent observations and sufficiently large expected counts in each cell (a common rule of thumb: expected count >= 5). If observations are clustered (e.g., multiple responses from the same household) or expected counts are small, results can be invalid.
What if some cells have small counts?
If cells have low expected counts, use Fisher's exact test for small tables, or combine categories, or use a Monte Carlo simulation / exact chi-square approximation. Another option for larger problems is to collect more data.
If the chi-square is significant, how do I know which groups differ?
A significant result only says there is an overall association. To find where differences lie, examine cell residuals, perform pairwise comparisons with appropriate corrections, or compute effect-size measures (like Cramer's V) to judge practical importance.