What is Propensity Score Matching?
Propensity score matching (PSM) is a statistical method that helps make fair comparisons between two groups in non-randomised studies by matching individuals who have similar characteristics. It reduces bias from observable differences so you can estimate how an intervention or treatment is associated with an outcome.
When you can’t run a randomised experiment, groups being compared often differ in ways that affect the result (for example, people who opt into a programme may be older or more engaged). Propensity score matching addresses this by: 1) estimating each person’s propensity score — the probability they received the treatment given measured characteristics (age, gender, prior behaviour, etc.); 2) pairing or grouping treated and untreated individuals who have similar propensity scores; and 3) comparing outcomes between those matched groups. The goal is to balance observable covariates between groups so differences in outcomes are less likely to be driven by those measured characteristics. PSM improves the credibility of comparisons in observational data but cannot account for unmeasured factors and requires sufficient overlap between groups.
Usage example
A council introduced translated survey invitations in some neighbourhoods but not others. To estimate whether translations increased response rates without a controlled trial, analysts use PSM: they calculate each household’s probability of receiving a translated invite based on factors like neighbourhood language mix, income, and prior response history, match households with similar propensity scores across the two groups, then compare response rates in the matched sample.
Practical application
PSM matters because many real-world programmes and outreach efforts can’t be randomly assigned. For teams running multilingual consultations, surveys or community programmes, PSM lets you make more defensible statements about whether an intervention (for example, offering a translated survey, running a targeted outreach campaign, or changing consent wording) is associated with better participation or different responses. It reduces bias from observable differences between groups, improves comparability, and can strengthen evidence used for decisions. Remember: results depend on the quality of measured covariates and sufficient sample overlap; PSM doesn’t prove causation when important confounders are unmeasured.
FAQ
What data do I need to use propensity score matching?
You need: (a) a clear ‘treatment’ indicator (who received the intervention), (b) an outcome to compare (response rate, satisfaction score, etc.), and (c) a set of measured covariates that predict both treatment and outcome (demographics, prior behaviour, location). More and better covariates improve matching. If key confounders aren’t measured, PSM can’t remove that bias.
Does propensity score matching prove causation?
No. PSM reduces bias from observed variables and makes comparisons fairer, but it cannot account for unmeasured confounders or hidden selection effects. It strengthens inference in observational settings, but causal claims still require careful caveats or complementary evidence (sensitivity checks, robustness tests, or a randomized design when possible).
How do I know if matching worked?
After matching, you should check balance: compare the distribution of covariates between treated and matched control groups (for example using standardized mean differences). Good balance means the groups look similar on measured variables. Also check common support (there are matches across the range of propensity scores) and consider how many cases were discarded during matching.
Can PSM be used with survey data across languages or communities?
Yes. PSM is commonly used with survey and administrative data to compare groups (for instance, those who received translated materials versus those who didn’t). It helps isolate the association of the intervention with outcomes like response rates or reported satisfaction, provided you have the right covariates (language preference, socioeconomics, prior engagement) and sufficient sample sizes in each group.