Z-test
A z-test is a statistical test that uses the standard normal distribution to decide whether a sample result differs from a hypothesized value, or whether two proportions differ, when the population standard deviation is known or the sample is large.
How it works
A z-test turns the difference between what was observed and what was expected into a z statistic: how many standard errors apart they are. A z of 1.96 or more in either direction corresponds to a two-sided p-value of 0.05 or less, the common threshold for calling a difference statistically significant.
The one-sample z-test for a mean needs the population standard deviation, which is rarely known outside standardized measures; the more typical case estimates it from the sample, which calls for a t-test. The z-test also covers proportions. With reasonably large samples the normal approximation to the binomial holds, so the share of customers who prefer each of two concepts can be compared with a two-proportion z-test.
z = (x̄ - μ0) / (σ / √n), where x̄ is the sample mean, μ0 the hypothesized mean, σ the population standard deviation and n the sample size
A worked example
A standardized customer-effort score has a population mean of 50 and a standard deviation of 10. A sample of 100 customers who used a new self-service flow averages 52.5. Then z = (52.5 - 50) / (10 / 10) = 2.5, above 1.96, so the difference is significant at the 5 percent level.
Related
Automate this in Pulse
When a Pulse survey asks the same question of two groups, a two-proportion z-test says whether the gap in their answers is larger than chance.