Suppose a study is conducted in which some schools are randomized to follow a new math curriculum and others are randomized to follow a standard curriculum. Suppose the study includes schools of vastly different sizes. At the end of four years, researchers intend to compare students’ scores on a standardized math test using “Student’s” two sample t-test. Why would this test be an inappropriate choice in this case and what might be some better options?

What will be an ideal response?

Ans: Randomization by clusters induces a correlation between students within the same school in that students from the same school tend to be more similar than students from different schools. The two sample t-test assumes that responses from different students are independent, so using this test would result in p-values that are too small or confidence intervals that are too narrow. Better options include methods that appropriately account for clustering such as linear mixed regression models (or mixed effects regression models) or generalized estimating equations.