Suppose that you have just read a review of the literature of the effect of beauty on earnings

You were initially surprised to find a mild effect of beauty even on teaching evaluations at colleges. Intrigued by this effect, you consider explanations as to why more attractive individuals receive higher salaries. One of the possibilities you consider is that beauty may be a marker of performance/productivity. As a result, you set out to test whether or not more attractive individuals receive higher grades (cumulative GPA) at college. You happen to have access to individuals at two highly selective liberal arts colleges nearby. One of these specializes in Economics and Government and incoming students have an average SAT of 2,100; the other is known for its engineering program and has an incoming SAT average of 2,200. Conducting a survey, where you offer students a small incentive to answer a few questions regarding their academic performance, and taking a picture of these individuals, you establish that there is no relationship between grades and beauty. Write a short essay using some of the concepts of internal and external validity to determine if these results are likely to apply to universities in general.
What will be an ideal response?

Answer: Students will consider various points that pose a threat to internal and external validity. Obviously there is a difference in populations (external validity) between highly selective liberal arts colleges and universities in general. SAT scores at these colleges are much higher than for the average university. In addition, the gender composition may be quite different, especially for engineering school, where males dominate in terms of student numbers. Even in economics, the ratio of female to male students is typically 1:2. This is an example of sample selection bias (internal validity). Other potential problems with this study may include errors-in-variables from students not reporting the correct GPA. However, this may not be a severe problem since GPA is the dependent variable. There could be a problem if there are systematic problems in inflating the GPA for lower GPAs. It is also not clear from the setup how beauty was judged. If judges were chosen who are friends of the individuals, then their judgments may be biased, which is more severe since beauty is an explanatory variable. The setup also does not indicate what the control variables are. In the absence of controls, there will be omitted variable bias (internal validity) since intelligence will clearly be a determining factor of cumulative GPAs.