What is statistical power and how can it be increased?

What will be an ideal response?

The concept of statistical power is central to conclusion validity and is related to both Type I and Type II errors—but more directly to Type II error. Statistical power is technically defined as the probability that you will conclude there is a relationship when in fact there is one. In other words, power is the odds of correctly finding the needle in the haystack. Like any probability, statistical power can be described as a number between 0 and 1 . For instance, if we have statistical power of .8, then it means that the odds are 80 out of 100 that we will detect a relationship when it is really there. Or, it's the same as saying that the chances are 80 out of 100 that we will find the needle that's in the haystack. We want statistical power to be as high as possible. Power will be lower in our study if there is either more noise (a bigger haystack), a smaller needle (a weaker signal) or both. So, improving statistical power in your study usually involves important trade-offs and additional costs.

Increase the sample size: One thing you can usually do is collect more information—use a larger sample size. Of course, you have to weigh the gain in power against the time and expense of having more participants or gathering more data.

Increase the level of significance: If you were to increase your risk of making a Type I error—increase the chance that you will find a relationship when it's not there—you would be improving statistical power of your study. In practical terms, you can do that statistically by raising the alpha level or level of significance. For instance, instead of using a .05 significance level, you might use .10 as your cutoff point. However, increasing the level of significance also makes it more likely for a Type I error to occur (which negatively affects conclusion validity).

Increase the effect size:
Because the effect size is a ratio of the signal of the relationship to the noise in the context, there are two broad strategies here. To raise the signal, you can increase the salience of the relationship itself. This is especially true in experimental studies where you are looking at the effects of a program or treatment. If you increase the dosage of the program (for example, increase the hours spent in training or the number of training sessions), it should be easier to see an effect. The other option is to decrease the noise (or, put another way, increase reliability). In general, you can improve reliability by doing a better job of constructing measurement instruments, by increasing the number of questions on a scale, or by reducing situational distractions in the measurement context. When you improve reliability, you reduce noise, which increases your statistical power and improves conclusion validity. Similarly, you can also reduce noise by ensuring good implementation. You accomplish this by training program operators and standardizing the protocols for administering the program and measuring the results.