How to correctly estimate the sample size in empirical research?

The key stage in the efficient implementation of empirical research is its correct planning. This involves the need to make certain assumptions and take a number of important decisions. Some of them concern various statistical aspects and require the researcher to have a practical knowledge of the basics of statistical inference.

Based on the adopted research objectives, hypotheses or research questions should be formulated that specify the effects of the occurrence or intensity of the phenomena under consideration that are of interest to the researcher, as well as their assumed or hypothetical causes. This allows for the preliminary selection of statistical methods that will enable the assessment of the magnitude of the effects under study and their statistical significance.

To ensure the statistical significance of the effects under study, four elements must be taken into account:

• the level of statistical significance α
• statistical power M
• effect size
• sample size

As for the first two elements, α = 0.05 (sometimes α = 0.01 or α = 0.001) and M = 0.80 (sometimes M = 0.90) are most often assumed.

Effect size (E_s) is related to the practical (research, clinical) evaluation of study results. For example, in a study of patients with sleep disorders, which assessed the effect of a certain sleeping pill (compared to a placebo) on the average length of sleep, the effect size was the difference between the average length of sleep between patients who received the drug and patients who received the placebo.

Determining the three elements mentioned above allows us to estimate the sample size required to demonstrate the statistical significance of the effect of interest to the researcher.

However, researchers often ask how to determine the effect size. Several sources can be indicated here:

• results of previous studies on the same problem, which were conducted by the researcher or published by other researchers in available publications
• results of preliminary studies involving a relatively small number of subjects (units)
• adopt several variants of expected effect sizes that indicate practical significance (research)
• choose one of the standardised measures of effect (e.g. the size of the difference between the means when comparing two groups relative to the standard deviation, i.e. the so-called Cohen’s d measure) and determine the required level of effect size (0.2 or 0.5 or 0.8).

When determining the sample size, it may be helpful to perform several analyses (simulations), e.g., using the capabilities offered by the Statistica program.

The results of the sample size estimation for the aforementioned study of patients with sleep disorders are presented below. The difference in sleep duration of 1 hour and a standard deviation of 1.5 hours were taken as an example effect size.

As can be seen, the estimated number of patients was 37 in each of the compared groups.

When determining the sample size, graphs showing the relationship between the required sample size and the standardized effect size and power may also be helpful.

For a better understanding of the issues briefly presented here and to learn how to estimate sample size in practice, we invite you to attend our training.

Author: Janusz Wątroba, Director of Education and Scientific Cooperation