{"id":101,"date":"2017-04-05T20:34:18","date_gmt":"2017-04-05T10:34:18","guid":{"rendered":"http:\/\/www.torusresearch.com.au\/?p=101"},"modified":"2020-05-20T09:21:19","modified_gmt":"2020-05-19T23:21:19","slug":"sampling-power-sample-size","status":"publish","type":"post","link":"https:\/\/www.torusresearch.com.au\/?p=101","title":{"rendered":"Sampling, power and sample size"},"content":{"rendered":"

Introduction<\/h1>\n

Here we\u2019ll take a brief, but hopefully practical, overview of sampling in the context of designing a research project, and how to select a suitable sample size.<\/p>\n

The participants you include in your study are an important consideration, to ensure that the results of your study can be extended or generalised to a larger group than your actual study participants. This is known as \u2018generalisability\u2019, or \u2018external validity\u2019.<\/p>\n

Sample size is important to make sure that you include enough subjects to have a reasonable chance of detecting a difference between groups, for example difference in response rates to a new treatment compared with an old treatment, if there is one, and that you do not unnecessarily include too many subjects which might be unethical or an unjustifiable use of scarce research funding or resources. Having said that, a larger group is usually very desirable to improve the precision of the findings of your study and enable more detailed analyses to be run.<\/p>\n

Study protocols and ethics applications invariably require a justification for sample size.<\/p>\n

What is a sample?<\/h1>\n

If we want to know the exact \u2018truth\u2019 of something, we can measure every person (or item) in a whole population. This is a \u2018census\u2019 \u2013 \u2018a study that involves the observation of every member of a population\u2019. Measuring an entire population, for example in the periodic Australian census, is expensive and inconvenient, and not usually feasible in the context of medical research.<\/p>\n

Instead we select a much smaller number of subjects who we hope are representative of the population we\u2019d like to study.<\/p>\n

We assume that this reflects the population, and that the results are estimates of the \u2018truth\u2019. Hence, summary statistics such as mean, median or proportion are given with standard deviation, standard error or a confidence interval, to indicate the precision of our estimate.1<\/sup><\/a><\/p>\n

We do a research study, in the biomedical or veterinary context, in order to apply the result to some ‘target population’. For example, if you are a respiratory clinician, you might like to know\u00a0whether dornase alfa administered to\u00a0children and adolescents aged 5 \u2013 18 with cystic fibrosis was effective in improving lung function in this group.<\/p>\n

The target population in this example is \u2018children and adolescents aged 5 \u2013 18 with cystic fibrosis\u2019. It is self-evident that it would not be feasible to test the drug in ALL people with cystic fibrosis aged 5 \u2013 18 in the world; however we would like to apply the results to this group in general. Thus, the research needs to be conducted in a group which represents<\/strong> all people with cystic fibrosis aged 5 \u2013 18 years.<\/p>\n

The research process can be pictured as an idea, followed by development of a structured research question, typically along the PICO guideline of Population, Intervention, Comparator. This is followed by development of specific statistically tested hypothesis or hypotheses. We then go ahead and conduct out study, think about our results and draw our conclusions.<\/p>\n

\"\"
Figure 1. The research cycle from the idea to application.<\/figcaption><\/figure>\n

 <\/p>\n

Hopefully, the conclusions will be applied to our original wider population of interest. This is the essence of the process of translation of research into practice.<\/p>\n

There is a big \u2018step down\u2019 from the target population to the actual study sample. It is very useful to conceptualise this using a diagram like this on \u2013 you can see how important the selection of the study sample is.<\/p>\n

You have a large \u2018target population\u2019, the population to which you wish to apply the conclusions of your study.<\/p>\n

Everitt defines a target population as<\/p>\n

\u2018The collection of individuals, items, measurements, etc. about which is is required to make inferences. Often the population actually sampled differs from the target population, and this may result in misleading conclusions being made\u2019.<\/em> 1<\/sup><\/a><\/p>\n

The sampling frame<\/h1>\n

Then you have a \u2018sampling frame\u2019. This is the group of subjects to which you have access \u2013 essentially a list – from which you hope to select your study sample.<\/p>\n

It could be quite specific, for example, you may have an actual list of all the patients in your clinic. You may have a list for potential control subjects.<\/p>\n

It might be less specific; you may not have a list of patients in your clinic, but plan just to invite everyone who comes to clinic over a specified time period, say 3 months or 6 months.<\/p>\n

It may not be practical to invite every single person in your sampling frame to participate in the study. For example, if you are recruiting over a 3 month time period, those attending over the other nine months of the year will not be invited. Thus, not everyone in the sampling frame is likely to be invited to participate in your study.<\/p>\n

You might have a list of 6,000 people from whom to select your study subjects \u2013 it is not practical to invite all of them to participate in your study.<\/p>\n

So the invited study subjects are a subset of the sampling frame. (See selecting a random sample in later slides).<\/p>\n

The study sample<\/h1>\n

Of those who are invited to participate, not all will agree or consent to being part of your study. So in turn, the initial study sample will be a subset of the invited subjects.<\/p>\n

Finally, of those people who consent to participate in your study, some may prove ineligible for the study, if there is any kind of screening for eligibility, and some are likely to drop out.<\/p>\n

The subjects left at the end of the study, for whom outcome information is available, constitute the final study sample<\/strong>.<\/p>\n

\"\"
Figure 2. Target population to study sample.<\/figcaption><\/figure>\n

Dropout<\/h1>\n

The reason for drop out is important. Participants may drop out of studies because of simply electing not to continue with a study, by moving away, or because of illness or death.<\/p>\n

Any study dropout is undesirable, but drop out because of illness or death must be taken into account in the statistical analysis. For example, in our cystic fibrosis example, the study was conducted over 96 month. If, hypothetically, some had dropped out because of illness (with very poor lung function) or death, especially if there were more dropouts in the active treatment group, those surviving until the end of the study will be those with the best lung function. This might produce a spurious \u2018result\u2019 of improved lung function in the treatment group.<\/p>\n

The importance of the reason for dropout will vary depending on the nature of the research that you are doing, but it is very important to consider the implications of dropout at the study design stage.<\/p>\n

\"\"
Figure 3. Dropout from the initial study sample.<\/figcaption><\/figure>\n

Summary and practical example \u2013 target population to study sample<\/h1>\n

Figure 4 below is a summary of the population to which the study results are intended to be applied, and the flow of patients through a study. Note that essentially all of the \u2018standards of reporting\u2019 such as CONSORT .2<\/sup><\/a>and STROBE,3<\/sup><\/a> by which editors and reviewers are guided, strongly recommend the use of a flowchart in your manuscript. If you are writing a Cochrane Review a flowchart is essential.4<\/sup><\/a><\/p>\n

\"\"
Figure 4. Selection process through a study.<\/figcaption><\/figure>\n

Hypothetically, suppose we want to run a trial of some drug, or an observational study, in people with cystic fibrosis. We want our conclusions to be applicable to all people with cystic fibrosis. We are based in Queensland and have access to patients of the Queensland clinic. Are our potential study subjects representative of all people with CF?<\/p>\n

Consider the flow of subjects through the study. We are restricted by feasibility to the Queensland clinics, probably restricted to the Brisbane clinic. This might give us a sampling frame of 300 people. From this, we consider the inclusion and exclusion criteria, which will render a proportion of subjects ineligible for the study at this step. Additionally, we may have budgetary constraints that limits the number of subjects we can include in the study. We have to \u2018guestimate\u2019 the likely study sample, the potential dropouts to the study, and work backwards to invite a suitable number of participants. Let’s say we invite 80 people to participate.<\/p>\n

Of those invited to participate, not all will consent. Some patients will be found to be ineligible during the patient information step of obtaining informed consent. Here, we have 20 patients who either refuse consent or are ineligible when we initially invite them to participate.<\/p>\n

Additionally, sometimes it is not possible to assess subjects for eligibility before consent, for example if some screening test, like a blood test, needs to be applied. In this hypothetical example, we lost some subjects who are ineligible at this point, and others who may drop out for reasons either unrelated or related to the study. We lose another 15 subjects here.<\/p>\n

Thus our final study sample is 45 people (Figure 5).<\/p>\n

\"\"
Figure 5. Flowchart for a hypothetical study in cystic fibrosis.<\/figcaption><\/figure>\n

What sample size do I need to select?<\/h1>\n

This brings us nicely to the topic of sample size \u2013 \u2018how many subjects do we need in our study\u2019, because the statistical analysis will be performed on the study sample, i.e. those subjects who have actually participated in the study.<\/p>\n

Estimates and measures of uncertainty<\/h1>\n

To expand a little on estimates and confidence intervals:<\/p>\n

Any statistic (mean, median, proportion) calculated from the study sample is only an estimate of the \u2018true\u2019 statistic we would get if we measured every person in the target population<\/p>\n

If we started again and selected a different sample, we would get a slightly different statistic. For this reason, statistics calculated from samples should be reported with a confidence interval. For a 95% confidence interval \u2013 if you repeated your study 100 times, 95 times out of 100 the statistic would lie within the confidence intervals.5-7<\/sup><\/a><\/p>\n

For example for 20 subjects, the mean FEV1<\/sub>\u00a0and 95% confidence interval might be 60% \u2013 80%. As the number of subjects increases the confidence interval decreases. If you measured 40 subjects, the mean FEV1<\/sub>\u00a0might still be 70% or something close to it, but the confidence interval would be narrower, say 65% – 75%.<\/p>\n

\"\"
Figure 6. The confidence interval narrows (estimate becomes more precise) as the number of study subjects increases.<\/figcaption><\/figure>\n

Clustering<\/h1>\n

Valid conclusions assume that the study sample constitutes a random sample. Everitt1<\/sup><\/a> defines a random sample as:<\/p>\n

‘\u2026.a sample of n individuals selected from a population in such a say that each sample of the same size is equally likely’<\/em>. 1<\/sup><\/a><\/p>\n

For example, in the diagram, if the blue circle represents the sampling frame, and the red dots represent subjects in your study, your comparisons assume the first picture, not the second.<\/p>\n

\"\"
Figure 7. Random sample and clustered sample<\/figcaption><\/figure>\n

If there is clustering:<\/p>\n

Firstly, the results will be biased \u2013 because the sample is not representative. You may have selected subjects who are sicker, older, younger, richer, poorer, or different in some other attribute, than the average in the target population. Similar comments apply to any control or reference group.<\/p>\n

OR it may be possible to adjust for clustering in analysis \u2013 you can use techniques like stratifying your study subjects, or more sophisticated techniques which are outside the scope of this article.<\/p>\n

Practical calculation of sample size<\/h1>\n

Calculating sample size is a whole field of study in itself, but in its basic form is easy. For example, in a cross sectional study, you might wish to measure the proportion of mothers at a particular hospital who deliver by caesarean section. How many mothers do you need to assess for mode of delivery to obtain a reasonably accurate or precise estimate?<\/p>\n

You might want to know if term babies born by caesarean section are of \u2018normal\u2019 mean birth-weight. You may already know what a normal birth-weight is for Australian born babies.<\/p>\n

Probably the most common type of sample size calculation is for a comparison of two means. Most sample size calculations come down to two main types:<\/p>\n

Planning your study: sample size questions<\/h1>\n

How many subjects do I need to show a statistically significant effect (if there is one) of my intervention\/risk factor? An alternate way of phrasing this question is \u2018how many subjects to I need to show a clinically significant difference of X between two groups?\u2019 \u2013 in this latter example you need to decide yourself what you regard as clinically important, for example you may decide that an average in birth-weight between two groups of 100g is clinically significant.<\/p>\n

Resources for calculating sample size (or power): online calculators and statistics software<\/h1>\n

There are many on-line calculators for sample size. Two good ones are:<\/p>\n

http:\/\/sampsize.sourceforge.net\/iface\/<\/a><\/p>\n

www.stat.ubc.ca\/~rollin\/stats\/ssize<\/a><\/p>\n

Statistical software<\/p>\n

STATA (www.stata.com)<\/p>\n

SAS (www.sas.com)<\/p>\n

SPSS (www.ibm.com\/software\/au\/analytics\/spss<\/strong>\/products\/statistics<\/strong>\/)<\/p>\n

R (www.r<\/strong>–project<\/strong>.org\/<\/a>)<\/p>\n

Commonly used abbreviations<\/strong><\/p>\n

Statistical notation often uses greek letters. Mean is often represented mu (m) and standard as sigma (s). The Type I error is represented as alpha (\u03b1) and Type II error as beta (b).<\/p>\n

Sample size for prevalence (proportion)<\/h1>\n

You will need to \u2018guess\u2019 the expected prevalence, based perhaps on prior studies or pilot studies, and you need to specify how precise you want your estimates in terms of precision and confidence interval. Typically researchers use 5% precision and 95% confidence interval.<\/p>\n