A quick guide to survey research

Abstract

Questionnaires are a very useful survey tool that allow large populations to be assessed with relative ease. Despite a widespread perception that surveys are easy to conduct, in order to yield meaningful results, a survey needs extensive planning, time and effort. In this article, we aim to cover the main aspects of designing, implementing and analysing a survey as well as focusing on techniques that would improve response rates.

Full text

5
REVIEW
Ann R Coll Surg Engl 2013; 95: 5–7
doi 10.1308/003588413X13511609956372
A quick guide to survey research
TL Jones1, MAJ Baxter1, V Khanduja2
1University of Cambridge, UK
2Cambridge University Hospitals NHS Foundation Trust, UK
ABSTRACT
Questionnaires are a very useful survey tool that allow large populations to be assessed with relative ease. Despite a widespread
perception that surveys are easy to conduct, in order to yield meaningful results, a survey needs extensive planning, time and
effort. In this article, we aim to cover the main aspects of designing, implementing and analysing a survey as well as focusing
on techniques that would improve response rates.
KEYWORDS
Survey – Questionnaire – Design – Research – Guide
Accepted 5 November 2011
CORRESPONDENCE TO
Vikas Khanduja, Consultant Orthopaedic Surgeon, Addenbrooke’s Hospital, Hills Road, Cambridge CB2 0QQ, UK
T: +44 (0)1223 256 405; F: +44 (0)1223 317 207; E: vk279@cam.ac.uk
Ann R Coll Surg Engl 2013; 95: 5–7
Medical research questionnaires or surveys are vital tools
used to gather information on individual perspectives in
a large cohort. Within the medical realm, there are three
main types of survey: epidemiological surveys, surveys on
attitudes to a health service or intervention and question-
naires assessing knowledge on a particular issue or topic.1
Despite a widespread perception that surveys are easy to
conduct, in order to yield meaningful results, a survey needs
extensive planning, time and effort. In this article, we aim
to cover the main aspects of designing, implementing and
analysing a survey as well as focusing on techniques that
would improve response rates.
Clear research goal
The rst and most important step in designing a survey is to
have a clear idea of what you are looking for. It will always
be tempting to take a blanket approach and ask as many
questions as possible in the hope of getting as much infor-
mation as possible. This type of approach does not work as
asking too many irrelevant or incoherent questions reduces
the response rate2 and therefore reduces the power of the
study. This is especially important when surveying physi-
cians as they often have a lower response rate than the rest
of the population.3 Instead, you must carefully consider the
important data you will be using and work on a ‘need to
know’ rather than a ‘would be nice to know’ model.4
After considering the question you are trying to answer,
deciding whom you are going to ask is the next step. With
small populations, attempting to survey them all is manage-
able but as your population gets bigger, a sample must be
taken. The size of this sample is more important than you
might expect. After lost questionnaires, non-responders and
improper answers are taken into account, this sample must
still be big enough to be representative of the entire popula-
tion. If it is not big enough, the power of your statistics will
drop and you may not get any meaningful answers at all. It
is for this reason that getting a statistician involved in your
study early on is absolutely crucial. Data should not be col-
lected until you know what you are going to do with them.
Directed questions
After settling on your research goal and beginning to design
a questionnaire, the main considerations are the method of
data collection, the survey instrument and the type of ques-
tion you are going to ask. Methods of data collection include
personal interviews, telephone, postal or electronic (Table 1).
Collected data are only useful if they convey information
accurately and consistently about the topic in which you
are interested. This is where a validated survey instrument
comes in to the questionnaire design. Validated instruments
are those that have been extensively tested and are correctly
calibrated to their target. They can therefore be assumed
to be accurate.1 It may be possible to modify a previously
validated instrument but you should seek specialist advice
as this is likely to reduce its power. Examples of validated
models are the Beck Hopelessness Scale5 or the Adden-
brooke’s Cognitive Examination.6
The next step is choosing the type of question you are
going to ask. The questionnaire should be designed to
answer the question you want answered. Each question
should be clear, concise and without bias. Normalising
statements should be included and the language level tar-
geted towards those at the lowest educational level in your
cohort.1 You should avoid open, double barrelled questions
and those questions that include negative items and assign
causality.1 The questions you use may elicit either an open
1540 Khanduja.indd 5 29/11/2012 16:16:55

6Ann R Coll Surg Engl 2013; 95: 5–7
JONES BAXTER KHANDUJA A QUICK GUIDE TO SURVEY RESEARCH
(free text answer) or closed response. Open responses are
more exible but require more time and effort to analyse,
whereas closed responses require more initial input in or-
der to exhaust all possible options but are easier to analyse
and present.
Questionnaire
Two more aspects come into questionnaire design: aes-
thetics and question order. While this is not relevant to
telephone or personal questionnaires, in self-administered
surveys the aesthetics of the questionnaire are crucial. Hav-
ing spent a large amount of time ne-tuning your questions,
presenting them in such a way as to maximise response
rates is pivotal to obtaining good results. Visual elements
to think of include smooth, simple and symmetrical shapes,
soft colours and repetition of visual elements.7
Once you have attracted your subject’s attention and
willingness with a well designed and attractive survey, the
order in which you put your questions is critical. To do this
you should focus on what you need to know; start by placing
easier, important questions at the beginning, group com-
mon themes in the middle and keep questions on demo-
graphics to near the end. The questions should be arrayed
in a logical order, questions on the same topic close together
and with sensible sections if long enough to warrant them.
Introductory and summary questions to mark the start and
end of the survey are also helpful.
Pilot study
Once a completed survey has been compiled, it needs to be
tested. The ideal next step should highlight spelling errors,
ambiguous questions and anything else that impairs com-
pletion of the questionnaire.8 A pilot study, in which you ap-
ply your work to a small sample of your target population in
a controlled setting, may highlight areas in which work still
needs to be done. Where possible, being present while the
pilot is going on will allow a focus group-type atmosphere in
which you can discuss aspects of the survey with those who
are going to be lling it in. This step may seem non-essen-
tial but detecting previously unconsidered difculties needs
to happen as early as possible and it is important to use your
participants’ time wisely as they are unlikely to give it again.
Distribution and collection
While it should be considered quite early on, we will now dis-
cuss routes of survey administration and ways to maximise
results. Questionnaires can be self-administered electroni-
cally or by post, or administered by a researcher by telephone
or in person. The advantages and disadvantages of each
method are summarised in Table 1. Telephone and personal
surveys are very time and resource consuming whereas post-
al and electronic surveys suffer from low response rates and
response bias. Your route should be chosen with care.
Methods for maximising response rates for self-admin-
istered surveys are listed in Table 2, taken from a Cochrane
review.2 The differences between methods of maximising
responses to postal or e-surveys are considerable but com-
mon elements include keeping the questionnaire short and
logical as well as including incentives.
Table 1 Advantages and disadvantages of survey methods
Method of data
collection
Advantages Disadvantages
Personal • Complex questions • Expensive
• Visual aids can be
used
• Time inefficient
• Higher response rates • Training to avoid
bias
Telephone • Allows clarification • No visual aids
• Larger radius than
personal
• Difficult to
develop rapport
• Less expensive or time
consuming
• Higher response rates
Postal • Larger target • Non-response
• Visual aids (although
limited)
• Time for data
compilation
• Lower response rates
Electronic • Larger target • Non-response
• Visual aids • Not all subjects
accessible
• Quick response
• Quick data compilation
• Lower response rates
Table 2 Methods for improving response rates in postal and
electronic questionnaires2
Postal Electronic
Monetary or non-monetary
incentives
Non-monetary incentives
Teaser on the envelope Personalised questionnaires
Pre-notification Include pictures
Follow-up with another copy
included
Not including ‘survey’ in
subject line
Handwritten addresses Male signature
University sponsorship White background
Use recorded delivery Short questionnaire
Include return envelope Offer of results
Avoid sensitive questions Statement that others have
responded
Key points
– Involve a statistician early on.
– Run a pilot study to uncover problems.
– Consider using a validated instrument.
– Only ask what you ‘need to know’.
– Consider guidelines on improving response rates.
1540 Khanduja.indd 6 29/11/2012 16:16:55

7
Ann R Coll Surg Engl 2013; 95: 5–7
JONES BAXTER KHANDUJA A QUICK GUIDE TO SURVEY RESEARCH
Analysis
The collected data will come in a number of forms depend-
ing on the method of collection. Data from telephone or
personal interviews can be directly entered into a compu-
ter database whereas postal data can be entered at a later
stage. Electronic questionnaires can allow responses to
go directly into a computer database. Problems arise from
errors in data entry and when questionnaires are returned
with missing data elds. As mentioned earlier, it is essential
to have a statistician involved from the beginning for help
with data analysis. He or she will have helped to determine
the sample size required to ensure your study has enough
power. The statistician can also suggest tests of signicance
appropriate to your survey, such as Student’s t-test or the
chi-square test.
Conclusions
Survey research is a unique way of gathering information
from a large cohort. Advantages of surveys include having
a large population and therefore a greater statistical pow-
er, the ability to gather large amounts of information and
having the availability of validated models. However, sur-
veys are costly, there is sometimes discrepancy in recall ac-
curacy and the validity of a survey depends on the response
rate. Proper design is vital to enable analysis of results and
pilot studies are critical to this process.
References
1. Alderman AK, Salem B. Survey research. Plast Reconstr Surg 2010; 126:
1,381–1,389.
2. Edwards PJ, Roberts I, Clarke MJ et al. Methods to increase response to postal
and electronic questionnaires. Cochrane Database Syst Rev 2009;
3: MR000008.
3. Martin BC. Don’t Survey Physicians! Chicago, IL: American Medical
Association; 1974.
4. Jones D, Story D, Clavisi O et al. An introductory guide to survey research in
anaesthesia. Anaesth Intensive Care 2006; 34: 245–253.
5. Beck AT, Weissman A, Lester D, Trexler L. The measurement of pessimism: the
hopelessness scale. J Consult Clin Psychol 1974; 42: 861–865.
6. Mioshi E, Dawson K, Mitchell J et al. The Addenbrooke’s Cognitive Examination
Revised (ACE-R): a brief cognitive test battery for dementia screening.
Int J Geriatr Psychiatry 2006; 21: 1,078–1,085.
7. Mahon-Haft TA, Dillman DA. Does visual appeal matter? Effects of web survey
aesthetics on survey quality. Surv Res Methods 2010; 4: 43–59.
8. van Teijlingen ER, Hundley V. The importance of pilot studies. Social Research
Update 2001; 35.
1540 Khanduja.indd 7 29/11/2012 16:16:55