The National Survey for Wales
A Questionnaire Development & Testing case study
Every year, the Welsh Government gives Welsh people the opportunity to express their views on a wide range of topics (including public services, local area and safety, personal health and wellbeing, leisure activities) through the National Survey for Wales (NSW).
To ensure that the NSW provides a true picture, it's important that as many people as possible take part in the survey. Survey advance materials play a significant role in this process, which is why the Welsh Government asked for advance documents to be desk-reviewed and cognitively tested to ensure they are as compelling as possible.
How we helped
In 2015, NatCen’s Questionnaire Development & Testing (QDT) Hub undertook a thorough review of five existing surveys in Wales, so they can be combined and used as a single comprehensive, and cost-effective survey.
In order to evaluate the materials and questions, NatCen researchers carried out the desk review and two rounds of cognitive testing interviews. Based on the desk review, the problematic questions were recommended for cognitive testing.
Twenty-four face-to-face cognitive interviews were carried out across two rounds of hall testing. The first round of testing was used to identify any major problems with a selection of new questions. The second round of testing was used to test new questions and to retest modifications to questions deemed problematic in round one.
Consequently, QDT experts suggested changes to improve the questionnaire, where necessary. These included: adding or removing certain response options, providing examples and definitions within the text of the questions, revisions of the wording of the questions or response options, improving the interviewer notes.