Use of the Short Set to Identify People with Disabilities
The WG Short Set have been carefully developed and tested and should be used without any changes to the wording of questions, order of questions, and response categories.
The WG question response categories capture a range of severity in the difficulty experienced. Multiple disability scenarios can be described depending on the domain(s) of interest and the choice of severity cut-off. There is more than one way to capture disability through the application of this set of core questions; resulting in not one but several possible population prevalence estimates that will vary in both size and in composition of the group identified as having a disability.
The WG recommends that the following cut-off be used to define the populations with and without disabilities for the purpose of computing prevalence and differentials in participation for international reporting and cross-national comparability:
- The population with a disability includes everyone with at least one domain that is coded as a lot of difficulty or cannot do it at all.
Countries using the WG short set of questions should not feel restricted to only producing data based on the above cut-off. Data can be presented by individual domains of functioning, and at several levels of severity from very mild (some difficulty) to very severe (unable to do at all). For example, discovering that in rural regions of a country participation restrictions tend to kick in at lower levels of activity limitations than in urban areas could reflect that barriers to participation are more significant in areas with poorer infrastructure.
For a more detailed discussion see the Interpreting Disability research paper.
Importance of not modifying the Short Set
Extensive testing and experience in a variety of contexts has demonstrated that making changes to the questions in an attempt to ‘improve’ them tends to have unforeseen consequences in terms of reducing accuracy. Using the questions as developed also allows for comparability of data across communities and contexts.
A possible exception is if pre-testing highlights aspects of the questions that are not relevant or confusing in particular contexts. In this situation small adaptations, such as removal of reference to the use of hearing aids in contexts where they are not used, can be made. If in any doubt, it is best to contact the Washington Group and ask about the implications of any adaptations. Contact the Washington Group
If additional information on functioning is required, questions should be added to the short set, rather than taking away or changing the set items. For example, programs may want more information on availability and use of assistive devices. In this instance a question similar to that included in the WG Extended Set such as ‘Do you wear glasses?’ can be included. If questions are added they should follow the WG questions instead of preceding them or being interspersed among them, as the order in which questions are asked can influence the responses. Again, readers are encouraged to contact the WG if more information or explanation is needed.
Introductory Sentence to the Short Set
The short introduction to the question set was developed given the census context where questions cover many disparate areas. The introductory sentence ‘The next questions ask about difficulties you may have doing certain activities because of a HEALTH PROBLEM’ was included as a way of transitioning from one section of the questionnaire to another. Programs may choose not to use this introductory sentence, but it is important to not replace this with a sentence that uses the term ‘disability’. The WG Short Set has deliberately been developed to focus attention on functioning and does not use the term ‘disability’ given the many different ways it can be interpreted and the stigma that can be associated with the term.
Programs reporting on data that have been gathered using the WG Short Set need to articulate clearly how the questions were used, the age range of participants and the cut-off points used to determine disability status.
Translation and cognitive testing
An accurate translation that conveys the context or conceptual meaning (rather than a literal translation) of the WG Short Set is crucial. If a particular word does not translate, then try another one that captures the same content, e.g. if ‘step’ is not in the local vocabulary, perhaps ‘stair’ is a better choice. Avoid situations where bilingual enumerators translate the questions at the time of administering the survey, as it means that enumerators will be asking different questions.
The WG has collected translations of the questions in several languages. If a country has appropriately translated the Short Set for use in their national census, use this version. It is desirable for all programs in the country to use the same translation. The use of different translations by organisations within the same country will reduce comparability of data.
Pooling resources and sharing the same translated versions would overcome this concern. A translation protocol exists which should be used in conjunction with cognitive testing. Do not change the wording of the questions unless cognitive testing highlights particular concerns with the current wording.
Training of enumerators
Local partners differ in their understanding of disability, and their capacity to appropriately use the WG Short Set in communities and then analyse and use the data to inform programs. Training enumerators in how to use the Short Set is therefore crucial and would ideally be followed up by ongoing supervision in the field. Key points to emphasise in training include:
- The reasons why WG focuses on ‘functioning’ as opposed to ‘disability’, noting that the questions are not diagnosis, disease or condition-based, but instead are looking at difficulties that anyone might experience.
- The use of screening questions (e.g. ‘Do you have a disability?’) or introductory statements (e.g. ‘The next set of questions are about disability’) will affect responses to the subsequent short set questions. Screening questions or statements should not be used under any circumstances, whether in a census or in a household survey.
- Questions need to be asked exactly as they have been worded. If questions are explained to participants using inappropriate or negative language, this may influence the way participants respond. This includes the response categories. These should be kept as is, and definitely not changed to yes/no responses.
- Enumerators should never skip questions or fill in the answers based merely on their observations (e.g. if they observe that respondent is using a wheelchair): they must ask all the questions to the participant. Many disabilities are “invisible” and will not be apparent to an enumerator.
Protocols for Implementing Test of the WG Short Set:
Overview of Implementation Protocols for Testing the Washington Group Short Set of Questions on Disability: Short Set Implementation Protocol
- Appendix 1: Question batteries and question by question specifications
- Appendix 2: Translation Protocol
- Appendix 3: Cognitive Testing Interview Guide
- Appendix 4: Field test approaches, instruments and instructions
- Appendix 5: Interviewer Instructions
- Appendix 6: Analysis Plan for Pre-testing the WG Short Measurement Set on Disability
If you require further help or additional information regarding using the UN Washington Group questions please get in touch with the Secretariat at firstname.lastname@example.org