Training for National Statistical Offices
July 13-15 2016: Technical Implementation Meeting in Fiji, for National Statistical Offices in the Pacific
26 delegates from 14 nations attended the Technical Implementation Meeting in Fiji in July 2016. Represented countries included:
- Marshall Islands
- Papua New Guinea
- Solomon Islands
The training provided technical training for the Pacific countries implementing the Washington Group questions in an upcoming national census.
The meeting also began the process of creating a formalised Pacific Disability Statistics Working Group to increase the institutional capacity of the region.
Past Annual Washington Group Meetings
The 16th meeting of the Washington Group on Disability Statistics (WG) will be held 6-9 December 2016 in Pretoria, South Africa.
The formal WG meeting (7-9 December) will be preceded by two pre-meeting sessions:
- a brief orientation for new WG members providing background on the history and the development of the WG’s data collection tools
- a session directed to representatives from national statistical offices (NSOs) during which issues related to the implementation of the WG questions and the analysis of the resulting data will be discussed. NSOs currently planning or embarking on data collection or analyses are encouraged to attend to share experiences or seek more directed guidance.
The main agenda will begin on the 7th of December and will cover:
- The presentation of final guidelines for analyzing data obtained from the Extended Question Set on Functioning (WG ES-F)
- A session led by the workgroup on child functioning focusing on:
- the finalization and launch of the UNICEF/WG Module on Child Functioning (including results of field testing in Samoa and Serbia)
- a review of the progress in the development of the module on inclusive education
- Progress updates by the workgroups dealing with special methodological issues on the following topic areas:
- the development of a disability module for use in labor force surveys
- the development of measures specific to mental health
- the development of the WG Extended Set of Questions on Environmental Factors and Participation (WG ES-E/P)
In addition to updates on other WG related activities, partner organizations will be invited to report on their recent activities related to disability statistics, and we invite country reports on disability measurement. The Steering Committee chair will lead a discussion on strategic issues including planning next steps, and strategies for implementation and dissemination of data collected using the WG questions.
Fifteenth meeting: 27-29 October 2015 in Copenhagen, Denmark
WG-15 was hosted by the Statistics Denmark. The meeting was attended by 72 persons.Objectives for WG-15 were to:
- Provide an update on the analysis of the WG short and extended sets of questions and approve guidelines for analyzing data obtained from the WG extended set on functioning (ES-F)
- Present additional work on the UNICEF/WG Module on Child Functioning and Disability
- Review progress in the development of module on inclusive education
- Present work on the development of disability modules on labor force surveys
- Review progress in developing an extended set of questions on environmental factors and participation
- Present progress in work on measures specific to mental health
- Review recent international activities in disability statistics, including the use of outcome indicators for monitoring the UN Convention on the Rights of Persons with Disabilities (CRPD) and attainment of the Post 2015 Sustainable Development Goals (SDGs)
- Approve work plan
- 32 representing national statistical authorities from 14 countries (Albania, Australia, Austria, Botswana, Brazil, Canada, China – Hong Kong SAR, Denmark-7, Hungary-2, Ireland, Italy-2, Kenya, Kosovo, Latvia, Mexico, Mongolia, Oman, Palestine, Romania, South Africa, Spain, Sweden, Uganda-2);
- 15 representatives from universities or national institutes of public health or other national research bodies or ministries (Brazil-2, Denmark,-3 Finland, France, Ireland, Morocco, South Africa, Turkey-2, United Kingdom-3);
- 4 representatives from the US National Center for Health Statistics (WG Secretariat);
- 13 representative from an organization representing persons with disabilities (Danish Institute for Human Rights-2, Disabled People’s Organization Denmark-3, Handicap International-2, Higher Council for Affairs of Persons with Disability (Jordan), International Disability Alliance, National Rehabilitation Center for Persons with Disabilities (Japan), Sightsavers-2, Swedish Institute for Participation); and
- 8 representatives from international organizations (DFAT, DFID, Eurostat, ILO, JICA, UN DESA/UN Secretariat of the CRPD, WHO-FIC Collaborating Center – Netherlands)
WG Short Set and Extended Set on Functioning
The extended set of disability questions on functioning (ES-F) were added to the US National Health Interview Survey (NHIS) beginning in 2010. During the past year, analyses of individual domains were finalized using data from the 2010 and 2013 NHIS. A review of the algorithms developed for combining multiple domain questions into single domain indicators of disability and the standards for determination of cut-points were presented. The presentation was followed by a discussion of the proposed algorithms and cut-points for each domain. The delegates agreed on the completed domains and identified certain domains that required some slight modifications or further analysis. A small workgroup will finalize the analysis for the outstanding domains. All analyses will be compiled and presented in a document describing the properties of individual domains of functioning. The document will also include examples of various cut-points that can be used to estimate disability prevalence rates and provide a recommendation for the cut-point to use for internationally comparable reporting.
Child Functioning (formerly: Child Functioning and Disability)
The workgroup on the development of specific question modules designed to measure functioning and disability among children presented the work accomplished in the previous year as part of its collaboration with UNICEF. A review of past and recent work on the development of the module was presented. The Module on Child Functioning was included as part of a Demographic and Health Survey in Samoa. Findings from preliminary analyses of the Samoan data were presented. Further analysis of the Samoan data and data collected from additional field testing of the module will be used to inform the development of guidelines for producing statistics on children with disabilities. The workgroup has also been working with UNICEF to develop a user’s manual to accompany the Module on Child Functioning. The manual will provide background information on the development of the module, as well as technical information that will be useful for the module’s implementation. Both the manual and the guidelines for analysis are expected to be finalized in 2016. The session also included a presentation by a representative from the London School of Hygiene and Tropical Medicine showing results using a pre-final version of the Module on Child Functioning in Cameroon, India and Fiji.
The WG has also collaborated with UNICEF on the development of a module designed to measure facilitators and barriers to school participation. An update on the work accomplished in the past year, including the presentation of sample questions from the current version of the module were presented. Cognitive testing of the UNICEF/WG Module on Inclusive Education was carried out in the United States by the Question Design Research Laboratory (QDRL) at the National Center for Health Statistics (NCHS) in 2015. Results from the cognitive tests will be used to inform revisions to the module. Additional cognitive testing and field testing of the revised module are scheduled to take place in 2016. The final module is expected to be ready by the next WG meeting.
Labor Force Surveys
The WG has recently began a collaboration with the International Labor Organization (ILO) on the development of a disability module intended for use in labor force surveys. A representative from ILO provided a presentation describing the current practices used for collecting information on disability in labor force surveys and explaining the need for the development of a module to collect information on the barriers people with disabilities face in the labor market. Another presentation included a review of the development of a proposed module and provided examples of some of the questions that have been drafted. The module will include sections on barriers to participation in the labor force, workplace accommodations, social attitudes and social protection. The QDRL at NCHS is scheduled to conduct cognitive testing of the module in the United States in 2016. Revisions will be made based on the cognitive test results; followed by cognitive and field testing in additional countries.
The workgroup investigating the development of measures specific to mental health, currently led by the representative from South Africa, presented the work accomplished in the previous year. The presentation included a review of the some of the issues related to developing measures specific to mental health including: a brief description of the global impact of mental, neurological and substance abuse disorders; definitions of two general classifications of mental disorders (severe and common); and citing some examples of existing measures used to collected data on mental health. The presentation also included review of the proposed next steps for the workgroup. A systematic review of existing questions on activity limitations, participation restrictions and environmental barriers commonly associated with common and severe mental disorders is planned. The workgroup will also examine the existing WG questions to determine the extent to which they address the measurement of activity limitations and participation restrictions commonly associated with severe and common mental disorders. A status update will be provided at the next WG meeting. The workgroup has been operating with contributions from a small number of members and the representative from South Africa serving as the interim chair. At the conclusion of the session, a call was made for the participation of additional members and a volunteer to co-chair the workgroup. A representative from the Italian National Institute of Statistics (ISTAT) agreed to co-chair the workgroup along with the representation from South Africa.
Environmental Factors and Participation
Following the ninth WG meeting in Dar es Salaam, a workgroup was formed to look more closely at the development of a set of questions on environmental factors as they relate to the measurement of disability. An evaluation of the available approaches and questions were presented at subsequent meetings. Work on the development of questions on environmental factors and participation was revisited at the meeting in Copenhagen. A presentation was provided by a representative from the WG Secretariat on behalf of the workgroup chair. The presentation included a review of the purpose for developing the question set as well as an overview of two approaches that can be used to capture information about the environment: 1) directly – measuring the environment independently of the person; and 2) indirectly – measuring the environment through the person’s participation in selected activities. The major challenges associated with each approach and sample questions were also provided. The presentation was followed by a discussion of the next steps for the workgroup. The delegates in attendance were asked to provide feedback on the approach the workgroup should follow, the specific-activity domains to be covered and the types of questions that should be included. The delegates agreed that the workgroup should follow the strategies adopted by the workgroups developing questions on inclusive education and labor force participation. It was agreed to proceed with measuring the environment indirectly through the person’s participation in selected activities. The workgroup was asked to write up the proposal and draft a set of questions for one domain to illustrate the approach for discussion at the next meeting.
Frameworks for Indicators to Address Monitoring Disparities by Disability Status
WG-15 also included a session focusing on the development of outcome indicators that can be used to measure the implementation of the UN Convention on the Rights of Persons with Disabilities (CRPD) and attainment of the Post 2015 Sustainable Development Goals (SDGs). The session opened with a discussion on the use of the WG short set questions for the purpose of data disaggregation by disability status. Four initiatives were presented:
- the work of the Danish Institute of Human Rights and the Danish Social Research Institute on development of a set of 10 statistical outcome indicators (Gold Indicators) that correspond with 10 key elements from the UN CRPD.
- the Incheon Strategy in UN Economic and Social Commission for Asia and the Pacific (ESCAP) countries. The Incheon Strategy was launched in November 2012 and is based on 10 goals designed to help realize the CRPD.
- Sightsavers highlighting work on a disability disaggregation. The objectives of the project are to determine whether people with disabilities are accessing the services provided by Sightsavers and making Sightsavers projects more inclusive of people with disabilities.
- work by the UN Secretariat of the CRPD on SDGs: Drafting a technical note recommending that disaggregation by disability should be carried out wherever possible and highlighted the key targets for disability indicators.
The WG continues to monitor the collection of disability data internationally, and annually requests detailed information from member countries covering survey periodicity, sample size and frame, mode of data collection, language(s) used, and exact question wording along with response options. Annual reports were completed by 57 countries. To date, 41 countries have indicated that the WG short set (or some variant) was included in their most recent census and 54 countries have indicated that the short set has been pre-tested or added to a national survey. Countries were also asked whether the WG short set were included on a national survey that also collects information on school attendance, employment status and information and communications technology (ICT). This information will be used to respond to future data requests. The format of the country report form was updated in 2015 to obtain information on the use of the extended set of questions on functioning. Reponses were received from 5 countries indicating that the extended set (or a subset) have added to a national survey.
Individual country activities were presented by representatives from Brazil, Denmark and Morocco. These covered, respectively, a comparison of results from the 2000 and 2010 Populations Censuses in Brazil, practical experiences from the Danish disability registry and results from the 2004 and 2014 Censuses and 2014 National Survey on Disability in Morocco.
A representative from the Australian Department of Foreign Affairs and Trade (DFAT) provided an overview of DFAT’s collaborations with the WG, University College London, UNICEF, UN Statistics Division (UNSD) and the Australian Bureau of Statistics (ABS). DFAT has provided funding to the WG to improve the collection and analysis of disability statistics globally by strengthening the WG’s capacity to broadly disseminate and provide technical assistance to support the consistent implementation of the WG’s existing data collection tools; and engage in new work on measures of participation and functional limitations related to mental health. University College London will assist the WG by providing administrative and technical support for this collaboration. DFAT is also partnering with UNICEF and the WG to improve the availability, quality and use of data on children with disabilities. The funding provided by DFAT will support UNICEF’s and the WG’s efforts to develop the Module on Child Functioning and Disability and Module on Inclusive Education. DFAT is working with the UNSD to enhance the capacity of national statistical offices to produce and disseminate quality statistics on disability for evidence-based policy making and monitoring. UNSD is planning to make an inventory of existing disability measurements and produce technical guidelines based on best practices. The funding provided to UNSD by DFAT will also be used for the capacity building of national statistical systems to efficiently collect and generate disability statistics that can be used to formulate, implement and monitor disability inclusive development policies and programs. DFAT is also endorsing the use of the WG short set questions through the Pacific statistics strengthening program. The program aims to provide technical assistance to Pacific island countries and territories to improve their collection, analysis and dissemination of economic and social statistics. A recommendation at the recent 2020 census planning workshop to use of the WG short set questions in 2020 Pacific census.
A representative from ABS provided an overview of their collaboration with DFAT to assess the impact of transitioning from the disability questions currently used in Australia’s national data collections to implementing the WG short set questions.
An overview of the United Kingdom’s ‘Leave No One Behind’ promise was presented by a representative from the United Kingdom (UK) Department for International Development (DFID). The promise was launched by UK’s Prime Minister at the 2015 United Nations General Assembly and is based on 8 principles with the aim of eradicating extreme poverty and leaving no one behind. Elements of the current Framework include: advocating the use of the WG questions to obtain data on disability, producing guidance on disaggregation, and advocating for disaggregation in the SDGs.
An overview of the WG’s collaboration with Handicap International (HI) was provided by representatives from HI. The presentation included a few examples of projects conducted by HI that incorporate the use of disability data.
A post-meeting session on the Global Network on Monitoring and Evaluation for Disability-inclusive Development (MEDD) was led by a representative from the UN DESA/UN Secretariat of the CRPD. The session included a short presentation followed by a discussion. The MEDD Global Network is an informal group of experts, researchers and practitioners in the areas of disability policy, data and statistics, monitoring and evaluation. The Network’s aim is to support the production of a UN flagship report to compile and analyze national policies, programs, best practices and available statistics related to persons with disabilities.
Key agreements of WG-15 and next steps:
The WG agreed to its work plan for 2016. Among other items, the plan delegates specific responsibilities to working groups that meet throughout the year (via email and telephone conferences). Next steps for the steering committee and work groups include:
Work on the short set questions (WG SS) and extended sets of questions on functioning (WG ES-F):
- Finalize analyses of WG ES-F data from the US NHIS
- Finalize guidelines for analyzing data obtained from the WG ES-F
- Provide examples using different cut-points for estimating disability prevalence rates
- Provide a recommended cut-point to be used reporting internationally comparable disability prevalence rates
- Focus on development of strategies for implementation and dissemination of data collected using the WG questions
Work on the UNICEF/WG Module on Child Functioning and Disability:
- Continue field testing and use test results to inform additional changes to the module
- Continue collaboration with UNICEF to finalize a manual for implementation of the module
- Present a final version of the module and manual for implementation at the 16th meeting
Work on module measuring inclusive education:
- Continue collaboration with UNICEF on the development of the module including cognitive and field testing
- Continue collaboration with UNICEF on the development of a manual for implementation
- Present a draft version of the survey module and manual for implementation at the 16th meeting
Work on the development of extended sets of questions on environmental factors and participation
- Draft a report describing the different approaches for question development, including the challenges and limitations
- Provide example of questions for one domain following the approach adopted by the workgroups focusing on inclusive education and labor force participation
Work on other methodological areas:
- Continue work on disability module for use in labor force surveys
- Continue work on development of measures specific to mental health
- Final results from the analysis of data provided by countries using the WG short set in recent national data collections (censuses and surveys) showed that although countries continue to report disparate disability prevalence rates; with some exceptions, those that use the WG as intended, using a cut-off of least one domain that is coded as a lot of difficulty or cannot do it at all, have reported disability prevalence rates that are comparable – in the range 7 – 10%. Using 2010 and 2011 NHIS data, algorithms for combining multiple domain questions into single domain indicators of disability and developing standards for determination of cut-points using the WG extended set of questions on functioning (ES-F) were presented. A small workgroup was formed to finalize the analysis.The WG/UNICEF workgroup on disability among children and inclusive education and the workgroup on mental health presented the work accomplished in the previous.Two representatives from the Saudi Disability Registry Group (SDRG) presented on their experience developing a national disability registry. The delegates agreed that the workgroup should continue its efforts and provide a status update at the next meeting.
- Results from the continued analysis of data provided by countries using the WG Short Set in recent national data collections (censuses and surveys) and analyses using data from the 2010 and 2011 US NHIS addressing how best to combine information from several questions per domain on the WG extended set on functioning and develop standards for determination of cut-points were presented.The WG/UNICEF workgroup on the development of specific question modules designed to measure disability among children presented the results of cognitive testing that was completed in Oman, Belize, India, Montenegro and USA and the revised Module on Child Functioning and Disability.The presentation on whether and how issues related to mental health could be incorporated into the work of the WG resulted in the creation of a workgroup to further address this. Interest was also expressed in having the WG look into the use of data registers in the compilation of disability statistics.
- Results from further analysis of data provided by countries using the WG Short Set in recent national data collections (censuses and surveys) were presented. Using data from the 2011 NHIS, results from further analysis of the WG extended set of questions on functioning (ES-F) were also presented. Representatives from NCHS provided a presentation focusing on the use of a mixed-method approach to assess validity and cross-subgroup comparability. Results from the 2010 NHIS using the ES-F questions related to pain were presented. The workgroup investigating environmental factors and participation provided an update on their work.The workgroup (in collaboration with UNICEF) on the development of specific question modules designed to measure disability among children presented a review of the conceptual framework for question development and a proposed set of questions. An update on the Multiple Indicator Cluster Surveys (MICS) was presented on behalf of UNICEF, as well as, a presentation by a representative from ADAPT (formerly the Spastics Society of India) on the cognitive testing of the module on child functioning and disability in India.
- The Washington Group began to monitor the use the WG Short Set in national data collections (censuses and surveys). Preliminary analyses of data provided by countries using the WG Short Set were presented. Preliminary findings from analyses of the WG extended set of questions on functioning using data obtained from the US National Health Interview Survey (NHIS) were also presented at the meeting. The children’s workgroup presented a well-received proposal for the development of the extended set for children including a conceptual framework and examples of how the framework could be operationalized. A formal collaboration was arranged between the Washington Group and UNICEF to work on the development of specific question modules designed to measure disability among children. The workgroup investigating environmental factors also presented their work consisting of a conceptual framework and related questions sets.
- The 10th meeting of the Washington Group on Disability Statistics (WG) was held in conjunction with the Budapest Initiative Task Force on Measuring Health Status (BI). The primary focus of the meeting was to review results obtained from the 2010 round of cognitive and field testing of the extended set of disability questions that took place in Europe (Granada Group) and South-East Asia (UN ESCAP). In collaboration with the BI, a final version of a question set on health state (a subset of the extended set on functioning) was submitted to Eurostat for inclusion on the European Health Interview Survey (EHIS). It was determined that the workgroups were ready to begin developing question sets designed to measure disability for children and environmental factors.
- The primary focus of the 9th meeting was the presentation and discussion of results from the cognitive tests and preliminary field tests results in South-East Asia. Preliminary results from the field tests in two of the participating UN ESCAP countries (Maldives and Sri Lanka) were presented. The overall conclusion was that further analysis of the field test data was required before a final decision could be made regarding the extended set of questions. It was concluded that it was beyond the scope of the WG to address the issue of dealing with institutionalized populations at that time. Two workgroups were constituted to look more closely at the measurement of child disability and the development of a set of questions on environmental factors as they relate to the measurement of disability. It was strongly recommended by the delegates, particularly those from African countries, that projects similar to the one funded by UN ESCAP in the Asia and Pacific region, also be established in other regions.
- Work on the extended set continued by expanding upon the set of domains already covered in the short set, and adding supplementary questions within domains (cause, age at onset, duration). Development of the extended set/s was to be done in collaboration with the Budapest Initiative, Eurostat, and UN ESCAP. Methodological issues were raised during the 8th meeting concerning the development of questions for children and institutionalized populations and the use of proxy respondents. WG representatives from Canada and France volunteered to look at the work being done in the areas of children and institutionalized populations.
- The work group on the short set addressed the use of the short set as a screener and presented an alternative (optional) question on upper body function. The combined work group on data analysis and methodological issues provided further analyses of the pre-test data presented at the 6th meeting. A large part of the 7th meeting was dedicated to a discussion of work being done on the extended set of disability questions for surveys and survey modules. The extended set work group would coordinate its work with the work of the Budapest Initiative, Eurostat, and UN ESCAP.
- Based on the outcomes of the pre-tests, the WG endorsed the six question set for use in censuses. The set comprises questions on four core functional domains (seeing, hearing, walking, and cognition) as well as two additional domains desired by member countries (self-care and communication). The methodological work group and the data analysis work group merged to continue the detailed analysis of the pre-test data. The work group on extended measures was charged with drafting a position paper on the first extended set.
- Revisions were suggested for the short measurement set, the accompanying rationale, and the implementation protocols. A new work group was formed to plan and implement analyses of the WG pre-tests. All results pertaining to the six WG questions will be considered by the new work group including the WG sponsored pre-tests, the WHO/ESCAP test, and other testing activities.
- Agreement was reached on the following: Conceptual agreement on a draft set of questions for the general disability measure, but wording revisions were required prior to pre-testing; a new workgroup operating in conjunction with a consultant to develop six implementation protocols for pre-testing the short set of disability measures was formed; agreement to start development of the first extended measurement set was reached; and a new workgroup on methodological issues was formed.
- Since disability is multidimensional, it is not possible to ascertain the single “true” disabled population. Different purposes are related to different dimensions of disability or different conceptual components of disability models. Equalization of opportunities was selected as the purpose for which an internationally comparable short disability measure would be developed. A work group was designated to generate a draft set of questions related to this purpose. In addition, two other work groups were formed to propose methods for implementing the short set and to propose an approach for developing extended measurement sets related to the short set. Finally, a plan for WG governance was adopted.
- A link was established between the purpose/s of a short measure on disability and aspects of measurement. A conceptual matrix was developed linking the purpose of a short disability measure with conceptual definitions and question characteristics. An empirical matrix was developed evaluating the characteristics of short set(s) of disability measures currently in use according to the dimensions of the conceptual matrix. Both matrices helped the WG to identify gaps in disability measurement.
- Agreement was reached on the following: 1) it is important and possible to craft a short set/s of internationally comparable disability measures; 2) short and long set(s) of measures that are inter-related are needed; 3) the ICF model will be used as a framework in developing disability measures; and 4) census questions are the first priority.