Challenges in developing and implementing new data editing methods for business surveys

This paper discusses in detail issues dealing with the technical aspects of designing and conducting surveys. It is intended for an audience of survey methodologists.

A review of the Office for National Statistics (ONS) identified the need for new methods which would improve the efficiency of the data validation and editing processes in business surveys, but would not adversely impact the data quality. Methods for automating the correction of systematic errors, and for applying selective editing, were developed. However, the ways in which the organization and the procedures for ONS business surveys have evolved, presented a number of challenges in implementing these methods. This paper describes these challenges and how they were addressed and considers the relevance of these challenges to other organizations. Approaches to evaluating the impact of new methods on both quality and efficiency are also discussed.