Causes of incomplete data, adjustments and effects - ARCHIVED

The article provides a general overview of the concepts of incomplete data and non-response. It is recognized that non-response is an important indicator of data quality, as it affects the estimators by introducing bias and increasing variance due to a reduction in the effective sample size. The relationship between bias and the non-response rate is less obvious, since it depends on the extent of non-response and on the difference in the various characteristics between respondents and non-respondents.

The most effective way of dealing with the effects of non-response is to minimize its extent. However, any attempt to control the extent of non-response must be based on a good understanding of its origins. The causes and extent of non-response are fundamentally related to (i) the type of survey, (ii) the data capture methods, and (iii) the sample design. However, given a sample design, the extent of non-response will be influenced by factors such as the type of region and the type of non-response.

There are several ways to handle incomplete data. Each one, in the end, assigns a value to the missing or incorrect data, unless it is decided to publish “raw” data. The procedure for assigning values is called imputation and such an imputed value presumably describes the characteristic of the non-respondent.

The article provides a brief philosophical explanation about validation and imputation and their applications in the methodology of the various imputation procedures. These include weighting, replication, hot deck imputation using previous data and substitution by a zero value. The using of imputation compared with the methods used in the Canadian Labour Force Survey (LFS) is also discussed. A decision table is provided indicating the various steps to follow for a particular case of a partially completed LFS questionnaire.