Housing Statistics in Canada
Development of a composite quality indicator for statistical products derived from administrative sources

Introduction

In recent years, with the proliferation of data sources available, national statistical offices such as Statistics Canada have started to transition from survey-centric methods to produce official statistics, towards a survey-assisted model where sample surveys have become a complement to, or even completely replaced by alternative data sources (e.g. administrative sources or other types of data such as web scraping or remote sensing). Some of these new data sources, on their own or when combined with others, represent an opportunity for various statistical programs to improve the statistics they produce over several quality dimensions, such as accuracy and timeliness. Improvements in timeliness are subject to direct evaluation, but what about data accuracy? Measuring and communicating about data accuracy when using and combining alternative data sources becomes an important challenge, as many conventional methods and the associated nomenclature used are largely anchored in survey sampling theory.

Statistics Canada is required to inform users of the concepts and methodology used for collecting, processing, and analyzing its data; of the accuracy of these data; and of any other features that affect their quality or “fitness for use” as outlined in the Policy on Informing Users of Data Quality and Methodology (Statistics Canada, 2000). Statistics Canada sets out to convey to users the quality of its data through the use of quality indicators, which are designed to represent the amount of error present in the data. Errors or variability in the estimates may originate from various sources throughout the process such as data entry errors, or inherent errors in the methods used in different processing steps. In traditional sample surveys, sampling error, as the main source of error, is usually evaluated and reported in terms of quality indicators based on the coefficient of variation or in terms of confidence intervals. For data integrated from different administrative sources, measuring quality in a rigorous and interpretable way can be more challenging.

This paper describes the development of a new quality indicator that informs users about the quality of estimates produced using administrative data sources only. While Statistics Canada defines quality with six dimensions (relevance, accuracy, timeliness, interpretability, coherence and accessibility [Statistics Canada, 2019]), the purpose of the indicator described in this paper is to inform the user about the accuracy of the data. This new quality indicator, defined as the composite quality indicator (CQI), combines several indicators derived at various steps of the statistical process to create one single indicator of the overall accuracy of each estimate.

The proposed CQI has been applied experimentally for the first time for the Canadian Housing Statistics Program (CHSP). The CHSP is an innovative program that provides Canadians with wide-ranging housing statistics including information on residential properties and their owners. The CHSP, initiated in 2017, integrates administrative data from a variety of sources to build and maintain a database of residential properties and owner characteristics from Canadian provinces and territories. 

This paper is organized in four sections. First, individual quality indicators (QIs) are proposed to represent the quality of different statistical processes within the program. Second, a clustering and weighting methodology is presented, which combines the individual QIs into a CQI. The third section provides insights on the interpretation of the CQI. The fourth section highlights the limitations of the current methodology and some considerations for an improved version. Then, a brief conclusion is given. The appendix contains an illustration of the application of the CQI approach for the estimates that were published in three CHSP tables in September 2021.

1. Individual quality indicators

The sources of error for an administrative data only program are limited to non-sampling errors such as coverage errors, nonresponse errors, processing errors and measurement errors (Groves and Lyberg, 2010). The first step of the CQI approach is to derive individual quality indicators to indicate the impact of the non-sampling errors on the accuracy of administrative estimates. The experimental version of the CQI presented in this paper is based on a series of individual quality indicators that were readily available, and are related to nonresponse and processing errors only. The development of individual quality indicators on coverage is considered for future iterations. Below, we present the reader with the current set of individual quality indicators from the different steps of a statistical process. However, some basic terminology is defined first.

1.1 Basic terminology

Many data products released by Statistics Canada are aggregated tables of estimates, such as counts, percentages, totals, means or medians (estimate type). These estimates are often descriptions of a variable of interest for the whole population or for specific subgroups of the population (domain). Domains can be different categories of a single categorical domain variable or combinations of categories across the different domain variables. For instance, a data user consulting Canadian Housing Statistics Program (CHSP) table 46-10-0027-01: Residency participation of residential properties, by property type and period of construction (Statistics Canada, 2021), could be interested in the estimate of the average assessment value (variable of interest) of single-detached, resident-owned properties, built from 1961 to 1970, in Ontario (domain variables). Taken together, this domain is one of the possible combinations of categories of domain variables: geography, property type, period of construction and residency participation. In this example, assessment value is the variable of interest, that is, the variable for which an estimate is presented in the table. Each individual quality indicator presented in the next section is related to either a domain variable or a variable of interest. In the application of the CQI approach in Section 2, a value for each individual quality indicator will be provided for each variable, at the domain level.

1.2 Set of individual quality indicators at different processing steps

Listed below is a set of individual quality indicators about administrative nonresponse and processing aspects that can be considered in the evaluation of the accuracy of administrative estimates. The list does not provide an exhaustive list of all possible indicators, nor does it cover all processing steps.

a. Coding

To provide coherent and consistent concepts to users, a coding process often occurs to convert the individual values from different administrative data sources to a common code set. When no data are available, or the data are unable to be coded, other processes, such as imputation, may be used to provide a final value for all units. A coding rate is defined as the percentage of administrative units that are coded out of all units included in the domain. In general, a high coding rate reflects a high level of quality for that domain.

One such coding rate is the geocoding rate, which refers to a coding process where an administrative unit is assigned to a pre-defined detailed geographic area (e.g. census subdivisions),^Note  based on the unit’s address information.

b. Geocoding

In addition to the geocoding rate, another individual quality indicator is called the average geocoding confidence score. This score can be used to demonstrate the quality of the assigned detailed geographic area and is defined as follows:

Average geocoding confidence score: At Statistics Canada, the geography service responsible for the geocoding process can provide a confidence score value between 0 and 1 for each unit to indicate the confidence of the detailed geographic area coding. A confidence score of 1 indicates that the address information allows for accurate identification of the detailed geographic area data, whereas a confidence score of 0 implies that the assignment of the detailed geographic area data was unsuccessful (not coded). A value between 0 and 1 implies that a detailed geographic area was assigned with less or more accuracy. The average geocoding confidence score is the average of the geocoding confidence scores for all units included in the domain. The higher the average geocoding confidence score is, the higher the quality of the geocoded data will be.

c. Record Linkage

A linkage process can be performed to link information from different administrative data sources to provide a more comprehensive set of data. The quality of the linkage process and the ability to correctly match records have a direct impact on the quality of the derived variables. Two linkage error rates: the false discovery rate (FDR) and the false negative rate (FNR), each independent quality indicators, are presented here to show the quality of the linkage process. The domain false discovery rate is the number of incorrectly accepted links divided by the total number of accepted links associated with units in the domain. The domain false negative rate is the number of true links that are rejected divided by the total number of true links associated with units in the domain. It is possible to estimate these two error rates by doing a manual review of linkages based on a sample of linked and unlinked units. These error rates vary inversely with the quality: the lower the linkage error rates are, the higher the quality of the derived variables will be.

Linkage error rates can also be presented using the complements of the false discovery rate, represented by the precision (1-FDR), and the false negative rate, represented by the recall (1-FNR).

d. Estimation for quantitative variables

During the estimation process, estimates are produced by calculating different statistics (totals, means, medians, etc.) of quantitative variables for all units in a given domain. The variable values can be either reported (available from an administrative source), imputed (a value is assigned through an imputation strategy), missing (no information is available from the administrative source, and no imputation is performed), or not applicable. Two quality indicators are proposed for assessing the quality of estimates based on the availability and source of information, the inclusion rate and the reporting rate, defined as follows:

Inclusion rate: For a given domain, the inclusion rate is defined as the percentage of units in the domain reported and imputed among the units that contribute (reported, imputed or missing) to the estimate calculation. The higher the inclusion rate, the higher the coverage of the estimate will be.

Reporting rate: For a given domain, the reporting rate is defined as the percentage of units in the domain with reported values among the units that contribute (reported and imputed) to the estimate calculation. The higher the reporting rate is, the higher the quality of the estimate will be.

2. The composite quality indicator

The aim of the CQI approach is to assign an indicator of the level of quality related to the accuracy for each estimate, which could then be presented alongside estimates in a table using letter grades from A to F. The CQI is calculated by combining the individual quality indicators (QIs) related to the domain variables and variable of interest that were used to create the domain estimate. The CQI or letter grade is relative, in the sense that a CQI value of A indicates the estimate has the highest level of accuracy relative to estimates for the same variable of interest across the other domains. A value of A is usually assigned when each of the applicable QIs has a very high value. A CQI value of B, C, D, E or F reflects lower levels of accuracy relative to the other domains for some of the applicable QIs, with F representing the lowest relative accuracy.

Combining the QIs is done using clustering. Clustering is an unsupervised machine learning technique that groups together observations—here, the domains—that are similar between themselves and dissimilar to observations assigned to other groups. The dissimilarity is based on a distance metric calculated from clustering variables—here, the applicable QIs. The K-means is the chosen clustering algorithm.

Algorithms based on distance metrics such as clustering are sensitive to the scale. Variables with larger variance have more impact on the creation of the groups unless all variables are standardized before clustering. In the application of clustering to create the CQI, some QIs should have more impact because they are related to the variables the most strongly associated with the domain estimates. To control the relative importance of QIs in the analysis, each QI is standardized and then multiplied by a weight. QIs multiplied by the highest weights will have the largest variance once weighted, and consequently, the most impact on the creation of the groups. Because weights are determined based on the nature of the estimate (the estimate type or the variable of interest), different series of weights may be needed for different estimates within the same table. For instance, weights assigned to QIs of domain variables could be of equal importance when producing the CQI for the counts of administrative units in the domain, while weights assigned to the same QIs could be proportional to the strength of the relationship between these domain variables and the variable of interest in the case of a mean or median estimate.

Clustering creates unordered groups. To determine which group represents the highest level of accuracy, clusters are ordered according to a global score calculated for each cluster. The global score is the average by cluster of the weighted average of QIs in each domain. The group with the highest global score is initially assigned the value A, the group with the second highest global score is initially assigned the value B, etc. Visualizations of cluster profiles are used to better understand the composition of each cluster, to confirm the letter grade automatically assigned to the group or to assign a different letter grade to the group, as will be discussed in the next section.

Examples of the calculation of the weights and data visualization are presented in the Appendix for selected CHSP tables released in September 2021, which included the CQI letter grades for the first time.

3. Interpretation of the composite quality indicator

The CQI letter grades are attached to standard quality labels used at Statistics Canada for sample surveys, as defined in the table below.



Despite the use of the same standard quality labels, there are major differences in the way the CQI letter grades are created compared with sample surveys. Users must be aware of these differences to properly interpret the CQI labels.

Generally, sample surveys use letter grades A to F to indicate the magnitude of the sampling variance of the estimates, which is typically measured by the coefficient of variation (CV). The CV is the ratio of the standard error of the estimate to the average value of the estimate across all possible samples. It is usually expressed as a percentage, and the lower the CV, the better the precision of the estimate is. The range of possible CV values is divided into intervals, and each interval is assigned to a letter grade. While different intervals may be used to assign the accuracy level from one survey to the other, comparing the accuracy of estimates within the same sample survey is straightforward. Moreover, CVs can be used both as an indicator of the precision of the estimates and for the purpose of making inferences, that is, to draw conclusions from statistically significant difference between estimates.

The CQI is very different, as it is not tied to variance estimation. The goal is to help users evaluate the fitness for use of the estimates relative to their own needs. It cannot be used to make statistical inference. The use of the clustering algorithm also has significant implications that affect the comparability of the resulting CQI letter grades among different tables and among estimates of different types or from different variables of interest.

First, the clustering model produces CQI values that are relative and not absolute. The accuracy of an estimate is not assessed against predefined standards, but against the accuracy of estimates in the other domains. It would be difficult to set predefined standards that would be consistent but equally relevant in all contexts since the choice of applicable QIs is based on processes that are specific to one particular statistical program, and the calculation of the weight to be applied to each QI is specific to each table and variable of interest. Although clustering gives relative, albeit data-driven results, it is the most objective way to create groups based on the various QIs without using arbitrary rules.

To mitigate the risk of assigning inconsistent values and labels as the final CQI letter grade for one group, for instance assigning an A to the best cluster despite suboptimal average accuracy in the cluster, profiling the clusters to confirm the letter grade assigned automatically or to assign a more appropriate letter grade is essential. Based on data visualization of the frequency distributions of the QIs in each cluster, a decision could be made to assign a lower grade than A to the best cluster. The same principle applies to the other clusters as well. If the profile from the second-best cluster indicates an overall accuracy that is not deemed sufficient to be qualified as “Very good,” this cluster could be assigned a lower grade than B. A consensus should be reached with subject-matter experts before assigning final CQI letter grades, while considering corresponding standard labels.

Second, the clustering model groups together domains that have similar values in terms of the applicable QIs, while considering all of them simultaneously. This approach makes the interpretation more meaningful. Since all domains that are grouped together show similar patterns, it is possible to provide an informative description of the QIs for each CQI letter grade. For example, it is possible to explain that for a given variable of interest, the accuracy of the estimates in domains assigned to a C is deemed good, because one QI indicates a lower level of accuracy for a given processing step, while the remaining QIs report an excellent level of accuracy for all the other evaluated processing steps. This kind of rationale, provided along with CQI letter grades and standard labels, makes the interpretation simpler for data users.

Finally, it is recommended to use the CQI letter grades to compare the accuracy of the same variable of interest and estimate type between domains, but not to compare the accuracy between different variables of interest or estimate types within the same domain.

4. Limitations of the composite quality indicator approach

Combining a set of QIs using clustering has many advantages. It is a simple and fast way to summarize a large number of QIs into a categorical quality rating while allowing for assigning relative importance of each QI to the overall quality of the estimates. This helps derive an interpretation of CQI values that is meaningful, which will also help data users to assess fitness of the data for the intended uses.

The main limitation of the CQI is the nature of the applicable QIs used in the clustering model. In the context of a first experimental CQI implementation, the development of a readily available set of QIs might consist of defining simple rates at different steps of processing, which indicates only the number of administrative units for which each step has been completed, but does not provide an indication of how well the step has been completed. A good example is the coding rate. In an improved version of the CQI, the goal would be to have QIs that better measure the quality of each processing step and the input to the processing step. For example, an indicator of the quality of the coding as well as an indicator of how accurately the data reflect the reality would be more representative of the quality than a coding rate.

Another limitation of the CQI is related to the level at which the applicable QIs are available. An example is when the geographic level of a QI is higher (more aggregated) than the level at which estimates are released. To have CQI values that better reflect the quality of the released estimates, it is recommended to define the QIs at the same level as the estimates, or to complement with other applicable QIs available at the proper level. This is another aspect to consider in an improved version of the CQI. 

Conclusion

For information products derived from administrative data sources only, traditional measures of accuracy such as CV and margins of error are not easily applicable or calculable. The CQI approach that was developed and presented in this paper was integrated for the first time in a Statistics Canada product released in fall 2021. The aim of this quality measure is to communicate to data users information about the accuracy of estimates by taking into consideration the quality of administrative data at the different processing steps (e.g. linkage, coding) and reporting rates. In addition, the use of weights and clustering algorithms in the development of the CQI has proven to be a simple and effective approach to summarize the large quantity of indicators needed in the context of multidimensional tables. Given that relevant QIs are defined and produced along with the estimates, the approach could also be implemented in other statistical programs that are based on administrative data exclusively.

Appendix A

This appendix explains how the method presented in this paper was developed and first implemented in the context of Canadian Housing Statistics Program (CHSP) tables released on September 17, 2021.

A1. Presentation of tables used to develop the composite quality indicator

The three following CHSP data tables were updated with new data for reference year 2020 for Ontario, British Columbia, Nova Scotia and New Brunswick:

• Table 46-10-0027-01: Residency participation of residential properties, by property type and period of construction
• Table 46-10-0053-01: Ownership type and property use by residential property type and period of construction
• Table 46-10-0054-01: Residency ownership and property use by residential property type and period of construction

For simplicity, these tables will be referred as tables 27, 53 and 54, respectively.

All tables are comprised of the three domain variables: geography, property type and period of construction. Four other domain variables are included in at least one of the three tables: ownership type (Table 53), property use (tables 53 and 54), residency ownership (Table 54) and residency participation (Table 27). Variables of interest vary across the different tables but include counts and percentages of properties, and mean and median of assessment value, total living area and assessment value by square foot. A description of CHSP concepts and variables can be consulted in existing documentation.^Note 

A2. Selection of quality indicators

Geocoding rates and average geocoding confidence scores were used to describe the accuracy of the domain variable geography, which includes the categories: province, census metropolitan area (CMA), census agglomeration (CA)^Note  and census subdivision (CSD). As CHSP data sources are based on provincial and territorial registers, the province is always known so both quality indicators (QIs) are fixed at 100% for all provincial-level estimates.

The accuracy related to domain variables ownership type, period of construction and property type was described using coding rates. However, as the coding rates for ownership type and property type were always equal to 100%, they were not used in the clustering models. To be part of discriminant quality criteria to consider in a clustering model, QIs must have non-zero variance.

Domain variables property use, residency ownership and residency participation are created using linked data between two or more data sources. For this reason, their accuracy was described using the complements of the linkage error rates: the precision (1-FDR [false discovery rate]) and the recall (1-FNR [false negative rate]). These QIs are preferred to the estimated linkage error rates, because all selected QIs should ideally vary in the same direction: the higher the value, the better the accuracy.

Finally, reporting rates were used for the continuous variables of interest including assessment value and total living area. The accuracy of the total living area was also described using an inclusion rate to indicate the proportion of properties considered for the estimate calculation in a given domain. The accuracy of assessment value by square foot was described using the reporting rates of assessment value and total living area and the inclusion rate of total living area, because assessment value by square foot is the ratio of assessment value and total living area.

Table A2-1 summarizes the variables used in the construction of tables 27, 53 and 54 as well as selected QIs relevant to each variable.



A3. Weighting for counts and percentages

Tables 27, 53 and 54 present the count and percentage of properties in each domain. In Table 53, the percentages of properties are available by property use and ownership type, while in Table 54, they are available by property use and residency ownership. For count and percentage estimates, the weights are calculated in two steps. First, equal weights are assigned to the variables involved in the calculation of the estimates. Second, the weight of a variable is transferred to the related QIs by splitting it equally among QIs when there is more than one QI related to the variable. An example is illustrated in Figure A3-1 for Table 27.

Description for Figure A3-1

This figure is a hierarchical chart that shows how the weight is split among the QIs.



Data table for Figure A3-1
Table summary
This table displays the results of Data table for Figure A3-1. The information is grouped by Variable (appearing as row headers), Variable weight, Quality indicator and Quality indicator weight (appearing as column headers).

Variable	Variable weight	Quality indicator	Quality indicator weight
Geography	0.33	Geocoding rate	0.17
		Average geocoding confidence score	0.17
Period of construction	0.33	Coding rate	0.33
Residency participation	0.33	Precision	0.17
		Recall	0.17
Note: All numbers are rounded to two decimal places

The accuracy assessment of the estimated count of properties in Table 27 is related to the quality of the four domain variables: geography, period of construction, property type and residency participation. Since the QI available for property type was always equal to 100%, it could not be used in clustering. Consequently, a weight of 0.33 was assigned to geography, period of construction and residency participation. Two QIs are used for geography, so a weight of 0.17 was assigned to both geocoding rate and average geocoding confidence score. The quality of period of construction is described by only one QI, so the coding rate was assigned a weight of 0.33. A weight of 0.17 was assigned to both precision and recall because they are the QIs related to residency participation. Table A3-1 presents the weights assigned to each QI by released table.



A4. Weighting for mean and median of continuous variables

Table 27 presents estimates for one continuous variable of interest: assessment value. Tables 53 and 54 present estimates for three continuous variables of interest: assessment value, total living area and the ratio between these two variables called assessment value by square foot. For continuous variables of interest, weights were assigned proportionally to the strength of the relationships between domain variables and variables of interest. For each variable, this was measured using analysis of variance (ANOVA) models, which perform the calculation of the proportion of total variance explained by each domain variables, referred to as the effect size. In each model, the outcome (Y) variable corresponds to the variable of interest, while the effects (Xs) are the domain variables. The geography variable used for the analysis is the CSD because it is the most detailed geography level. CMAs and CAs typically regroup one or more CSDs, while provinces include multiple CMAs, CAs and the category “Outside CMA or CA.” Modelling is performed at property-level. Results are presented in Table A4-1.



Results for Table A4-1 show that of all domain variables included in tables 27, 53 or 54, geography (26.1%) is the one with the largest effect size on the assessment value, followed by property type (7.2%). For total living area, property type has the largest effect (24.4%), followed by the geography (9.9%) and period of construction (5.7%). Geography (26.7%) also has the largest effect on assessment value by square foot. The other domain variables account for less than 1% of the variance of the three variables of interest.

The last line, the residual, indicates the amount of variance unexplained by the model. The variables in tables 27, 53 and 54 explain between 28% and 41% of the total variance only, which means that these effect sizes might potentially be biased because of omitted variables. Omitted variables might exist in the CHSP databases or be unavailable. The residual was ignored because the primary goal of calculating weights was to use models that reflect the construction of tables 27, 53 and 54.

From the results of the ANOVA models, the weights were derived as follows:

1. Equal weights were assigned to domain variables involved in the calculation of estimates, excluding domain variables for which a discriminant QI was not available.
2. For variables of interest, the weight assigned to a variable was split equally among the QIs related to the variable.
3. For the domain variables, the sum of the variable weights of domain variables is multiplied by the ratio of the effect size of a given domain variable to the sum of the effect sizes of all domain variables included in the released table, excluding variables for which a QI was not available or could not be used in the composite quality indicator (CQI) model. The effect size of CSD was used to calculate the weight for geography.
4. The weight assigned to each variable was split equally between QIs related to the variable.

An example is illustrated in Figure A4-1 for Table 27.

Description for Figure A4-1

This figure is a hierarchical chart that shows how the weight is split among the QIs.



Data table for Figure A4-1
Table summary
This table displays the results of Data table for Figure A4-1. The information is grouped by Variable (appearing as row headers), Variable weight, Quality indicator and Quality indicator weight (appearing as column headers).

Variable	Variable weight	Quality indicator	Quality indicator weight
Geography	0.75 x 0.96 = 0.72	Geocoding rate	0.72 x 0.5 = 0.36
		Average geocoding confidence score	0.72 x 0.5 = 0.36
Period of construction	0.75 x 0.04 = 0.03	Coding rate	0.03
Residency participation	0.75 x 0.00 = 0.00	Precision	0.00
		Recall	0.00
Assessment value	0.25	Reporting rate	0.25
Note: All numbers are rounded to two decimal places

Five variables are involved in the calculation of the mean of the assessment value in Table 27: assessment value, geography, period of construction, property type and residency participation.

1. In the first step, since there was no QI that could be used for property type, a weight of 0.25 was assigned to each of the four other variables.
2. In the second step, because the reporting rate was the only QI for assessment value, a weight of 0.25 was assigned to its reporting rate.
3. In the third step, the sum of the variable weights of domain variables was 0.75. The sum of the effect sizes of all domain variables included in Table 27 was 27.2% (from Table A4-1: 26.1% + 1.1% + 0.0%). The ratio of the effect size for geography divided by sum of the effect sizes was 0.96 (26.1% / 27.2%). The ratio of the effect size for period of construction divided by sum of the effect sizes was 0.04 (1.1% / 27.2%). The ratio for residency participation was 0. Each of these ratios was multiplied by 0.75, as shown in Figure A4-1.
4. In the fourth step, the weight assigned to geography was split in two between geocoding rate and geocoding average confidence score.

Table A4-2 presents the weights used for each continuous variable by released table.



A5. Clustering, ordering and profiles of clusters

QIs were standardized to remove the scale effect and then weighted before performing K-means clustering. The optimal number of clusters was selected using the graph of approximate expected over-all R-squared for values of the number of clusters (K) between 2 and 9. Since clusters were unordered, a preliminary order was determined by sorting clusters in decreasing order of a global quality score calculated for each cluster. The global score of each cluster k is defined as the following:

Where M is the total number of domains, ${𝕝}_{ik}$ is the indicator function taking value 1 if domain i is included in group k, or 0 otherwise, and ${\overline{\text{QI}}}_i$ is the weighted average of the quality indicators in domain i:

Where J is the number of quality indicators used in the CQI model, ${\text{QI}}_{\text{ij}}$ is the value of the quality indicator j for domain i, and $w_j$ is the weight of quality indicator j.

Data visualizations were used to study the profile of clusters. The relative frequency distribution of each QI was graphed within a grid for each cluster. An example is illustrated in Figure A5-1 for assessment value by square foot in Table 53. Precision and recall were removed from the data visualization to improve readability.

Description for Figure A5-1

This figure is a grid. In each cell, there is a bar plot that illustrates the relative frequency distribution of one QI (column) within one cluster (row).



Data table for Figure A5-1
Table summary
This table displays the results of Data table for Figure A5-1. The information is grouped by Interval of quality indicator values in percentage (%) (appearing as row headers), Relative frequency by interval - Group A, Relative frequency by interval - Group B and Relative frequency of in the interval - Group C, calculated using percent units of measure (appearing as column headers).

Interval of quality indicator values in percentage (%)	Relative frequency by interval - Group A	Relative frequency by interval - Group B	Relative frequency of in the interval - Group C
	percent
1. Average geocoding confidence score
[0,10]	0.0	1.8	41.8
[10,20]	0.0	0.4	4.6
[20,30]	0.0	0.5	3.3
[30,40]	0.0	0.5	8.8
[40,50]	0.0	1.8	35.2
[50,60]	0.0	2.4	0.9
[60,70]	0.0	17.2	0.0
[70,80]	0.0	37.2	3.1
[80,90]	3.3	38.2	1.3
[90,100]	96.7	0.0	1.1
2. Geocoding rate
[0,10]	0.0	0.0	57.6
[10,20]	0.0	0.0	4.8
[20,30]	0.0	0.0	1.3
[30,40]	0.0	0.0	11.0
[40,50]	0.0	0.1	25.1
[50,60]	0.0	0.3	0.2
[60,70]	0.0	0.8	0.0
[70,80]	0.0	1.3	0.0
[80,90]	0.1	2.1	0.0
[90,100]	99.8	95.4	0.0
3.Period of construction coding rate
[0,10]	0.4	1.8	7.9
[10,20]	0.1	0.3	0.9
[20,30]	0.2	0.4	3.5
[30,40]	0.3	0.7	4.2
[40,50]	0.6	1.7	9.0
[50,60]	0.6	0.8	1.3
[60,70]	0.9	1.0	3.7
[70,80]	1.4	1.6	3.5
[80,90]	2.6	2.7	1.8
[90,100]	93.0	89.0	64.2
4. Assessment value reporting rate
[0,10]	2.2	4.4	11.4
[10,20]	0.9	0.4	0.9
[20,30]	0.6	0.5	0.0
[30,40]	0.4	0.8	1.5
[40,50]	0.5	2.3	9.2
[50,60]	0.2	0.6	0.0
[60,70]	0.5	1.8	0.4
[70,80]	1.2	3.8	0.4
[80,90]	3.5	6.0	0.7
[90,100]	90.0	79.5	75.4
5. Total living area reporting rate
[0,10]	0.8	3.6	9.7
[10,20]	0.1	0.3	0.9
[20,30]	0.1	0.3	3.5
[30,40]	0.2	0.4	3.1
[40,50]	0.4	1.4	6.6
[50,60]	0.2	0.4	1.5
[60,70]	0.5	1.0	4.2
[70,80]	1.0	2.0	0.9
[80,90]	1.9	3.4	1.5
[90,100]	94.8	87.2	68.1
6. Total living area inclusion rate
[0,10]	0.0	0.0	0.0
[10,20]	0.1	0.1	0.4
[20,30]	0.2	0.3	0.0
[30,40]	0.5	0.6	0.0
[40,50]	1.0	2.0	3.5
[50,60]	0.9	0.8	0.0
[60,70]	1.6	2.5	0.7
[70,80]	2.8	4.1	2.0
[80,90]	5.2	5.9	0.9
[90,100]	87.7	83.7	92.5
Note: All numbers are rounded to one decimal place

Figure A5-1 shows that difference between the three clusters is mostly attributed to average geocoding confidence score (column 1) and geocoding rate (column 2).

A6. Interpretation of final composite quality indicator letter grades

After analyzing and comparing the cluster profiles for all models used to assess the quality of estimates in tables 27, 53 and 54, final CQI values are assigned to each cluster. Documentation is provided to data users referring to standard label values A to F. A description of quality components for each value is also provided to users, as shown in Table A6-1.



References

Groves, R.M. and L. Lyberg. 2010. “Total Survey Error: Past, Present, and Future.” Public Opinion Quarterly 74: 849–879. https://doi.org/10.1093/poq/nfq065

Statistics Canada. 2000. Policy on Informing Users of Data Quality and Methodology. https://www.statcan.gc.ca/eng/about/policy/info-user

Statistics Canada. 2019. Statistics Canada Quality Guidelines, Sixth Edition, Catalogue no. 12-539-X, Ottawa: Statistics Canada. https://www150.statcan.gc.ca/n1/pub/12-539-x/12-539-x2019001-eng.htm

Statistics Canada. 2021. Table 46-10-0027-01: Residency participation of residential properties, by property type and period of construction [Data table]. https://doi.org/10.25318/4610002701-eng

Statistics Canada. 2021. Table 46-10-0053-01: Ownership type and property use by residential property type and period of construction [Data table]. https://doi.org/10.25318/4610005301-eng

Statistics Canada. 2021. Table 46-10-0054-01: Residency ownership and property use by residential property type and period of construction [Data table]. https://doi.org/10.25318/4610005401-eng