Chapter 3.3: Enhancing how surveys are conducted

1.1.3.1 Full use of the Business Register

Statistics Canada uses its Business Register (BR) as the common frame for all business surveys. The BR is a database that is updated through a number of sources, which include administrative data files, feedback on Statistics Canada business surveys, and profiling activities that involve direct contact with companies to obtain information about their operations and Internet research findings. Using the BR ensures quality, while providing an integrated tool that can measure overlap and minimize response burden to the greatest extent possible. The development of the BR underwent a series of improvements over the years.

More than twenty-five years ago, the BR included only businesses with employees and, thus, covered only a subset of the Canadian economy. At that time, the employer's account data was the only administrative data source available that was reliable for identifying and maintaining businesses on the register. In the late 1980's, it was supplemented with Tax Records but with no integration between the two sources. In the late 1990's, Canada Revenue Agency (CRA) introduced the Business Number (BN) to help administer its various programs, and this greatly increased the potential use of administrative data.

The BN allowed Statistics Canada to link between multiple administrative data sources, which significantly enriched information on the BR frame for the economic survey program. During the past ten years, administrative data has become a key component of the Canadian BR.

The role of the BR is to provide statisticians, who produce and analyze economic statistics, with the highest quality frame in terms of coverage and data elements. In addition to having updates from administrative files, the BR is connected to the collection tools, and as such, the BR is updated daily with information coming from interviewers in the field.

The BR encompasses some of the fundamental concepts of the System of National Accounts, and provides the infrastructure to store, browse, maintain, and retrieve frame information. It also has the capacity for storage of the contact name, address and telephone number and the survey questionnaire identifiers. The Register can also generate an accurate list of contacts, which is required for the survey-data collection process. It monitors the level of response burden imposed on individual businesses by Statistics Canada, and provides relevant information to effectively manage response issues. Finally, it provides statistical information regarding the composition of the population of businesses in Canada in terms of organizational structure, industrial activity, size and geography.

In Canada, the BR contains approximately 6 million active businesses, 99% of which are single-unit businesses. There are about 35,000 businesses that have more than one operating entity (complex businesses), and they account for more than 45% of the Canadian economy. Different types of organizations are defined as a business such as: a corporation, a self-employed individual, a government entity, a non-profit organization, a partnership, or a financial fund.

In addition to being used as a frame for survey sampling, the BR also serves to

• browse the business structures;
• analyse the population of business structures;
• update the business and operational information;
• manage, monitor and control respondent burden;
• contact the businesses (support data collection);
• run longitudinal studies; and
• disseminate data from Canadian Business Counts.

In terms of maintenance, for simple business structures, the Business Number (BN) Registration file provides administrative data (tax data) to the BR, which updates automatically. For complex businesses, face-to-face or telephone interviews and coherence analysis are used to complete a full profile maintenance. The integrated nature of the IBSP also allows automated updates of the BR using a harmonized survey feedback process that applies to all business surveys.

In terms of response burden, a control module for excessive response burden was implemented on the Business Register system in December 2014. The objectives of this module are to reduce the excessive response burden for small and medium enterprises, as well as calculate the potential accumulated response burden for all active enterprises in the Business Register system. This is calculated over a three-year period. Depending on the class size of the enterprise and the total number of hours spent completing questionnaires, Statistics Canada determines if an enterprise has exceeded its response-burden limit. If an enterprise is overburdened for its class size, it will be excluded from all surveys for a duration of one calendar year.

1.1.3.2 The use of electronic questionnaires as the principal mode of collection

To gain on efficiency, but also to respond to businesses' desire to interact electronically with the Government of Canada, the use of electronic questionnaire (EQ) is now the preferred option for business survey data collection. In 2015, 46% of the business surveys converted to EQ with some form of follow-up, by either mail, fax or computer-assisted telephone interviews.

In addition, Statistics Canada has begun to develop and implement an Integrated Collection and Operation Systems Initiative (ICOS) and within it a Business Collection Portal to support business survey programs. The objective is to develop an integrated collection systems environment to achieve the targeted level of flexibility between modes and sites, and to fully exploit the use of the Internet for e-questionnaires. For more information about ICOS, refer to Chapter 3.4: Data Collection Planning and Management.

1.1.3.3 A metadata-driven approach to questionnaire development, sampling, edit and imputation, allocation and estimation processes

Statistics Canada has a long history of developing corporate metadata repositories for managing publications, services, and statistical holdings. However, relatively few survey programs have well developed metadata repositories for managing survey operations. The UES program did implement a metadata system that housed processing edits, along with variable cell numbers and cell descriptions. For IBSP, this metadata framework has been expanded to cover all aspects of survey processing. This approach increases efficiency, robustness, and responsiveness in delivering processing services for IBSP programs.

In the IBSP model, metadata are stored in easily modifiable tables that are used to drive systems programs. This has been a shift from the UES model where metadata were often hard-coded into programs. IBSP systems programs simply access information from metadata tables to direct their execution.

A key advantage of the IBSP metadata-driven system is that changes are required as program needs evolve, and can be accommodated by modifying metadata, rather than by rewriting system code. This provides more control for the processing team and more flexibility for users.

According to metadata management guidelines, metadata should be active, created for a purpose, and used in "downstream" processes. While the "no data without metadata" principle is often applied to final data output, in the form of descriptive metadata, this is also true for processing, especially in the IBSP. When a variable is created, it is tagged with descriptive elements, such as a name and an origin. However, metadata will also indicate how validation, editing and imputation must be done, and will track the variable's passage through the various processing steps.

Users have a single point of entry into the IBSP, since metadata will be integrated into every processing step, and managing metadata, along with the processes they direct, will naturally form components of the same seamless portal. This integration enables the system to instantly check whether run conditions are being met. For example, if a user chooses to execute a process, the interface can prompt the user to input the necessary metadata and to ensure that other prerequisites are in place. Inputs can then be validated automatically, and, as a result, the system would either stop to give a warning or allow the user to proceed with subsequent steps. The metadata interface approach does not require the user to deal with multiple applications, and does not require knowledge about the order of the various steps necessary to run processing.

While the efficiency goal of a metadata-driven system is to minimize rework and facilitate re-use, improving quality and coherence is an equally important outcome. The integration of metadata in processing operations facilitates automatic coherence checks. Data integrity rules are enforced through the system's database to ensure the quality of inputs. Metadata also generates invaluable management information to aid in monitoring progress, thereby improving the overall quality of survey processing.

Building an integrated metadata-driven infrastructure also facilitates the implementation of harmonized conceptual framework. For IBSP, this began with the mainstreaming of statistical units, populations, concepts, variables, classifications and sets of questions. All IBSP surveys are now required to apply statistical standards, including

• the North American Industry Classification System (NAICS) to classify the target population by industry
• the North American Product Classification System (NAPCS) to categorize and collect business input and output data
• the Chart of Accounts (COA) as the reference taxonomy for organizing business financial information. (e.g., revenue, expenses, assets and liabilities).

There are a number of financial variables that are common across many economic surveys. By harmonizing the definitions of these variables, and systematically applying standards, common content has been developed and implemented across programs.

The IBSP content model is based on a series of generic modules that cover common variables, and are applied to surveys without modifications from one survey to the next. This approach plays a critical role in creating coherence across programs and in minimizing the effort required to build, test and implement survey content.

In essence, the standardized modules are a series of business survey questions used to collect information to meet stakeholder requirements. There are standardized modules for income-statement data (revenues and expenses), sales data by type of client, sales data by client location, and by purchased service inputs.

The objective of using tax information to its full potential guided the development of questionnaire content. Specifically, IBSP revenue and expense variables have been mapped directly with information available from tax files. This direct link eliminates the need for collecting financial information from small and medium enterprises, since data for these can be easily accessed from administrative sources.

One key issue that had to be resolved in developing financial data content was how to ensure that the conceptual needs of the Canadian System of National Accounts would be met through the use of administrative data. The COA bridges the two sets of concepts. As part of developing the IBSP content model, the COA was reviewed and revised to ensure that COA variables, which are directly linked to tax concepts, meet the information requirements of national accountants.

To add flexibility and meet specific survey requirements, subject-matter experts can customize certain modules that appear on their IBSP questionnaires. For example, products appearing on manufacturing questionnaires will be different from those appearing on service-industry questionnaires. And some of the standardized modules might not be required because they are not relevant for the industry. In constructing IBSP survey questionnaires, staff need to simply select relevant standardized content modules, and then focus efforts on developing industry-specific content, where required. This greatly reduces the time needed to develop, implement and test new questionnaires, while improving the quality and coherence of the data collected.

1.1.3.4 The implementation of rolling estimates and active collection-management based on quality indicators

Another feature added to IBSP is its capacity to use historical and partially collected data to produce key estimates (called rolling estimates) and quality indicators, while collection is still underway. These quality indicators are then compared to previously set quality targets to determine if more effort is required or if active collection can be terminated. If collection needs to continue, item scores are calculated in order to gauge a unit's impact on the quality indicator of each key estimate. These scores are then aggregated within each unit to create a global unit score. These scores enable decisions regarding follow-up activities to be made^{Endnote 3}.

1.1.3.5 An increased use of administrative data to reduce response burden

Statistics Canada's rich history of using administrative data for business statistics has helped to reduce response burden. In fact, under the UES business survey model, tax data have already been used as a direct substitute for a sub-sample of sampled units and for imputation of non-response records.

Over time, tax data imputation methods have improved through the use of administrative data, and the quality of information from surveys has improved. This has led to an even greater reliance on tax data as a primary information source. In the transition to the IBSP model, methods were adapted to take full advantage of tax data availability, and, in turn, will reduce additional response burden across survey programs.

Statistics Canada aims to reduce the time businesses spend responding to surveys—either by reducing the number of surveys and survey questions, by limiting the time that a business can be part of a sample, or by using more efficient data collection methods. The agency is also working to improve relations with respondents through its choice of communications tools. For more details about initiatives undertaken by the agency to better manage respondent burden, including the creation of an Ombudsman for businesses, refer to Chapter 4.5: Relations with survey respondents.