Data quality, concepts and methodology: Survey methodology

View the most recent version.

Foreword

The information in this document is intended primarily to be used by science and technology (S&T) policy makers, both federal and provincial, largely as a basis for interprovincial and intersectoral comparisons. The surveys which generate these statistical estimates also provide input for the development of a national aggregate Research and Development (R&D) series, which are used to populate the Canadian components of international questionnaires for the Organization for Economic Co-operation and Development (OECD) and the United Nations Education, Scientific and Cultural Organization (UNESCO).

These statistical estimates are the aggregates of provincial government science surveys that are collected by individual provinces, and processed by Statistics Canada under contract with the provinces for the period 2005/2006 to 2009/2010. The provincial government sector consists of all provincial government departments, ministries, selected provincial agencies and provincial research organizations (PRO). The PRO are surveyed separately and included in this paper.

In the past, surveys have been conducted in as many as nine provinces, the exception being Prince Edward Island. For this reference period, surveys were conducted by Ontario, Manitoba, Saskatchewan, Alberta and British Columbia. The following ministries or departments are the respective survey sponsors: Ontario Ministry of Research & Innovation; Manitoba Department of Science, Technology, Energy & Mines; Saskatchewan Advanced Education, Employment and Labour; Alberta Advanced Education and Technology and British Columbia Ministry of Jobs, Tourism and Innovation.

2009/2010 reference year data were not obtained from New Brunswick. 2008/2009 reference year data were not obtained from British Columbia. Forecasted figures were used from these provinces' respective 2008/2009 and 2007/2008 survey results. These figures are marked through-out the tables as "use with caution", as they were not reported as final expenditures. As such, caution should be used when comparing these statistics for New Brunswick and British Columbia over time.

The difference in year-over-year expenditures, in Saskatchewan, can be attributed to the differences between the 2007/2008 pilot survey of seven ministries and the 2008/2009 survey of fifteen ministries. 2007/2008 expenditures by Saskatchewan are also marked as "use with caution".

The Institut de la Statistique du Québec conducts a similar survey that collects research and development (R&D) data but does not collect related science activities (RSA) data. For this reason, S&T data are not available for Quebec.

Science surveys, like other surveys, depend on the respondents' understanding of concepts, definitions and methods of calculation. Accounting records are rarely available in formats which use science-based classifications. Extensive efforts are undertaken each year to support provinces in communicating standard explanations of concepts, definitions and calculations to promote statistical coherence and provincial comparability. The same standards are applied to the data of each province as are applied to data of the federal government and all sectors, according to the principles of the OECD Frascati Manual that sets the international standard for the definition and measurement of S&T (R&D and RSA).

Recognizing that survey data are estimates, they still offer a good representation of science expenditures for the provinces. As in any ongoing statistical exercise, revisions will be necessary as definitions and procedures are clarified by respondents.

For Gross Domestic Expenditures on R&D (GERD), no estimates are made for provinces for which there are not corresponding surveys. Gross Domestic Expenditures on Research and Development in Canada and the Provinces, National Estimates 2000 to 2011 Provincial Estimates 2005 to 2009 (Catalogue no. 88-221) are scheduled for publication in the fall of 2011.

Statistics Canada thanks respondents who supported provincial and PRO surveys. Without their invaluable help and cooperation, the production of this report would not have been possible.

History of provincial government science and technology surveys

Prior to 1974, estimates were made for provincial government S&T expenditures using provincial estimates and Public Accounts.

In 1974, Ontario, Alberta and Nova Scotia sought the assistance of Statistics Canada to conduct surveys of S&T spending by their respective governments. In 1975, Saskatchewan joined this group, followed by British Columbia in 1977, Manitoba and New Brunswick in 1984, Newfoundland and Labrador in 1986 and Quebec in 1989.

In 1993/1994, three provinces, Newfoundland and Labrador, New Brunswick and Nova Scotia, did not contract surveys with Statistics Canada due to budget constraints. In 1994/1995, the province of Quebec first began collecting only R&D expenditures instead of total S&T. In 2001/2002 Saskatchewan did not contract a survey with Statistics Canada.

Saskatchewan conducted a limited pilot of 7 ministries when collecting data for reference year 2007/2008. For 2008/2009 the survey included 15 ministries. This difference in survey coverage contributes to the year-over-year expenditure and personnel changes.

In 2008/2009, British Columbia did not contract Statistics Canada to conduct a survey, but they returned to the survey activity in 2009/2010. In 2006/2007, the province of Newfoundland and Labrador participated in the survey. In 2008/2009, New Brunswick successfully completed collection for three fiscal years 2007/2008, 2008/2009 and 2009/2010. We are pleased to announce the participation of Prince Edward Island, Ontario, Manitoba, Saskatchewan, Alberta and British Columbia in the Provincial Scientific Activities Survey for the 2010/2011 reference year.

Provincial research organizations

All of these organizations have been established by their respective provincial and territorial governments, with a variety of enabling legislation and powers, to:

1. provide technical support to primary and secondary industries;
2. assist in the exploitation of provincial and territorial natural resources; and
3. enhance the economy of their provinces and territories.

Small and medium-sized companies with limited in-house technical capability use the services of these provincial research organizations.

The questionnaire for the Scientific Activities of Provincial Research Organizations: Activities in the natural sciences and engineering was redesigned for reference year 2009.

Federal / provincial workshops on scientific and technological (S&T) statistics

In the fall of 1977, the first federal-provincial meeting was held in Ottawa. Representatives from British Columbia, Alberta, Saskatchewan, Ontario and Nova Scotia attended; as well as Statistics Canada and members of the Ministry of State for Science and Technology (MOSST).

The next meeting was held in 1984 with representatives from British Columbia, Alberta, Saskatchewan, Manitoba, Ontario, Quebec and New Brunswick attending. Statistics Canada sponsored the meeting and invited representatives from MOSST, Energy, Mines and Resources (EMR) and the Science Council. The objectives of the conference were to:

1. Provide provincial science policy and statistical users with an overview of products and services of the Science and Technology Statistics Division (STSD);
2. Provide a forum to allow discussion between STSD and provincial representatives to exchange views on science statistics; and
3. Achieve consensus on how to proceed with future provincial surveys.

In 1999, Ontario proposed that Statistics Canada renew federal/provincial conferences as an annual event. Statistics Canada agreed and co-hosted the 1999 conference in Toronto. The agenda included topics such as innovation surveys, biotechnology surveys, intellectual properties in higher education, e-commerce and provincial needs and proposals.

Quebec and Statistics Canada co-hosted the 2000 conference held in Québec City. Discussions included economic indicators, an innovation study for Ontario, and biotechnology measurement.

In the fall of 2001, British Columbia and Statistics Canada co-hosted the conference in Victoria. Provincial representatives discussed high technology indicators, innovation index, and user needs and challenges. Statistics Canada presented an overview of current program developments and future plans.

Alberta and Statistics Canada co-hosted the 2002 conference held in Edmonton. Discussions included provincial indicators and an overview of current program developments and future plans.

In the fall of 2003, Statistics Canada did not host the 5^th annual conference in Ottawa, which was postponed due to budget constraints. The conference has not been re-instated.

Definitions

This report covers those scientific and technological activities which involve the generation, dissemination and application of new scientific and technological knowledge. The central activity is research and experimental development (R&D). In addition, there are a number of activities closely related to R&D; these are termed related scientific activities (RSA).

R&D is creative work undertaken on a systematic basis in order to increase the stock of scientific and technical knowledge, including knowledge of humans, culture and society and the use of this stock of knowledge to devise new applications.

It requires the acquisition of knowledge and not just information. New knowledge involves the integration of newly acquired information into existing hypotheses or the re-evaluation of existing observations.

The major related scientific activities are education support, technical surveys, statistical surveys, information services, special services and studies, and museum services. Education support and museum services are largely self-explanatory.

Technical surveys are activities directed towards exploration and systematic description of the earth and its natural resources. The activities include gathering, processing, collating and analyzing of data on natural phenomena except when part of a research project or a museum service. The preparation of maps and survey reports, their printing and cataloguing, are also included.

Statistical surveys are activities directed toward the collecting, processing and disseminating of statistics on humankind, their economic and social activities. Included are the development of technical methodology, statistical analysis and vital statistics.

Information services are all work directed to recording, classifying, translating, and disseminating science and technology information. Included are the operations of specialized libraries and archives, the publication of scholarly journals and bibliographies, and the organizing of scientific conferences. Grants for the publication of scholarly works are also included.

Special services and studies in the natural sciences are activities directed towards the establishment of national and provincial standards for materials, devices, products and processes; the calibration of secondary standards; non-routine quality testing; feasibility studies and demonstration projects.

In the social sciences, special services and studies are systematic investigations carried out in order to provide information needed for planning or policy formulation, including feasibility studies and demonstration projects.

Scientific and technological activities take place in both natural sciences and engineering and the social sciences and humanities. The natural sciences and engineering consist of disciplines concerned with understanding, exploring, developing or utilizing the natural world. The social sciences and humanities embrace all disciplines involving the study of human actions and conditions and the social, economic and institutional mechanisms affecting humans.

Six performing sectors are identified

1. Intramural refers to the provincial ministry, department or agency performing a scientific activity.
2. Business enterprise denotes largely private corporations but also includes crown corporations with a commercial function (e.g., power utilities) and industrial research institutes not controlled by another institution.
3. Higher education sector covers post secondary educational institutions and affiliated teaching and research facilities.
4. Hospitals and health organizations – Canadian hospitals and health organizations which are not part of university medical schools, as well as private non-profit organizations related to health.
5. Provincial research organizations include: New Brunswick Research and Productivity Council, Centre de recherche industriel du Québec, Industrial Technology Centre (Manitoba), Saskatchewan Research Council, Northern Research Institute, Nunavut Research Institute, Aurora Research Institute (Aurora College N.W.T.)
6. Other includes the federal government, municipal governments, individuals, institutions not identified with any other sector, and foreign performers.

Departmental personnel are classified into three major categories. Scientific and professional includes persons in a job requiring at least one academic degree or nationally recognized professional qualification. The Technical category includes people in jobs requiring specialized vocational or technical training beyond the secondary level. Other includes clerical, secretarial, administrative, operational and other support personnel. Personnel data are reported in full-time equivalent which is simply the portion of a person's time spent on S&T activities.

Objectives of expenditures on scientific activities

The objectives listed in this survey do not represent the total range of possible objectives; however, they are intended to cover the major areas of current technological interest. Respondents are asked to report expenditures under the objective which is primary to that expenditure. The following list of examples although not exhaustive was presented in the survey guide to assist respondents.

Socio-economic objectives allow departments/ministries/agencies to classify their S&T resource allocations according to the purpose for which the expenditure is intended. The objectives are listed on the questionnaire at the highest level of aggregation with sub-levels given here for clarification of categories. In many cases, projects have multiple objectives and a department should assign its expenditures consistent with the stated objectives of the department. Care must be taken to avoid "double counting".

Values are identified by either R&D or RSA, and by either intramural performers or extramural performers.

The objectives are based on the Nomenclature for the Analysis and Comparison of Scientific Programs and Budgets (NABS) produced by the Statistical Office of the European Communities (Eurostat).

1. Exploration and exploitation of the earth - scientific activities with objectives related to the exploration of the earth's crust and mantle, seas, oceans and atmosphere, and scientific activities on their utilization. It also includes climatic and meteorological research (e.g., drought management and the analysis of precipitation standards), polar exploration (under various headings, as appropriate) and hydrology.

1. General scientific activities
2. Mineral, oil and natural gas prospecting
3. Exploration and exploitation of the sea-bed
4. Earth's crust and mantle excluding sea-bed and studies of soil for agriculture (6)
5. Hydrology - excludes scientific activities on: water supplied and disposal (2) and water pollution (3)
6. Sea and oceans
7. Climatic and meteorological services
8. Atmosphere
9. Other scientific activities on the exploration and exploitation of the earth

Excludes: scientific activities on pollution, soil improvement, land-use and fishing.

2. Infrastructure and general planning of land use - scientific activities on infrastructure and land development, including research on the construction of buildings. More generally, it covers all scientific activities relating to the general planning of land use. This includes scientific activities into protection against harmful effects in town and country planning but not scientific activities into other types of pollution.

2.1 Transportation systems – covers scientific activities on transport systems, including road accident prevention and ancillary services such as electronic traffic aids and radar stations. Also included is general scientific activities on transport systems, road and rail traffic, inland waterway and sea transport, air traffic, pipeline transport systems, works transport systems, combined transport systems and scientific activities on the potential effects on the environment of the planning and operation of transport systems. Scientific activities on transport equipment is included only when it forms part of the co-ordinated programs for the development of improved and safer transport systems, otherwise, such research is classified in Objective 9.

2.2 Telecommunication systems – covers scientific activities on telecommunications services and the planning and organization of telecommunications networks. It includes, in particular, general scientific activities on telecommunications systems, telephones, telex, data transmission, radio and television (including cable TV).

2.3 Other scientific activities on the infrastructure and general planning of Land use i.e.

1. General scientific activities
2. General planning of land use
3. Construction and planning of building
4. Civil engineering - excludes scientific activities on building materials and industrial processes (objective 9)"
5. Water supply

3. Control and care of the environment – covers scientific activities into the control of pollution, aimed at the identification and analysis of the sources of pollution and their causes, and all pollutants, including their dispersal in the environment and the effects on man, species (fauna, flora, micro organisms) and biosphere. Development of monitoring facilities for the measurement of all kinds of pollution is included. The same is valid for the elimination and prevention of all forms of pollution in all types of environment.

1. General scientific activities on the environment
2. Protection of atmosphere and climate
3. Protection of ambient air
4. Solid waste
5. Protection of ambient water
6. Protection of soil and groundwater
7. Noise and vibration
8. Protection of species and habitats
9. Protection against natural hazards
10. Radioactive pollution
11. Other scientific activities on the environment

Protection and improvement of human health - scientific activities aimed at protecting, promoting and restoring human health broadly interpreted to include health aspects of nutrition and food hygiene.It ranges from preventative medicine, including all aspects of medical and surgical treatment, both for individuals and groups, and the provision of hospital and home care, to social medicine and paediatric and geriatric research.

1. General scientific activities
2. Medical scientific activities, hospital treatment, surgery
3. Preventative medicine
4. Biomedical engineering and medicines
5. Occupational medicine
6. Nutrition and food hygiene
7. Drug abuse and addiction
8. Social medicine
9. Hospital structure and organization of medical care
10. Other medical scientific activities

Production, distribution and rational utilization of energy - covers scientific activities into the production, storage, transportation, distribution and rational use of all forms of energy. It also includes scientific activities on processes designed to increase the efficiency of energy production and distribution, and the study of energy conservation.

1. General scientific activities
2. Fossil fuels and their derivatives
3. Nuclear fission
4. Radioactive waste management including decommissioning with regard to fuel/energy
5. Hydroelectric energy
6. Nuclear fusion
7. Conservation
8. Renewable energy sources
9. Rational utilization of energy
10. Other scientific activities on production, distribution and rational utilization of energy

6. Agricultural production and technology - covers scientific activities on animal products, veterinary medicine, crops, agricultural technology, agricultural biotechnology and other scientific activities on agricultural production and technology. It includes: scientific activities on chemical fertilizers, pesticides, insecticides, herbicides, biological pest control and the mechanization of agriculture; evaluation of the impact of scientific activities promoting productivity and technology in agriculture.

7. Fishing – covers scientific activities on fishing, salting, drying and initial freezing of products (but not on preparation and canning; objective 9). Scientific activities on fish-farming, exploration of new fishing grounds, exploration and development of new and unconventional sources of seafood.

8. Forestry – covers scientific activities into the ecological and economic aspects of forestry and timber production.

9. Industrial production and technology - covers scientific activities on the improvement of industrial production and technology. It includes scientific activities on industrial products and their manufacturing processes except where they form an integral part of the pursuit of other objectives (e.g. energy, agriculture, fishing, forestry).

1. General scientific activities
2. Increasing economic efficiency and competitiveness
3. Manufacturing and processing techniques
4. Extraction and processing of non-energy minerals and derived products
5. Products of the chemical industry
6. Petrochemical and coal by-products
7. Pharmaceutical products
8. Manufacture of motor vehicles and other means of transport
9. Aerospace equipment manufacturing and repairing
10. Manufacture of motor vehicles and parts
11. Manufacture of other modes of transportation
12. Electronic and related industries
13. Manufacture of office machinery and data-processing equipment
14. Manufacture of radio, television and communications equipment and apparatus
15. Software development
16. Manufacture of electrical machinery and apparatus
17. Manufacture of non-electronic and non-electrical machinery
18. Manufacture of instruments
19. Manufacture of medical and surgical equipment and orthopaedic appliances
20. Manufacture of food products and beverages
21. Manufacture of clothing and textiles and leather goods
22. All other manufacturing products
23. Recycling

10. Social structures and relationships – scientific activities on social objectives, as analysed in particular by social and human sciences, which have no obvious connection with other objectives.This analysis includes quantitative, qualitative, organizational and forecasting aspects of social problems.

1. General scientific activities
2. Education, training, recurrent education and retraining
3. Cultural activities, sport and recreation
4. Human resources
5. Management of businesses and institutions
6. Improvement of working conditions
7. Social security system
8. Political structure of society
9. Social change, social processes and social conflicts
10. Urban and regional studies
11. Other scientific activities with regard to society

11. Exploration and exploitation of space - all civil space scientific activities.Although civil space research is not, in general, concerned with particular objectives, it frequently has a specific goal, such as the increase of general knowledge (e.g., astronomy), or relates to particular applications (e.g., telecommunications satellites).

1. General scientific activities
2. Applied research programs
3. Other research on the exploration and exploitation of space

12. Basic research (advancement of science) - basic activities motivated by scientific curiosity with the objective of increasing scientific knowledge. It also includes funding used to support postgraduate studies and fellowships.

1. Mathematics and computer sciences
2. Physical sciences
3. Chemical sciences
4. Biological sciences
5. Earth and related (environmental) sciences
6. Engineering sciences
7. Medical sciences
8. Agricultural sciences
9. Social sciences
10. Humanities

13. Other civil research - civil scientific activities which cannot (yet) be classified to a particular objective.