Machine Learning Classifier Evaluation Criteria: application to price statistics - ARCHIVED

Articles and reports: 11-522-X202100100012
Description: The modernization of price statistics by National Statistical Offices (NSO) such as Statistics Canada focuses on the adoption of alternative data sources that include the near-universe of all products sold in the country, a scale that requires machine learning classification of the data. The process of evaluating classifiers to select appropriate ones for production, as well as monitoring classifiers once in production, needs to be based on robust metrics to measure misclassification. As commonly utilized metrics, such as the Fß-score may not take into account key aspects applicable to prices statistics in all cases, such as unequal importance of categories, a careful consideration of the metric space is necessary to select appropriate methods to evaluate classifiers. This working paper provides insight on the metric space applicable to price statistics and proposes an operational framework to evaluate and monitor classifiers, focusing specifically on the needs of the Canadian Consumer Prices Index and demonstrating discussed metrics using a publicly available dataset.

Key Words: Consumer price index; supervised classification; evaluation metrics; taxonomy

Issue Number: 2021001
Author(s): Goussev, Serge; Spackman, William
Main Product: Statistics Canada International Symposium Series: Proceedings
Format Release date More information
PDF November 5, 2021

Related information

Subjects and keywords

Subjects

Keywords

Date modified: