Comparison of Physical Activity Adult Questionnaire results with accelerometer data

View the most recent version.

Archived Content

Information identified as archived is provided for reference, research or recordkeeping purposes. It is not subject to the Government of Canada Web Standards and has not been altered or updated since it was archived. Please "contact us" to request a format other than those available.

Archived

This page has been archived on the Web.

by Didier Garriguet, Sylvain Tremblay and Rachel C. Colley

Physical activity is a key element in health behaviour surveillance programs.^Note 1 In 1994/1995, Statistics Canada adopted the Minnesota Leisure Time Physical Activity Questionnaire (LTPA) in the National Population Health Survey (NPHS). In addition to the nine NPHS cycles (1994/1995 to 2011/2012),^Note 2 this questionnaire was used in the Canadian Community Health Survey (CCHS)^Note 3 from 2001 to 2014 and in the Canadian Health Measures Survey (CHMS) from 2007 to 2011.^{Note 4,Note 5}

In 2007, the CHMS also included a direct measure of physical activity. Respondents were asked to wear an accelerometer (a device that provides time-stamped objective measurement of human movement) on their waist for seven days. Correlation between self-reported leisure-time physical activity and the accelerometer measure of moderate-to-vigorous physical activity (MVPA) was low.^Note 6

Revised Canadian Physical Activity Guidelines were published in 2011.^{Note 7,Note 8} However, the self-reported data collected by the physical activity questionnaire and how the Guidelines specify physical activity differed.^Note 6 For example, the Guidelines refer to a weekly amount of physical activity, whereas the questionnaire pertained to physical activity in the past three months.  As well, the Guidelines specify consecutive minutes of activity accumulated in bouts, but the questionnaire did not.

To address these issues, Statistics Canada modified the physical activity questionnaire used in its health surveys. The goals were to:

• estimate adherence to the Canadian Physical Activity Guidelines^Note 7;
• assess physical activity in multiple domains (work, transportation, leisure time, etc.);
• limit respondent burden with a short questionnaire that can be included in large general health surveys; and
• perform qualitative and quantitative validation.

Before developing a new questionnaire, the possibility of using existing questionnaires was explored. For example, consideration was given to the International Physical Activity Questionnaire (IPAQ), a validated questionnaire used in many countries,^Note 9 and in the 2012/2013 CHMS. However, IPAQ questions do not allow assessment of adherence to the Guidelines for children, and the average time to answer the questionnaire is 15 minutes, which is a high respondent burden.

The Global Physical Activity Questionnaire (GPAQ)^Note 10 was also considered. Again, adherence to the Guidelines cannot be assessed for children, because the GPAQ questions relate to the amount of time spent during a typical day of a typical week (an average), whereas the Guidelines recommend 60 minutes of MVPA every day.

Since existing questionnaires did not meet Statistics Canada’s criteria, two new ones were developed: the Physical Activity Adult Questionnaire (PAAQ) and the Physical Activity Youth Questionnaire (PAYQ). This paper summarizes the qualitative validation process and presents the quantitative validation of the questionnaire by comparing the results of the IPAQ  and the PAAQ with  accelerometer data. The PAYQ also underwent qualitative and quantitative validation, but the limited sample size precludes a proper validation analysis at this time.

Methods

Data source

Data for this study were collected at one CHMS site (Vancouver). Interviews took place in March and April 2013. Ethics approval was obtained from Health Canada’s Research Ethics Board.^Note 11

As well as an in-person household interview to obtain demographic, socioeconomic, health and lifestyle information, the CHMS involved a subsequent visit to a mobile examination center (MEC) for direct physical measures. For the comparative study, the PAAQ was administered to respondents aged 18 to 79 during their MEC visit.

A total of 255 people aged 18 to 79 responded to the household questionnaire and went to the MEC. Of these, 159 wore and returned the accelerometer; 133 answered the PAAQ; and 112 had at least four valid days of accelerometer data.

Self-reported physical activity

The IPAQ,^Note 9 which was part of the CHMS household interview, captures behaviour that occurred the previous week. IPAQ questions pertain to the number of days and the average time spent on a given day doing MVPA in five domains: work, transportation, garden or yard, home, and leisure time. Respondents report activities that last at least 10 consecutive minutes. The test-retest reliability of the IPAQ has been reported at around 0.8, and the validity criterion has been reported at about 0.3 using Spearman correlation.^Note 9

The PAAQ captures behaviour in the seven days before the mobile examination centre visit. Questions pertain to total time spent doing MVPA in three domains: transportation, leisure time and other (includes work, home and volunteering), and how much time was dedicated to vigorous activity (compared with total MVPA) (Appendix A). Respondents were asked to report activities that lasted at least 10 consecutive minutes.

Qualitative testing

The PAAQ, which was largely inspired by the IPAQ and the GPAQ, was reviewed by a panel of Canadian experts in health measurements.

Qualitative testing was then performed by the Statistics Canada Questionnaire Design Research Centre. Testing consisted of cognitive interviews with English and French respondents to assess their understanding of the concepts, terminology, questions and response categories, and their ability and willingness to answer the questions.

In September 2012, a longer version of the questionnaire was tested on 18 respondents in Ottawa. After the first review, content about muscle- and bone-strengthening activities was dropped because participants had difficulty understanding the concept, and the interview was too long.

A second round of testing on a shorter version of the questionnaire took place in March 2013 on 50 respondents in Montreal, Winnipeg and Vancouver. Except for the chores concept, the questions were well understood. This version was added to the CHMS questionnaire for further quantitative testing.

A final round of review was conducted from May 21 to June 3, 2013 in Ottawa and Montreal on 17 respondents to test specific concepts related to the translation of some questions. This led to minimal changes.

Accelerometry data reduction

Upon completion of their MEC visit, ambulatory respondents were asked to wear an Actical accelerometer (Phillips – Respironics, Oregon, USA) on a belt over their right hip during their waking hours for seven consecutive days. The Actical measures and records time-stamped acceleration in all directions, providing an index of movement intensity. The digitized values are summed over a user-specified interval of one minute, resulting in a count value per minute (cpm). The Actical has been validated to measure physical activity in adults and children.^Note 12-15

The accelerometers were initialized to start collecting data in one-minute epochs at midnight following the MEC visit. All data were blind to respondents while they wore the device. Respondents were given a prepaid envelope in which to return the accelerometers to Statistics Canada, where the data were downloaded, and the devices were checked to ensure that they were still within the manufacturer’s calibration specifications.^Note 14

Respondents aged 18 or older with four or more valid days of accelerometer wear time13 were included in this study. A valid day was defined as 10 or more hours of wear-time. Wear-time was determined by subtracting non-wear-time from 24 hours. Non-wear-time was defined as at least 60 consecutive minutes of zero counts, with allowance for two minutes of counts between 0 and 100.^Note 16 For each minute, the movement intensity threshold for MVPA was set at 1,500 cpm.^Note 13 Results are presented for all minutes of MVPA and for MVPA accumulated in bouts of at least 10 minutes. A bout of at least 10 minutes allows for 2 minutes below the threshold. For each respondent, minutes of MVPA were summed for each day and averaged for valid days.

Meeting the Guidelines

The Canadian Physical Activity Guidelines^{Note 7,Note 8} recommend that adults obtain 150 minutes of MVPA per week, accumulated in bouts lasting at least 10 minutes.

The IPAQ collected usual time spent in MVPA on one of the days in the week before the household interview when the respondent was active. This time was averaged for the entire week. The PAAQ collected the total time dedicated to MVPA during the seven days before the MEC visit. Both questionnaires asked about activities that lasted at least 10 minutes. In this study, self-reported time from the two questionnaires and accelerometer-measured MPVA (total and accumulated in bouts of at least 10 minutes) were compared with the recommended 150 minutes a week.

Outlier detection

Of the 112 records with full answers, 4 were removed from the PAAQ analysis because they reported 1,500 minutes or more of physical activity during the week (more than 3.5 hours a day), most of it from transportation (3 records). The final PAAQ sample was 108 records.

For the IPAQ analysis, 18 of the 112 records with full answers were removed because they reported 214 minutes or more of physical activity during the day (more than 3.5 hours). The final IPAQ sample was 94 records.

For some analyses comparing PAAQ and IPAQ results, when outliers were removed from both questionnaires, the final sample size was 90 records.

Statistics

Descriptive statistics were used to present results and mean differences between self-reported minutes of MVPA obtained from the IPAQ, the PAAQ, and accelerometry. Distributions of the difference between IPAQ or PAAQ minutes of MVPA and accelerometer-measured minutes of MVPA are presented in 25-minute intervals (within 12.5 minutes of each other; ±12.5 to 37.5 minutes; ± 37.5 to 62.5 minutes; more (or less) than 62.5 minutes’ difference).

Pearson and Spearman correlations were completed based on each valid day pairing of minutes of MVPA from the IPAQ, the PAAQ, and accelerometery. Respondents were classified as meeting or not meeting the Canadian Physical Activity Guidelines based on self-reported results from the two questionnaires and on the accelerometer measures.

All statistical analyses were performed using SAS v9.2 (SAS Institute, Cary, NC).

Results

The average age of both the PAAQ and the IPAQ samples was 47, and men made up 53% of both groups (Tables 1 and 2). The PAAQ sample was slightly more active, averaging 3 more minutes of accelerometer MVPA than the IPAQ sample. For both groups, self-reported physical activity exceeded the measured value: 45 minutes more for the PAAQ sample; 91 minutes more for the IPAQ sample.

Whether they were compared with all MVPA minutes or MVPA minutes accumulated in bouts of at least 10 minutes, the PAAQ estimates were more highly correlated with the accelerometer results than were the IPAQ estimates:  Pearson correlation of 0.4, compared with less than 0.2 (Table 3). Spearman correlations for the IPAQ data were slightly higher, showing skewness in the estimates, but still below the correlations for the PAAQ data. The Pearson correlation between the IPAQ and PAAQ estimates was 0.375.

The percentage of the sample who met the Canadian Physical Activity Guidelines differed depending on the source of the estimate of MVPA minutes—61% based on PAAQ data and 90% based on IPAQ data, both well above the percentage based on accelerometer MVPA minutes accumulated in bouts of at least 10 minutes—29% (Table 4). However, based on all accelerator MVPA minutes, the percentage of people meeting the Guidelines was 70%, which exceeded the PAAQ estimate (61%).

Misclassification of respondents as meeting or not meeting the Guidelines  between the self-reported PAAQ results and accelerometer data for MVPA accumulated in bouts was substantial:  13% of respondents who met the Guidelines according to the PAAQ results did not according to the accelerometer; 19% who met the Guidelines according to the accelerometer did not according to the PAAQ results (data not shown).

Figure 1 shows the distribution of the difference between accelerometer-measured MVPA (counting all minutes) and minutes of MVPA based on self-reported data from the PAAQ and from the IPAQ. The IPAQ distribution was highly skewed, yielding a higher average number of minutes of difference. The PAAQ distribution was more symmetrical, with 44% of respondents reporting MVPA within 12.5 minutes of the accelerometer estimates. For 11% of respondents, the difference was more than 62.5 minutes, almost all of them positive (higher estimates of MVPA from PAAQ).

Discussion

This analysis tested the ability of the PAAQ to estimate daily minutes of MVPA against a direct measure—accelerometry. The questionnaire went through expert review and qualitative testing and showed test content and response processed validity. The results were also compared with validity tests of a control questionnaire, the IPAQ. The PAA demonstrated reasonable validity against the acceleromter criterion with a correlation coefficient greater than 0.4.

The PAAQ estimates of MVPA were closer to the accelerometer results than were estimates based on the IPAQ. The average number of minutes of MVPA reported was 45 for the PAAQ, 91 for the IPAQ, and 35 for the accelerometer (counting all minutes). The Pearson correlation with accelerometer MVPA was higher for the PAAQ than for the IPAQ: 0.41 versus 0.18 (counting all minutes). The difference between the numbers of minutes was symmetrical and centered at 0 for the PAAQ data, but not for the IPAQ data. The percentage of the sample meeting the Canadian Physical Activity Guidelines based on the PAAQ (61%) was closer to the estimate based on all accelerometer MVPA minutes (70%) than was the percentage derived from the IPAQ (90%). As well, using the PAAQ meant fewer outliers compared with the IPAQ (4 versus 18). When comparisons were made with accelerometer MVPA accumulated in bouts of at least 10 minutes, differences increased. According to accelerometer results, the sample averaged about 15 minutes of MVPA a day, and 29% met the Guidelines.

Most other validity studies of physical activity questionnaires reported Pearson or Spearman correlations around 0.3.^Note 17  In particular, the IPAQ was reported to have a Spearman correlation of 0.3,^Note 9 which was higher than the 0.23 observed in this sample, but lower than what was observed for the PAAQ.

In the present analysis, the PAAQ performed better than the IPAQ. The main difference is that the PAAQ asks respondents to report the total amount of physical activity during the last seven days rather than on just one of those days, which has been noted as a potential source of over-reporting in the IPAQ.^Note 18 For people involved in a variety of activities during the week, the PAAQ would seem to be a better tool.

The first two CHMS cycles and previous Statistics Canada health surveys used the Minnesota LTPA Questionnaire. A study comparing CHMS results with accelerometer data found that the Pearson correlation varied between 0.22 and 0.26, depending on the age group.^Note 6 Average leisure-time physical activity exceeded accelerometer-derived MVPA by 3, 5 and 17 minutes, respectively, among people aged 18 to 39, 40 to 59 and 60 to 79. More than 40% of respondents were classified differently based on LTPA versus accelerometer results. By comparison, PAAQ data were more highly correlated with accelerometer results, overall differences were similar (8 minutes), and 32% of respondents were classified differently according the PAAQ and accelerometer results (based on all MVPA minutes). Preliminary estimates from the PAAQ show the same trend as results from the LTPA Questionnaire, fewer minutes of accelerometer MVPA in older adults compared with younger age groups, and more minutes of self-reported physical activity in older age groups. This discrepancy merits further investigation when sample size allows it.

Limitations

This study has several limitations. The sample was too small to be representative of the entire adult population. As well, people in the sample were more active than the Canadian population overall, based on estimates for 2007-to-2011.^Note 6 However, in the context of a validation study, the range of answers is most important, and was achieved in this sample. Because of the small sample size, vigorous physical activity could not be assessed, although it is part of the Guidelines, and this information is collected in the PAAQ.

The accelerometer data and PAAQ results do not pertain to the same week. Nonetheless, with seven days of data, reliability of more than 0.7 can be expected for the accelerometer results.^Note 19 As well, most self-reported questionnaires, including the IPAQ, show test-retest reliability of 0.8.^Note 17 The PAAQ was inspired by the IPAQ, so similar reliability is expected. In addition, respondents’ recollections may not be precise, and activities differ in any given week; therefore, correlation between self-reported and measured variables is necessarily limited.

Accelerometers cannot accurately measure energy expended during swimming, bicycling and load-bearing activities, which results in a small underestimation of physical activity.^{Note 20,Note 21} Similarly, muscle- and bone-strengthening activities cannot be assessed with the PAAQ, because these questions were excluded after qualitative testing.

As with any self-reported data, a desirability bias could affect respondents’ accounts of physical activity.^Note 22

Another limitation is that the amount of energy expended can differ considerably between self-reports and accelerometer results. For example, respondents may report playing hockey for an hour, but  may accumulate only 20 minutes of measured MVPA because they were not on the ice throughout the game. This inconsistency may also help explain why older adults self-report more activities than younger people, but accumulate fewer accelerometer minutes of MVPA—the time they report being active does not register on an accelerometer because their activities do not reach the intensity threshold.

Conclusion

As of 2014, the PAAQ has been used in Statistics Canada health surveys.  At more than 0.4, the PAAQ’s correlation with accelerometer MVPA surpassed that of most self-reported questionnaires, notably, the two used previously in Statistics Canada health surveys. The absolute difference between PAAQ and accelerometer estimates was small at the population level, and the difference between the percentage of the population meeting the Canadian Physical Activity Guidelines based on PAAQ data and on accelerometer results (counting all minutes) was less than 10%. However, in terms of the specific definition of physical activity in the Guidelines—accumulating at least 150 minutes of MVPA a week in bouts of at least 10 minutes—discrepancies were greater, although the correlation was similar. This may be attributable to inherent differences between self-reports of time spent doing an activity and actual activity levels measured by an accelerometer.

Acknowledgments

The authors acknowledge Dr. Mark Tremblay, Dr. Bertrand Nolin and the Public Health Agency of Canada for their expert review of the questionnaire.