Analytical Studies: Methods and References
Zeno: A Tool for Calculating Confidence Intervals of Rates in Health Analytical Studies: Methods and References
Zeno: A Tool for Calculating Confidence Intervals of Rates in Health

View the most recent version.

by Philippe Finès and Gisèle Carriere
Health Analysis Division, Statistics Canada

Acknowledgements

The authors thank Claude Nadeau for his comments on a preliminary version of this paper.

Abstract

Hospitalization rates are among commonly reported statistics related to health-care service use. The variety of methods for calculating confidence intervals for these and other health-related rates suggests a need to classify, compare and evaluate these methods. Zeno is a tool developed to calculate confidence intervals of rates based on several formulas available in the literature. This report describes the contents of the main sheet of the Zeno Tool and indicates which formulas are appropriate, based on users’ assumptions and scope of analysis.

Keywords: Confidence intervals, health, hospitalization, rates

1. Introduction

Hospitalization rates are among commonly reported statistics related to health-care service use. A recurrent problem encountered by health analysts is the choice of a method to compute confidence intervals (CIs) for these rates.

When hospital events are independent of one another (only one event is allowed per person; for example, a discharge) and when the rate is neither extremely low nor high, the CI can be computed using an approximation based on the normal distribution. However, circumstances different from these conditions often exist. For example, unlike vital events such as births or deaths, for a given person, repeat events may occur, even if they have a low probability of occurrence. These events are not independent, a situation that has implications for the validity of using approximations based on the normal distribution to calculate. Also, an examination of an individual’s hospital visits might consider all visits, or only the first visit for a specific disease. Depending on the scope of analysis, different assumptions are made, and therefore, different methods are needed to calculate CIs.

When only one event per person is examined and the probability of occurrence of the event is low, exact calculations based on specific distributions are recommended. Frequently, a Binomial or Poisson distribution is assumed. Other techniques have been proposed (Glynn et al. 1993; Carriere and Roos 1997; Fay and Feuer 1997; Kegler 2007), which assume specific distributions such as Gamma or Chi-square. A second issue, in addition to rare events, is that analyses of recurrent events (such as hospitalizations) as opposed to non-recurrent events (such as death) violate the Poisson assumption of independence (Carriere and Roos 1997).

The literature on CIs for rates that measure these different circumstances has expanded rapidly and can be daunting for researchers who must decide which formula to use in a specific context. However, few comparative studies of methods of calculating CIs for rates have been conducted. Typically, authors report their work (with their idiosyncratic notations) with little discussion of compatibility and comparability with the work of other researchers.

The objective of a previous version of this article was to catalogue all the methods available to compute CIs for rates. The challenges were numerous and included the reconciliation of notations between authors and the need to ensure that all pertinent literature had been taken into account. The task rapidly became unwieldy. Some papers described exact methods, while others presented approximations; some described how to compare rates (with rate differences or rate ratios); some focused on one event per person, whereas others allowed for multiple occurrences; some described formulas for crude rates, while others focused on standardized rates, etc. A systematic review of published formulas was needed. It became apparent that regrouping a family of formulas related to the general problem of calculating CIs for rates into a single tool would be useful.

Therefore, the original objective shifted from the development of a catalogue of formulas to the development of a tool that enables researchers to view the effects of applying one or another of several different methods. It addresses the calculation of CIs for rates, but omits formulas for hypothesis testing on rates (such as comparison between groups, or assessment of over-dispersion or zero-inflated distributions [van den Broek 1995]). This tool (worksheet), Zeno, is now available upon request. Since rates and number of events are related, CIs for both metrics are available. Although users have access to several formulas for CIs, the most appropriate formula is not necessarily the one that yields the narrowest CI. On the contrary, the most appropriate formula is the one that satisfies the conditions of assumptions used to describe event distribution; calculations are derived based on these assumptions.

The objective of this article is to describe the Zeno Tool. The next three sections present: the references for the original formulas (Section 2); notations used in the Tool and in this article (Section 3); and the contents of the main sheet of the Tool (Section 4). The Data and results section (Section 5) contains a pivot table extracted from the Tool. The Conclusion (Section 6) summarizes the description and suggests potential enhancements.

2. Literature review

Although a systematic review of all papers on this domain was not conducted, the references used are up-to-date and pertinent for the purpose of the Tool. The corollary is that, as mentioned before, notation varies in all references. To appreciate the intricacies of a given method and compare it with another, readers must mentally translate the notation used by one author into the terminology used by the other. For example, the concept of weight has surprisingly diverse definitions. The assumptions of the formulas selected for inclusion in the Tool are listed in Table 1. When the proportion of recurrent cases is relatively small, all the measures identified as “One event per person” can be used for the “All events” analyses.

3. Notations and formulas

The main concepts and their symbols are listed in Table 2. When all events are considered, rates are denoted by $r$, and numbers of events are denoted by $\#E$. For age-standardized outcomes, the symbols become $ASR$ and $AS\#E$. Within an age-stratum $i$, suffix $\_i$ is appended to the symbol. When only one event per person is considered (for example, the first event), the same notations are bracketed by “[“ and “]”, so that $r$ becomes $\left[r\right]$, $ASR$ becomes $\left[ASR\right]$, etc. In addition, other symbols are needed: $\#N\left(\#N\_i\right)$ is the total (stratum-specific) size of the population under study; and $\#NR\left(\#NR\_i\right)$ is the total (stratum-specific) size of the reference population, from which weights $w\_i=\#NR\_i/\#NR$ are computed.

In cases where figures represent both single events for some persons and recurrent, non-independent events for others, a more complicated notation is needed. The notation must be able to describe the contents of cells that contain values that encode the number of persons, and also, the number of events experienced per given person. Using notation from Kegler (2007), $C\_k=h$ is the number of persons for whom the number of events was equal to $h$. In Kegler (2007), $h$ ranges from 1 to 2; in the Tool presented here, $h$ ranges from 1 to 14. Thus, $\#\left\{n:ninstratumiandC\_k\left(n\right)=j\right\}$ reads “the size $[\#]$ of the set $\left[\left\{\mathrm{..}\right\}\right]$ made up of all the persons $n$ who $\left[n:\right]$ belong to stratum $i$ and for whom the number of events is equal to $j\left[C\_k\left(n\right)=j\right]$.” In other words, the formula expresses the number of persons in stratum $i$ who had $j$ events. For consistency with the other notations in Table 2, $\#E\_j\_i$ denotes $\#\left\{n:ninstratumiandC\_k\left(n\right)=j\right\}$; by extension, $\#E\_0\_i$ denotes the number of persons in stratum $i$ with no event, and $\#E\_0$ denotes the number of persons overall with no event.

Results can be defined according to the intended metrics, namely, the rates or the number of events based on the population (specific age strata or totals) and the scope (all events or only one event per person). Rates and number of events are related according to the general formula:

$$Rate=Numberofevents/Sizeofpopulation$$

which, using the notations of Table 2, gives, for example:

$$r=\#E/\#N$$

and

$$ASR=AS\#E/\#N.$$

Throughout, population sizes are considered fixed, as this is standard for several authors. Therefore, the rate or number of events can be calculated when the other metric is known. In fact, some of the formulas implemented in the Tool were initially introduced in referenced material to calculate the CIs for rates only (or number of events). The formulas corresponding to the other metric were then developed.

Although several of the referenced formulas focus on a specific calculation, it is possible to expand the formulas to apply to other cases. For example, from references that present the formulas for multiple visits, simplification to one visit per person is straightforward―only minor modifications of the original formulas are needed.

Also, an age-standardized rate is a weighted sum of the age stratum-specific rates.

$$ASR=\sum iw\_i*r\_i$$

(where the weights $w\_i$ are equal to $\#NR\_i/\#NR$ ). If in this formula, instead of weights $w\_i$, “pseudo-weights” $w’\_i$ (each equal to $\#N\_i/\#N$ ), are used, the result is:

$$\sum iw’\_i*r\_i=\sum i\#N\_i/\#N*\#E\_i/\#N\_i=\sum i\#E\_i/\#N=\#E/\#N=r.$$

This shows that $r$ (crude rate) can be expressed as a weighted sum of stratum-specific rates; in other words, $r$ is the age-standardized rate obtained when using “pseudo-weights.” This property has been used to convert the formula for CI of $ASR$ into a formula for CI of $r$ when the latter was not directly provided in the references.

To summarize, the Tool contains a complete series of formulas for the six metrics in the last two columns of Table 2, using the different assumptions in Table 1.

4. Contents of main sheet

The Zeno Tool is essentially a sheet (“main sheet”) that contains the data and the results. In fact, there may be as many “main sheets” as desired; the label of the sheet being analyzed must be indicated in sheet “Prep,” which reformats the results to produce the appropriate pivot charts and tables.

Cells of a main sheet are denoted by concatenation of column (letter) and row (number). The Excel sheet allows for 21 strata, for which data will be entered by the user on rows $i*$ =11 to 31.

As mentioned earlier, stratum-specific symbols are denoted by concatenation of symbol, “ $\_$ ” and row $i$ =1 to 21. Rows and strata are linked by this relation: $i*=i+10$. The essential components of any main sheet are described in Tables 3-1 to 3-4, where the identifier of the cell is described by what follows the “=” sign. For example: cell D11 contains the number of events for stratum 1; $r\_1$ is the rate for stratum 1. The only cells to be modified by users are:

• C1 and D1 to T1 (keyword and title)
• C2: unit of reference (denominator) for rates
• C3: Alpha level of CIs (for example, 0.05)
• area D11 to T31: respectively, for each age stratum $i$ =1 to 21: $\#E\_i,\#N\_i,\#NR\_i,\#E\_j\_i$ for $j$ =1 up to 14 (because up to 14 events per person are allowed in the Tool).

5. Data and results

The data used to test the Tool were originally gathered from administrative cancer databases in Manitoba. These data were chosen to illustrate the value of the Tool and to demonstrate a situation in which interest may focus on all the events (total rate of hospitalization for cancer including recurrences) or on only one event per person (rate of initial hospitalization for cancer among residents of the province).

Table 4 gives a sample of the results. Users can choose the significance threshold (for example, 95%, 90%)^Note 1 of the CIs and specify the metrics (for example, $r$, $ASR$, $\left[r\right]$, $\left[ASR\right]$ ), the statistics (estimate, lower [L] and upper [U] limits of CIs), the methods (formulas in Table 1), as well as whether these are required globally or for specific strata. Users must be aware of the appropriateness of the method based on the scope, as shown in Table 1.

6. Conclusion

The Zeno Tool regroups and extends formulas from several published sources. It presents, on a single sheet, the calculations proposed by different authors and allows users to compare the impact of using various methods on the resulting confidence intervals (CIs). The Tool also expands the referenced formulas. For example, in one reference, the CI for $ASR$ might be present, but not the CI for $r$; in another reference, the CI for $r$ might be present, but not the CI for $\left[r\right]$; or formulas for $r$ may exist, but not those for $\#E$. Thus, the Zeno Tool completes the set of formulas available for use in a broader range of circumstances that may exist in the data.

The Tool could be expanded to include features such as comparisons between groups, or tests on over-dispersion or zero-inflated distributions. Implementation of modelling of rates could also be considered. However, caution should be exercised before introducing additional features. While these features would be useful, ease of use could be compromised. The Tool might become cumbersome because additional analyses, such as hypothesis tests, would require more columns or rows, which might not be needed in all situations. As presented, this generalized tool is applicable to a broad array of circumstances, yet flexible and adaptable to suit other specific needs.

References

Carriere, K.C., and L.L. Roos. 1997. “A Method of Comparison for Standardized Rates of Low-Incidence Events.” Medical Care 35 (1): 57–69.

Daly, L. 1992. “Simple SAS macros for the calculation of exact binomial and Poisson confidence limits.” Computers in Biology and Medicine 22 (5): 351–361.

Fay, M.P., and E.J. Feuer. 1997. “Confidence intervals for directly standardized rates: A method based on the Gamma distribution.” Statistics in Medicine 16 (7): 791–801.

Fleiss, J.L. 1981. Statistical Methods for Rates and Proportions, 2nd Edition. New York: John Wiley and Sons Ltd.

Glynn, R.J., T.A. Stukel, S.M. Sharp, T.A. Bubolz, J.A. Freeman, and E.S. Fisher. 1993. “Estimating the Variance of Standardized Rates of Recurrent Events, with Application to Hospitalizations Among the Elderly in New England.” American Journal of Epidemiology 137 (7): 776–786.

Kegler, S.R. 2007. “Applying the compound Poisson process model to the reporting of injury-related mortality rates.” Epidemiologic Perspectives & Innovations 4 (1). DOI: 10.1186/1742-5573-4-1.

van den Broek, J. 1995. “A score test for zero inflation in a Poisson distribution.” Biometrics 51 (2): 738–743.