A short note on quantile and expectile estimation in unequal probability samples 2. Quantile estimationA short note on quantile and expectile estimation in unequal probability samples 2. Quantile estimation

• Previous
• Next

We consider a finite population with $N$  elements and a continuous survey variable $Y.$  We are interested in quantiles of the cumulative distribution function $F\left(y\right)\mathrm{<mo>=</mo>}{\displaystyle {\sum }_{i\mathrm{<mo>=</mo>\; 1}}^N}1\left\{Y_i\le y\right\}/N$  and define as

$$Q\left(\alpha \right)\mathrm{<mo>= </mo>}\text{inf}\left\{\arg \underset{q}{\min }{\displaystyle \sum _{i\mathrm{<mo>=</mo>\; 1}}^N}{w}_{\alpha }\left(Y_i-q\right)\left|Y_i-q\right|\right\}(2.1)$$

the Quantile function of $Y$ (see Koenker 2005), where

$$w_{\alpha }\left(\epsilon \right)\mathrm{<mo>=</mo>}(\begin{array}{ll}\alpha \hfill & \text{for}\epsilon \mathrm{<mo>></mo>\; 0}\hfill \\ 1-\alpha \hfill & \text{for}\epsilon \le 0.\hfill \end{array}$$

The “inf” argument in (2.1) is required in finite populations since the “arg min” is not unique. We draw a sample from the population with known inclusion probabilities ${\pi }_i,$ $i\mathrm{<mo>=</mo>\; 1,}\dots \mathrm{,}N.$ Denoting by $y_1\mathrm{,}\dots \mathrm{,}{y}_n$ the resulting sample, we estimate the quantile function by replacing (2.1) through its weighted sample version

$${\widehat{Q}}_N\left(\alpha \right)\mathrm{<mo>=</mo>}\inf \left\{\arg \underset{q}{\min }{\displaystyle \sum _{j\mathrm{<mo>=</mo>\; 1}}^n}\frac{1}{{\pi }_j}{w}_{\alpha \mathrm{,}j}\left|y_j-q\right|\right\}(2.2)$$

with $w_{\alpha \mathrm{,}j}\mathrm{<mo>=</mo>}{w}_{\alpha }\left(y_j-q\right)$ as defined above. It is easy to see that the sum in (2.2) is a design-unbiased estimate for the sum in $Q\left(\alpha \right)$ given in (2.1). Nonetheless, because we take the “arg min” it follows that ${\widehat{Q}}_N\left(\alpha \right)$ is not unbiased for $Q\left(\alpha \right).$ We therefore look at consistency statements for ${\widehat{Q}}_N\left(\alpha \right)$ as follows. Let $R_i\left(q\right)\mathrm{<mo>=</mo>}{w}_{\alpha }\left(y_i-q\right)\left|y_i-q\right|$ and

$${\overline{R}}_N\left(q\right)\mathrm{<mo>:=</mo>}\frac{1}{N}{\displaystyle \sum _i}{R}_i\left(q\right)\mathrm{.}$$

We draw a sample from $R_i\left(q\right)\mathrm{,}i\mathrm{<mo>=</mo>\; 1,}\dots \mathrm{,}N$ and assume we apply a consistent sampling scheme in that

$${\overline{r}}_n\left(q\right)\mathrm{<mo>:=</mo>}\frac{1}{N}{\displaystyle \sum _{j\mathrm{<mo>=</mo>\; 1}}^n}\frac{1}{{\pi }_j}{r}_j\left(q\right)$$

is design-consistent for ${\overline{R}}_N\left(q\right),$ where $r_j\left(q\right)$ denotes the sample of $R_i\left(q\right).$ Note that $r_j\left(q\right)$ and hence ${\overline{r}}_n\left(q\right),$ $R_i\left(q\right)$ and ${\overline{R}}_N\left(q\right)$ also depend on $\alpha$ which is suppressed in the notation for readability. Let $q_0$ be the minimum of ${\overline{R}}_N\left(q\right)$ which is not necessarily unique due to the finite structure of the population. We can take the “inf” argument, i.e., $q_0\mathrm{<mo>=</mo>}\inf \left\{\text{arg}\min {\overline{R}}_N\left(q\right)\right\},$ but for simplicity we assume a superpopulation model (see Isaki and Fuller 1982) by considering the finite population to be a sample from an infinite superpopulation. In the latter we assume that survey variable $Y$ has a continuous cumulative distribution function so $q_0$ results in a unique $\alpha$ quantile. We get for $\delta \mathrm{<mo>></mo>\; 0}$

$$P\left({\overline{r}}_n\left(q_0\right)\mathrm{<mo><</mo>}{\overline{r}}_n\left(q_0-\delta \right)\right)\iff P\left(\frac{1}{N}{\displaystyle \sum _{j\mathrm{<mo>=</mo>\; 1}}^n}\frac{1}{{\pi }_j}\left\{r_j\left(q_0\right)-r_j\left(q_0-\delta \right)\right\}\mathrm{<mo><</mo>\; 0}\right)\mathrm{.}$$

Note that the argument in the probability statement is a design-consistent estimate for ${\overline{R}}_N\left(q_0\right)-{\overline{R}}_N\left(q_0-\delta \right),$ which is less than zero since $q_0$ is the minimum of ${\overline{R}}_N(\cdot ).$ Hence, the probability tends to one in the sense of design consistency defined in Isaki and Fuller (1982). The same holds of course for $\delta \mathrm{<mo><</mo>\; 0.}$ With this statement we may conclude that the estimated minimum ${\widehat{q}}_0\mathrm{<mo>=</mo>}\text{arg}\min {\displaystyle {\sum }_{j\mathrm{<mo>=</mo>\; 1}}^n}1/{\pi }_j{r}_j\left(q\right)$ is a design-consistent estimate for $q_0$ so that ${\widehat{Q}}_N\left(\alpha \right)$ in (2.2) is in turn design-consistent for $Q_N\left(\alpha \right).$ It is easily shown that ${\widehat{Q}}_N\left(\alpha \right)$ is the inverse of the normed weighted cumulative distribution function

$${\widehat{F}}_N\left(y\right)\mathrm{<mo>:=</mo>}\frac{{\displaystyle \sum _{j\mathrm{<mo>=</mo>\; 1}}^n}1\left\{y_j\le y\right\}/{\pi }_j}{{\displaystyle \sum _{j\mathrm{<mo>=</mo>\; 1}}^n}1/{\pi }_j}$$

using the same notation as in Kuk (1988). Note that ${\widehat{F}}_N\left(y\right)$ is the Hajek (1971) estimate of the cumulative distribution function (see also Rao and Wu 2009) and as such not a Horvitz-Thompson estimate. As a consequence ${\widehat{Q}}_N\left(\alpha \right)$ is not design-unbiased. Nonetheless, ${\widehat{F}}_N\left(y\right)$ is a valid distribution function, and hence it can be considered as normalized version of the Lahiri or Horvitz-Thompson estimator of the distribution function (see Lahiri 1951) which is denoted by

$${\widehat{F}}_L\left(y\right)\mathrm{<mo>:=</mo>}\frac{1}{N}{\displaystyle \sum _{j\mathrm{<mo>=</mo>\; 1}}^n}1/{\pi }_{j}1\left\{y_j\le y\right\}.$$

Kuk (1988) proposes to replace ${\widehat{F}}_L(\cdot )$ with alternative estimates of the distribution function: Instead of estimating the distribution function itself he suggests to estimate the complementary proportion ${\widehat{S}}_R\left(y\right)$ which then leads to the estimate ${\widehat{F}}_R\left(y\right)$ defined through

$${\widehat{F}}_R\left(y\right)\mathrm{<mo>=</mo>\; 1}-{\widehat{S}}_R\left(y\right)\mathrm{<mo>=</mo>\; 1}-\frac{1}{N}{\displaystyle \sum _{j\mathrm{<mo>=</mo>\; 1}}^n}1/{\pi }_{j}1\left\{y_j\mathrm{<mo>></mo>}y\right\}\mathrm{.}$$

Resulting directly from these definitions we can express ${\widehat{F}}_R(\cdot )$ in terms of ${\widehat{F}}_N(\cdot )$ through

$${\widehat{F}}_R\mathrm{<mo>=</mo>\; 1}-\frac{1}{N}{\displaystyle \sum _{j\mathrm{<mo>=</mo>\; 1}}^n}1/{\pi }_j+{\widehat{F}}_L\text{and}{\widehat{F}}_L\mathrm{<mo>=</mo>}\frac{{\displaystyle \sum _{j\mathrm{<mo>=</mo>\; 1}}^n}1/{\pi }_j}{N}{\widehat{F}}_N\mathrm{.}(2.3)$$

Kuk (1988) shows that, under sampling with unequal probabilities, estimation of the median derived from ${\widehat{F}}_R$ outperforms median estimates derived from ${\widehat{F}}_N$ and ${\widehat{F}}_L$ in terms of mean squared estimation error. Note that the estimators ${\widehat{F}}_N,$ ${\widehat{F}}_L$ and ${\widehat{F}}_R$ coincide in the case of simple random sampling without replacement where ${\pi }_j\mathrm{<mo>=</mo>}\pi \mathrm{<mo>=</mo>}n/N.$

• Previous
• Next