A method to find an efficient and robust sampling strategy under model uncertainty
Section 4. Guiding the choice of sampling design with the help of a risk measure

• Previous
• Next

We have seen in Section 3 that even a simple misspecification of the working model might result in the strategy $\pi \text{ps}\left({\delta }_2\right)-\text{diff}\left({\delta }_1\right)$ not being optimal. It is therefore risky to accept a given model as correct without any type of assessment. While most of the information needed for an “objective” evaluation of the model is not available at the design stage, it is possible to reach some degree of confidence about the parameters in the working model that allows for comparing a set of designs and make the decision about which one to implement. In this section we propose a method to assist in the choice of the sampling design.

The model expected MSE (3.2) in Result 1 can be viewed as a function of $\beta$ and ${\sigma }^2,$ as everything else is available at the design stage. To begin with, let us assume that ${\sigma }^2$ is also known. Then we can write

$$L_{\text{p}}\left(\beta \right)\mathrm{<mo>=</mo>}{\text{MSE}}_{\xi \text{p}}\left(\beta |x\mathrm{,}\delta \mathrm{,}\sigma \right)\mathrm{<mo>=</mo>}{\text{MSE}}_{\text{p}}\left({\displaystyle \sum _s\frac{f\left(x_k|{\beta }_1\right)-f\left(x_k|{\delta }_1\right)}{{\pi }_k}}\right)+{\sigma }^2{\displaystyle \sum _U\left(\frac{1}{{\pi }_k}-1\right)g{\left(x_k|{\beta }_2\right)}^2.}$$

For any design, $p\left(\cdot \right),$ this function can be evaluated at any $\beta$ and it indicates the loss incurred by assuming that $\delta$ is the true parameter when it is, in fact, $\beta .$ We can assume a prior distribution on $\beta ,$ $h\left(\beta \right),$ and calculate the risk under $h,$

$$R\left(\text{p}\right)\mathrm{<mo>=</mo>}{\text{E}}_h\left({\text{MSE}}_{\xi \text{p}}\left(\beta |x\mathrm{,}\delta \mathrm{,}\sigma \right)\right)\mathrm{<mo>=</mo>}{\displaystyle {\int }_{\Theta }h\left(\beta \right)}\cdot {\text{MSE}}_{\xi \text{p}}\left(\beta |x\mathrm{,}\delta \mathrm{,}\sigma \right)d\beta \mathrm{,}(4.1)$$

where $\Theta$ is the sample space of $\beta .$ The design that yields the smallest risk should be chosen. Note that numerical integration methods (e.g., Monte Carlo simulation methods) may be needed to evaluate the risk (4.1).

In practice, ${\sigma }^2$ is unknown. We propose two ways for dealing with it. The first one is to see now the loss as a function of $\beta$ and $\sigma$ and calculate the risk as above, assuming a prior on the pair $\beta$ and $\sigma .$ The second one is to provide some “guess” about its value. This approach can use the fact that (Proof in the Appendix)

$${\sigma }^2\approx \frac{S_{f\mathrm{,}f}}{{\overline{g}}^2}\left(\frac{1}{R_{f\mathrm{,}y}^2}-1\right)(4.2)$$

where $S_{f\mathrm{,}f}\mathrm{<mo>=</mo>}{\displaystyle {\sum }_U{\left(f\left(x_k|{\beta }_1\right)-\overline{f}\right)}^2}/N,$ $\overline{f}\mathrm{<mo>=</mo>}{\displaystyle {\sum }_{U}f\left(x_k|{\beta }_1\right)}/N,$ ${\overline{g}}^2\mathrm{<mo>=</mo>}{\displaystyle {\sum }_{U}g{\left(x_k|{\beta }_2\right)}^2}/N$ and $R_{f\mathrm{,}y}$ is the correlation between $f\left(x|{\beta }_1\right)$ and $y.$ (In Example 3 below, we give a more convenient expression in a special case.) Although $R_{f\mathrm{,}y}$ is unknown, for repeated surveys we do have some previous knowledge about it. In other cases it is often possible to have some reasonable “guess” about it.

It remains to comment on the choice of the prior distribution $h\left(\beta \right).$ The choice of the distribution and its parameters is subjective and defined by the statistician. Nevertheless, it should reflect the available knowledge about the model parameter $\beta .$ In particular, $h\left(\beta \right)$ should be centered around $\beta \mathrm{<mo>=</mo>}\delta .$ Its variance should reflect how confident we are about the working model. Note that a full confidence on the working model would be a density with all its mass at $\beta \mathrm{<mo>=</mo>}\delta ,$ in which case the risk (4.1) would be minimized by the $\pi \text{ps}$ design given by condition 2 in Section 2.

It might be argued that by introducing $h\left(\beta \right)$ an additional source of subjectivity has been added to the choice of the sampling design. The prior may add a certain Bayesian flavor to the process, but note that $h\left(\beta \right)$ is only needed for choosing the design. Hence, the inference is still design-based. Furthermore, relying on an assumed model is also subjective in choice of assumption and it does involve a risk. The risk measure in (4.1) allows for quantification of that risk.

• Previous
• Article main page
• Next