5. Determination of the optimal inclusion probabilities

Piero Demetrio Falorsi and Paolo Righi

Previous | Next

The vector of $\pi ­$  values is determined by solving the following optimization problem:

$$\{\begin{array}{ll}\text{Min}\left({\displaystyle {\sum }_{k\in U}{\pi }_k{c}_k}\right)\hfill & \hfill \\ \text{AAV}\left({\widehat{t}}_{\left(dr\right)}\right)\le {\overline{V}}_{\left(dr\right)}\hfill & \left(d=1,\dots ,D;r=1,\dots ,R\right)\hfill \\ 0<{\pi }_k\le 1\hfill & \left(k=1,\dots ,N\right)\hfill \end{array},(5.1)$$

where $c_k$  is the cost for collecting information from unit $k$  and ${\overline{V}}_{\left(dr\right)}$  is a fixed variance threshold corresponding to ${\widehat{t}}_{\left(dr\right)}.$  System (5.1) minimizes the expected cost ensuring that the anticipated variances are bounded and that the inclusion probabilities lie between 0 and 1. If all the $c_k$  values are constants equal to 1, then the problem (5.1) minimizes the sample size. We note that in problem (5.1) the variances ${\sigma }_{rk}^2$  in $\text{AAV}\left({\widehat{t}}_{\left(dr\right)}\right)$  are treated as known; in practice they must be estimated. In Section 6, an empirical evaluation is conducted in order to study the sensitivity of the overall sample size with different estimated values of ${\sigma }_{rk}^2.$

To solve (5.1), we rearrange the inequality constraints to obtain

$${\displaystyle {\sum }_{k\in U}\frac{\left({\tilde{y}}_{rk}^2+{\sigma }_{rk}^2\right){\gamma }_{dk}}{{\pi }_k}}\le \frac{N-H}{N}{\overline{V}}_{\left(dr\right)}+{\displaystyle {\sum }_{k\in U}\left({\tilde{y}}_{rk}^2+{\sigma }_{rk}^2\right){\gamma }_{dk}}+{\text{AAV}}_{3\left(dr\right)}.(5.2)$$

By fixing the values of ${\text{AAV}}_{3\left(dr\right)}$  appropriately, the optimization problem becomes a classical Linear Convex Separate Problem (LCSP; Boyd and Vanderberg 2004). Figure 5.1 depicts the flow chart of the algorithm (A prototype software implementing the algorithm is available at http://www.istat.it/it/strumenti/metodi-e-software/software.), which is organized into two nested loops: the Outer Loop (OL) and the Inner Loop (IL). The two loops are updated according to a fixed point algorithm scheme. The convergence under some approximations is shown in Appendix A2.

Figure 5.1 Algorithm flowchart

Description for Figure 5.1

Initialization. At iteration $\alpha =0$  of the OL, set ${}^{\left(\alpha =0\right)}\pi =\left\{{}^{\left(\alpha =0\right)}\pi {}_k=\overline{\pi };k=1,\dots ,N\right\}$  with $0<\overline{\pi }\le 1.$  A reasonable choice is $\overline{\pi }=\mathrm{0.5.}$  At iteration $\tau =0$  of the Inner Loop, set ${}^{\left(\alpha \tau =0\right)}\pi ={}^{\left(\alpha \right)}\pi .$  Fix the $N$  vector, $\epsilon ,$  of small positive values.

Outer loop

• Fixing the values for the Inner Loop. In accordance with expressions (A1.4), (A1.7) and (A1.8) given in Appendix A1, the following real scalar values are computed

  $$a_{\left(dr\right)k}\left({}^{\left(\alpha \right)}\pi \right)={{\delta }^{\prime }}_k{\left[A\left({}^{\left(\alpha \right)}\pi \right)\right]}^{-1}{\displaystyle {\sum }_{j\in U}{\delta }_j{\tilde{y}}_{rj}{\gamma }_{dj}\left(1-{}^{\left(\alpha \right)}\pi {}_j\right)},(5.3)$$

  $$b_{\left(dr\right)k}\left({}^{\left(\alpha \right)}\pi \right)={{\delta }^{\prime }}_k{\left[A\left({}^{\left(\alpha \right)}\pi \right)\right]}^{-1}{\delta }_k{\sigma }_{rk}^2{\gamma }_{dk}\left(1-{}^{\left(\alpha \right)}\pi {}_k\right),(5.4)$$

  $$c_{\left(dr\right)k}\left({}^{\left(\alpha \right)}\pi \right)={\pi }_k^2{{\delta }^{\prime }}_k{\left[A\left({}^{\left(\alpha \right)}\pi \right)\right]}^{-1}\left[{\displaystyle {\sum }_{j\in U}{\delta }_j{{\delta }^{\prime }}_j{\sigma }_{rj}^2{\gamma }_{dj}{\left(1-{}^{\left(\alpha \right)}\pi {}_j\right)}^2}\right]{\left[A\left({}^{\left(\alpha \right)}\pi \right)\right]}^{-1}{\delta }_k.(5.5)$$

• Launch of the Inner Loop. The Inner Loop is executed until convergence.
• Updating or exiting. If the vector ${}^{\left(\alpha +1\right)}\pi$  is such that $\left|{}^{\left(\alpha +1\right)}\pi -{}^{\left(\alpha \right)}\pi \right|>\epsilon ,$  then the Outer Loop is iterated by updating the vector ${}^{\left(\alpha \right)}\pi$  with ${}^{\left(\alpha +1\right)}\pi .$  If $\left|{}^{\left(\alpha +1\right)}\pi -{}^{\left(\alpha \right)}\pi \right|\le \epsilon ,$  then the Outer Loop closes and ${}^{\left(\alpha \right)}\pi$  represents the optimal values solution to the problem of the system (5.1).

Inner Loop

• Fixing the values for the LCSP. The following values are computed:

  $$\begin{array}{lll}{}^{\left(\alpha \tau \right)}\text{A}{\text{AV}}_{3\left(dr\right)}\hfill & =\hfill & {{\displaystyle {\sum }_{k\in U}\left(1-{}^{\left(\alpha \tau \right)}\pi {}_k\right)a}}_{\left(dr\right)k}\left({}^{\left(\alpha \right)}\pi \right)\left[2{\tilde{y}}_{rk}{\gamma }_{dk}-{}^{\left(\alpha \tau \right)}\pi {}_k{a}_{\left(dr\right)k}\left({}^{\left(\alpha \right)}\pi \right)\right]\hfill \\ \hfill & +\hfill & {\displaystyle {\sum }_{k\in U}\left(1-{}^{\left(\alpha \tau \right)}\pi {}_k\right)\left[2b_{\left(dr\right)k}\left({}^{\left(\alpha \right)}\pi \right)-{}^{\left(\alpha \tau \right)}\pi {}_k{c}_{\left(dr\right)k}\left({}^{\left(\alpha \right)}\pi \right)\right].}\hfill \end{array}(5.6)$$

  in accordance with expression (A1.7) in Appendix A1.

• Solving the LCSP. Considering the ${}^{\left(a\tau \right)}\text{A}{\text{AV}}_{3\left(dr\right)}$  values as fixed, the ${}^{\left(\alpha \tau +1\right)}\pi$  is obtained by solving, by a standard algorithm for a classical LCSP, the following optimization problem:

  $$\{\begin{array}{l}\text{Min}\left({\displaystyle {\sum }_{k\in U}{}^{\left(\alpha \tau +1\right)}\pi {}_k{c}_k}\right)\hfill \\ {\displaystyle {\sum }_{k\in U}\frac{\left({\tilde{y}}_{rk}^2+{\sigma }_{rk}^2\right){\gamma }_{dk}}{{}^{\left(\alpha \tau +1\right)}\pi {}_k}}\le \frac{N-H}{N}{\overline{V}}_{\left(dr\right)}+{\displaystyle {\sum }_{k\in U}\left({\tilde{y}}_{rk}^2+{\sigma }_{rk}^2\right){\gamma }_{dk}+{}^{\left(\alpha \tau \right)}\text{A}{\text{AV}}_{3\left(dr\right)}}\hfill \\ 0<{}^{\left(\alpha \tau +1\right)}\pi {}_k\le 1\left(k=1,\dots ,N\right)\hfill \end{array}.(5.7)$$

• Updating or exiting. If the vector ${}^{\left(\alpha \tau +1\right)}\pi$  is such that $\left|{}^{\left(\alpha \tau +1\right)}\pi -{}^{\left(\alpha \tau \right)}\pi \right|>\epsilon ,$  then the Inner Loop is iterated by updating the vector ${}^{\left(\alpha \tau \right)}\pi$  with ${}^{\left(\alpha \tau +1\right)}\pi .$  If $\left|{}^{\left(\alpha \tau +1\right)}\pi -{}^{\left(\alpha \tau \right)}\pi \right|\le \epsilon$  then the Inner Loop closes and the updated vector ${}^{\left(\alpha +1\right)}\pi$  for the Outer Loop is given by ${}^{\left(\alpha \tau +1\right)}\pi .$

Remark 5.1. The problem of the system (5.7) can be solved by the algorithm proposed in Falorsi and Righi (2008, Section 3.1) which represents a slight modification of Chromy’s algorithm (1987), originally developed for multivariate optimal allocation in SSRSWOR designs and implemented in standard software tools (see for example the Mauss-R software available at: http://www3.istat.it/strumenti/metodi/software/campione/mauss_r/). Alternatively, the LCSP can be dealt with by the SAS procedure NLP as suggested by Choudhry et al. (2012).

Remark 5.2. The algorithm distinguishes the ${}^{\left(\alpha \tau \right)}\pi {}_k$  (updated in the Outer loop) from the ${}^{\left(\alpha \tau \right)}\pi {}_k$  (updated in the Inner loop). The innovation of the proposed algorithm lies precisely in this peculiarity. If this distinction between the inclusion probabilities is not made, i.e., ${}^{\left(\alpha \tau \right)}\pi ={}^{\left(\alpha \right)}\pi ,$  we have observed in several experiments that the iterate solutions of the LCSP for each Outer Loop do not converge to a stationary point.

Remark 5.3. After the optimization phase, in which the $\pi$  vector is defined as solution to problem of system (5.1), a calibration phase is performed (Falorsi and Righi 2008) to obtain calibrated inclusion probabilities, ${}_{\text{cal}}\pi {}_k,$  which modifies the optimal $\pi$  vector marginally in order to satisfy ${\displaystyle {\sum }_{k\in U}{}_{\text{cal}}\pi {}_k{\delta }_k}=n,$  where $n$  is a vector of integer numbers. The use of the Generalized Iterative Proportional Fitting algorithm (Dykstra and Wollan 1987) ensures that all resulting calibrated inclusion probabilities are in the $\left(0,1\right]$  interval.

Previous | Next