4 Consistency of PPL-BIC

Chen Xu, Jiahua Chen and Harold Mantel

Previous | Next

We now investigate the asymptotic behavior of the PPL-BIC procedure under the joint randomization framework. Suppose there is a sequence of finite populations, say $D_r$ with $r\to \infty .$ Each $D_r$ is an independent and identically distributed (i.i.d.) sample of size $N_r$ from a super-population modeled by (2.1) with random variable $\left(Y,X=\left\{X_1\mathrm{,}\dots \mathrm{,}{X}_p\right\}\right).$ Within each $D_r,$ a sample $d_r$ of size $n_r$ is drawn according to some sampling scheme. We assume that both $N_r$ and $n_r$ increase to infinity as $r\to \infty ,$ with the sampling fraction $n_r/N_r$ bounded by some constant $C<1.$ For simplicity of notation, we will drop the index $r$ in the following discussion.

Without loss of generality, we assume that the first $q$ coefficients are nonzero and denote the true value of $\beta$ by ${\beta }_0=\left\{{\beta }_{01}\mathrm{,}{\beta }_{02}\right\}$ with ${\beta }_{02}=0.$ Also, we use $s_0$ to denote the true model $\left\{1,\dots ,q\right\}$ to be identified. We establish the selection consistency of PPL-BIC in two steps. In the first step we show that, for appropriate choices of ${\varphi }_{\lambda }(\cdot ),$ the PPL can consistently identify the true $s_0$ so that $s_0\in S_{\Omega }$ with probability tending to 1. In the second step, we verify that BIC (3.4) consistently selects $s_0$ over $S_{\Omega }.$

For the asymptotic analysis, we define ${\phi }_{\lambda }=\max \left\{{{\varphi }^{\prime }}_{\lambda }\left(\left|{\beta }_{0j}\right|\right)\text{ for }j\in s_0\right\}$ and associate $\lambda$ with $n$ to make ${\phi }_{\lambda }$ a sequence. Under the joint randomization framework, we show the claim of step 1 as the following theorem.

Theorem 1 Under regularity conditions on model (2.1) and other requirements specified in the online supplement, if ${\phi }_{\lambda }\to 0$ as $n\to \infty ,$ then there exists a local maximizer ${\widehat{\beta }}_{\lambda }=\left({\widehat{\beta }}_{\lambda 1}\mathrm{,}{\widehat{\beta }}_{\lambda 2}\right)$ of the penalized pseudo-likelihood function (3.5) such that

$$\Vert {\widehat{\beta }}_{\lambda }-{\beta }_0\Vert =O_p\left(n^{-1/2}+{\phi }_{\lambda }\right)\text{ and }P\left\{{\widehat{\beta }}_{\lambda 2}=0\right\}\to 1$$

with $\Vert \mathrm{.}\Vert$ denoting the Euclidean norm.

The consistency result in Theorem 1 holds for popular nonconvex penalty functions. For example, for the $L_{\gamma }$ penalty with $\gamma \in \left(\mathrm{0,1}\right),$ consistency holds if $\lambda \to 0;$ for the SCAD penalty, consistency holds if $\lambda \to 0$ and $\sqrt{n}\lambda \to \infty .$ It also implies that with probability tending to 1, the true model $s_0$ is included in $S_{\Omega },$ which serves as a prerequisite for the selection consistency of BIC over $S_{\Omega }.$

We now establish the consistency of using BIC on $S_{\Omega }$ with a specified ${\varphi }_{\lambda }(\cdot )$ that satisfies Theorem 1. Following the notation used in Section 3.2, let $s_{\lambda }$ be the model corresponding to a PPL estimator ${\widehat{\beta }}_{\lambda },$ and let $\Omega$ be the range of $\lambda$ under consideration. We define two collections of candidate models as follows: 

• Over-fitted models: $S_{+}=\left\{s:s_0\subset s\mathrm{,}s\ne s_0\right\};$
• Under-fitted models: $S_{-}=\left\{s:s_0\cancel{\subset }s\right\}.$

Notation $\not\subset$ denotes there is at least one different element between two sets, so that $S_{-}$ is the collection of candidate models which does not include all variables in the true model. Then, $\Omega$ can be partitioned accordingly into

$${\Omega }_{+}=\left\{\lambda :s_{\lambda }\in S_{+}\right\}\mathrm{,}{\Omega }_{-}=\left\{\lambda :s_{\lambda }\in S_{-}\right\}\mathrm{,}{\Omega }_0=\left\{\lambda :s_{\lambda }=s_0\right\}\mathrm{.}\left(4.1\right)$$

By Theorem 1, we have shown that $P\left({\Omega }_0\ne \varnothing \right)\to 1.$ Therefore, the selection consistency of BIC over $S_{\Omega }$ is achieved if BIC is able to identify $s_0$ from any model $s_{\lambda }$ with $\lambda \in {\Omega }_{+}\cup {\Omega }_{-}.$ We use the following theorem to establish this consistency result.

Theorem 2  Under the same conditions as in Theorem 1,

$$P\left\{\underset{\lambda \in {\Omega }_{+}\cup {\Omega }_{-}}{\min }{\text{BIC}}_n\left(s_{\lambda }\right)\le {\text{BIC}}_n\left(s_0\right)\right\}\to \mathrm{0,}$$

where ${\Omega }_{+}$ and ${\Omega }_{-}$ are defined in (4.1).

Previous | Next