4. Computing response propensities using latent trait models

Alina Matei and M. Giovanna Ranalli

Previous | Next

The variable ${\theta }_k$ can be computed using a latent trait model. In general, latent variable models are multivariate regression models that link continuous or categorical responses to unobserved covariates. A latent trait model is essentially a factor analysis model for binary data (see Bartholomew, Steele, Moustaki and Galbraith 2002; Skrondal and Rabe-Hesketh 2007).

We start by creating the matrix with elements ${\left\{x_{k\ell }\right\}}_{k\in s;\ell =1,\dots \mathrm{,}m}.$ Figure 4.1 shows a schematic of the indicators $x_{k\ell }$ for respondents and nonrespondents. Then, we assume that the factors that drive unit response are the same as those that drive item response on selected variables of interest. In other words, item nonresponse is assumed nonignorable.

Figure 4.1 Schematic representing variables $x_{k\ell }$ for the sets $r$ and $\overline{r}$

Description for Figure 4.1

Let $q_{k\ell }$ be the probability of response of unit $k$ for item $\ell \mathrm{,}$ for all $\ell =1,\dots \mathrm{,}m$ and $k\in r\mathrm{.}$ As in the case of unit nonresponse, $q_{k\ell }$ is modelled as a function of the variable of interest using logistic regression as follows

$$q_{k\ell }=P\left(x_{k\ell }=1|y_{k\ell }\mathrm{,}{\theta }_k\mathrm{,}{R}_k=1\right)=\frac{1}{1+\exp \left(-\left({\beta }_{\ell 0}+{\beta }_{\ell 1}{\theta }_k+{\beta }_{\ell 2}{y}_{k\ell }\right)\right)}\mathrm{,}(4.1)$$

for $\ell =1,\dots \mathrm{,}m\mathrm{,}$ and $k\in r,$ where ${\beta }_{\ell 0}\mathrm{,}{\beta }_{\ell 1}$ and ${\beta }_{\ell 2}$ are parameters. Since $y_{k\ell }$ is known only for units with $x_{k\ell }=1,k\in r,$ Model (4.1) cannot be estimated. As in the case of unit nonresponse, we propose to estimate $q_{k\ell }$ as a function of an auxiliary variable related to the variable of interest, that is ${\theta }_k\mathrm{.}$ Model (4.1) is rewritten

$$q_{k\ell }=P\left(x_{k\ell }=1|{\theta }_k\mathrm{,}{R}_k=1\right)=\frac{1}{1+\exp \left(-\left({\beta }_{\ell 0}+{\beta }_{\ell 1}{\theta }_k\right)\right)}\mathrm{,}(4.2)$$

for $\ell =1,\dots \mathrm{,}m\mathrm{,}$ and $k\in r\mathrm{.}$ Model (4.2) is not an ordinary logistic regression model, because the ${\theta }_k’\text{s}$ are unobservable values taken by a latent variable. Latent trait models can be used in this case to estimate $q_{k\ell }\mathrm{,}{\theta }_k$ and the model parameters. Note that in the area of educational testing and psychological measurement, latent trait modelling is termed Item Response Theory.

The Rasch model (Rasch 1960) is a first simple latent trait model that is well known in the psychometrical literature and used to analyze data from assessments to measure variables such as abilities and attitudes. It takes the following form

$$q_{k\ell }=\frac{1}{1+\exp \left(-\left({\beta }_{\ell 0}+{\beta }_1{\theta }_k\right)\right)}\text{for }\ell =1,\dots \mathrm{,}m\text{ and }k\in r\mathrm{.}(4.3)$$

The parameters ${\beta }_{\ell 0}$ are estimated for each item $\ell$ and reflect the extremeness (easiness) of item $\ell :$ larger values correspond to a larger probability of a positive response at all points in the latent space. The parameter ${\beta }_1$ is known as the ‘discrimination’ parameter and can be fixed to some arbitrary value without affecting the likelihood as long as the scale of the individuals’ propensities is allowed to be free. In many situations the assumption that item discriminations are constant across items is too restrictive. The two-parameter logistic (2PL) model generalizes the Rasch model by allowing the slopes to vary. Specifically, the 2PL model assumes the form given in Equation (4.2). The parameters ${\beta }_{\ell 1}$ are now estimated for each item $\ell$ and provide a measure of how much information an item provides about the latent variable ${\theta }_k.$ To achieve identifiability of Model (4.2), we can fix the value of one or more parameters ${\beta }_{\ell 0}$ and ${\beta }_{\ell 1}$ in the estimation process. Moran (1986) showed that in the 2PL model, all the parameters are identifiable under wide conditions, provided the number of items exceeds two, and all the slopes are assumed to be strictly positive. A further generalization to Model (4.2) is considered in the literature - the 3PL model - that includes another parameter, the guessing parameter, to model the probability that a subject with a latent variable tending to $-\infty$ responds to an item. Such an extension does not seem necessary in the context at hand and will not be considered further.

4.1 Assumptions in latent trait models

Latent trait models typically rely on the following assumptions. The first one is the so-called conditional independence assumption, which postulates that item responses are independent given the latent variable (i.e., the latent variable accounts for all association among the observed variables $x_{k\ell }).$ Consequently, given ${\theta }_k\mathrm{,}$ the conditional probability of $x_k$ is

$$P\left(x_k|{\theta }_k\right)={\displaystyle \prod _{\ell =1}^{m}P\left(x_{k\ell }|{\theta }_k\right)}\mathrm{.}$$

Following Bartholomew et al. (2002, page 181) ‘the assumption of conditional independence can only be tested indirectly by checking whether the model fits the data. A latent variable model is accepted as a good fit when the latent variables account for most of the association among the observed responses.’

A second assumption of Models (4.2) and (4.3) is that of monotonicity: as the latent variable ${\theta }_k$ increases, the probability of response to an item increases or stays the same across intervals of ${\theta }_k.$ In other words, for two values of ${\theta }_k,$ say $a$ and $b,$ and arbitrarily assuming that $a<b\mathrm{,}$ monotonicity implies that $P\left(x_{k\ell }=1|{\theta }_k=a\right)<P\left(x_{k\ell }=1|{\theta }_k=b\right)$ for $\ell =1,\dots \mathrm{,}m\mathrm{.}$ Larger values of ${\theta }_k$ are associated with a greater chance of a response to each item.

Finally, the third, and possibly strongest, assumption of Models (4.2) and (4.3) is that of unidimensionality, implying that a single latent variable fully explains the willingness of unit $k$ to answer the questionnaire. All these basic assumptions imply that the dependence between the items $x_{k\ell }$ may be explained by the latent variable ${\theta }_k$ which represents the units’ willingness and that the probability that a unit $k$ responds to a given variable increases with ${\theta }_k.$

4.2 Estimation of the model

In what follows we focus on the two-parameter logistic (2PL) model given in (4.2). Let ${\beta }_{\ell }={\left({\beta }_{\ell 0}\mathrm{,}{\beta }_{\ell 1}\right)}^{\prime }$ and $\beta =\left\{{\beta }_{\ell }\mathrm{,}\ell =1,\dots \mathrm{,}m\right\}\mathrm{.}$ Model (4.2) can be fitted using maximum likelihood or bayesian methods. We focus here on the former. Under the maximum likelihood approach, three major methods - joint, conditional and marginal maximum likelihood - are developed. Here, we will concentrate on marginal maximum likelihood that can be applied to fit the 2PL model. This method is also used in the simulation studies of Section 6. It consists of maximizing the likelihood of the model after the ${\theta }_k$ are integrated out on the basis of a common distribution assumed on these parameters. In particular, it is assumed that ${\theta }_k$ is a random variable following a distribution with the density function $h(\cdot );$ typically ${\theta }_k\sim N\left(\mathrm{0,1}\right)\mathrm{.}$ It is also assumed that the response vectors $x_k$ are independent of one another and the conditional independence assumption holds.

For a set of $n_r$ respondents having the response vectors $x_k\mathrm{,}k=1,\dots \mathrm{,}{n}_r\mathrm{,}$ the marginal likelihood can be expressed as

$$L\left(\beta ;x_1\mathrm{,}\dots \mathrm{,}{x}_{n_r}\right)={\displaystyle \prod _{k=1}^{n_r}f\left(x_k|\beta \right)}\mathrm{,}$$

where $f\left(x_k|\beta \right)={\displaystyle {\int }_{-\infty }^{\infty }g\left(x_k|{\theta }_k\mathrm{,}\beta \right)h\left({\theta }_k\right)d{\theta }_k}\mathrm{,}$

$$g\left(x_k|{\theta }_k\mathrm{,}\beta \right)={\displaystyle \prod _{\ell =1}^m{q}_{k\ell }^{x_{k\ell }}{\left(1-q_{k\ell }\right)}^{1-x_{k\ell }}}={\displaystyle \prod _{\ell =1}^m\frac{\exp \left(x_{k\ell }\left({\beta }_{\ell 0}+{\beta }_{\ell 1}{\theta }_k\right)\right)}{1+\exp \left({\beta }_{\ell 0}+{\beta }_{\ell 1}{\theta }_k\right)}}\mathrm{,}$$

and $h$ now denotes the density of the $N\left(0,1\right)$ distribution. The method consists in maximizing the corresponding log-likelihood, given by

$$\log L\left(\beta ;x_1\mathrm{,}\dots \mathrm{,}{x}_{n_r}\right)={\displaystyle \sum _{k=1}^{n_r}}\log \left(f\left(x_k|\beta \right)\right)\mathrm{,}$$

with respect to $\beta$ using, for example, the EM algorithm. Estimates of ${\beta }_{\ell 0}$ and ${\beta }_{\ell 1}\mathrm{,}\ell =1,\dots \mathrm{,}m$ are thus provided. Afterwards, ${\theta }_k$ is estimated using the empirical Bayes method by maximizing the posterior density

$$h\left({\theta }_k|x_k\right)=\frac{g\left(x_k|{\theta }_k\mathrm{,}\beta \right)h\left({\theta }_k\right)}{g\left(x_k\right)}\propto g\left(x_k|{\theta }_k\mathrm{,}\beta \right)h\left({\theta }_k\right)\mathrm{,}$$

with respect to ${\theta }_k$ and keeping item parameters and observations fixed. Estimates of $q_{k\ell }$ are obtained using Expression (4.2), where ${\beta }_{\ell 0}\mathrm{,}{\beta }_{\ell 1}$ and ${\theta }_k$ are replaced with their estimates.

4.3. Goodness-of-fit measures of the model

Different goodness-of-fit measures are proposed in the literature to test whether the model given in (4.2) adequately fits the data (see e.g., Bartholomew et al. 2002). One uses two-way and three-way margins of the response items. Discrepancies between the expected $\left(E\right)$ and observed $\left(O\right)$ counts in these tables are measured using the statistic $R={\left(O-E\right)}^2/E.$ Large values of $R$ for the second-order or third-order margins will identify sets of items for which the model does not fit well. Note that the residuals ${\left(O-E\right)}^2/E$ are not independent and they cannot be summed to give an overall test statistics distributed as a chi-squared (see Bartholomew et al. 2002, page 186). Item fit indexes (Bond and Fox 2007) can be used to this end as well. On the basis of estimated latent variables and item parameters, the expected response of a unit to an item can be computed. The similarity between the observed and expected responses to any item can be assessed through two fit mean-square statistics: the outlier-sensitive fit statistic (item outfit) and the information-weighted fit statistic (item infit). The estimate produced by the item outfit is relatively more affected by unexpected responses different from a person’s measure, i.e., it is more sensitive to unexpected observations by units on items that are relatively very easy or very hard for them to answer. The item infit has each observation weighted by the information and, on the other side, is relatively more affected by unexpected responses closer to a person’s measure, i.e., it is more sensitive to unexpected patterns of observations by units on items that are roughly targeted on them according to their latent variable value. The expected value for both statistics is one. For infit and outfit values greater/less than one indicate more/less variation between the observed and the predicted response patterns, a range of 0.5 to 1.5 is generally acceptable (Bond and Fox 2007).

In addition, point-measure correlations (Olsson, Drasgow and Dorans 1982) can be used to estimate the correlation between the latent variable and the single item response. Items for which such measures take negative or zero values should be removed from the analysis or may be evidence that the latent construct is not unidimensional. Unidimensionality can be tested by running a Principal Components Analysis (PCA) of the standardized residuals for the items (Wright 1996). In this way the first component (dimension) has already been removed, and it is possible to look at secondary dimensions, components or contrasts. Unidimensionality is supported by observing that the eigenvalue of the first PCA component in the correlation matrix of the residuals is small (usually less than 2.0). If not, the loadings on the first contrast indicate that there are contrasting patterns in the residuals.

Finally, when items are used to form a scale, they need to have internal consistency. Cronbach alpha can be used to test whether items have the reliability property, i.e., if they all measure the same thing, then they should be correlated with one another.

4.4. Estimation of $p_k$

Two solutions are shown here to estimate $p_k$ using information from the latent trait model. The first solution uses logistic regression to estimate $p_k$ for all $k\in s\mathrm{,}$ and a two-stage approach.

Stage 1: First, an estimate ${\widehat{\theta }}_k$ of ${\theta }_k$ is provided. To compute a value ${\widehat{\theta }}_k$ for $k\in \overline{r},$ we assume again that unit nonresponse is just an extreme form of item nonresponse. Thus, a nonrespondent does not answer any item $\ell$ and thus $x_{k\ell }=0,$ for all $\ell =1,\dots \mathrm{,}m\mathrm{.}$ The computation of ${\widehat{\theta }}_k$ for $k\in \overline{r}$ is handled as follows: we add to the set $r$ a phantom respondent unit $\tilde{k}$ having $x_{\tilde{k}\ell }$ equal to 0, for all $\ell =1,\dots \mathrm{,}m\mathrm{.}$ We denote this new set by $\tilde{r}=r\cup \left\{\tilde{k}\right\}\mathrm{.}$ We estimate the parameters of Model (4.2) using all units $k\in \tilde{r},$ and compute the values ${\widehat{\theta }}_k\mathrm{,}k\in \tilde{r}\mathrm{.}$ Model (4.2) allows the computation of ${\widehat{\theta }}_k$ for all $k\in \tilde{r}\mathrm{.}$ Unit $\tilde{k}$ has an estimated value ${\widehat{\theta }}_{\tilde{k}}\mathrm{.}$ We assign to all units $k\in \overline{r}$ an estimate ${\widehat{\theta }}_k$ equal to ${\widehat{\theta }}_{\tilde{k}}\mathrm{.}$ Thus, the same value of ${\widehat{\theta }}_k$ is provided for all $k\in \overline{r}.$ Using this method, each unit $k\in s$ has associated an estimate ${\widehat{\theta }}_k\mathrm{.}$ This is the key feature for the estimation of the response probabilities $p_k$ provided in the next stage.

Stage 2: We use the estimate ${\widehat{\theta }}_k,$ for $k\in s,$ provided in the first stage as a covariate in Model (3.4) instead of the unknown value of ${\theta }_k;$ in particular

$$p_k=P\left(R_k=1|{\widehat{\theta }}_k\right)=\frac{1}{1+\exp \left(-\left({\alpha }_0+{\alpha }_1{\widehat{\theta }}_k\right)\right)}\mathrm{,}\text{for all }k\in s\mathrm{.}(4.4)$$

Model (4.4) provides estimates ${\widehat{p}}_k$ of $p_k,$ for all $k\in s\mathrm{.}$

One of the Referees suggested the following solution to estimate $p_k\mathrm{.}$ Let $S_k={\displaystyle {\sum }_{\ell =1}^m{x}_{k\ell }}$ be the raw score for unit $k,$ i.e., the number of items unit $k$ has responded to: if $k\in \overline{r},$ then $S_k=0;\text{ if }k\in r\mathrm{,}$ then $S_k>0.$ Then $p_k$ can be estimated by modelling $P\left(S_k>0|{\theta }_k\right)\mathrm{.}$ By the conditional independence assumption we have

$$\begin{array}{lll}{p}_k\hfill & =\hfill & P\left(S_k>0|{\theta }_k\right)=1-P\left(S_k=0|{\theta }_k\right)=1-P\left({\cap }_{\ell =1}^m\left(x_{k\ell }=0|{\theta }_k\right)\right)\hfill \\ \hfill & =\hfill & 1-{\displaystyle \prod _{\ell =1}^m}\left(1-P\left(x_{k\ell }=1|{\theta }_k\right)\right).\hfill \end{array}$$

We have $P\left(x_{k\ell }=1|{\theta }_k\right)=P\left(R_k=1|{\theta }_k\right)P\left(x_{k\ell }=1|{\theta }_k\mathrm{,}{R}_k=1\right)+P\left(R_k=0|{\theta }_k\right)P(x_{k\ell }=1|{\theta }_k\mathrm{,}$ $R_k=0)=p_k{q}_{k\ell }\mathrm{,}$ because $P\left(x_{k\ell }=1|{\theta }_k\mathrm{,}{R}_k=0\right)=0.$ As a result, we obtain

$$p_k=1-{\displaystyle \prod _{\ell =1}^m}\left(1-p_k{q}_{k\ell }\right)\mathrm{,}k\in r\mathrm{.}$$

The estimated response probability ${\widehat{p}}_k\mathrm{,}k\in r$ is obtained as a solution to the polynomial equation

$${\widehat{p}}_k=1-{\displaystyle \prod _{\ell =1}^m}\left(1-{\widehat{p}}_k{\widehat{q}}_{k\ell }\right)\mathrm{.}$$

This solution, although very elegant, has two drawbacks. If $m$ is large, the above polynomial equation is difficult or even impossible to solve. If it possible to solve the polynomial equation for moderate $m,$ the real solutions are not necessarily in (0, 1). This solution has not been considered here further.

Previous | Next