MULTILEVEL METHODS OF STATISTICAL ANALYSIS

PWP-CCPR-2001-006

Abstract

In hierarchical data structures, observational units at one level are nested within units at other levels. For longitudinal data on individuals, within-person observations are nested within individuals. Within-person (micro) observations are generally more like

each other than observations sampled randomly across individuals (contexts). In generalized regression models, several approaches can accommodate this lack of independence: (i) random effects; (ii) fixed effects; (iii) marginal models in the non-Gaussian case, and (iv) regression coefficient covariance matrix adjustment. Each approach has specific advantages and drawbacks. Fixed effects are often used in situations in which a key assumption of the random effects approach is thought to be implausible. In those instances, the use of marginal models or regression coefficient covariance matrix adjustment would also merit reconsideration. Where more than one approach can justifiably be applied—and there are many such cases—estimation results can be method-dependent, leading to different conclusions about the effects of specific covariates. In addition, particularly for the random effects approach, alternative methods of estimation can yield somewhat different results, as can different algorithmic implementations of the same method across software packages. The impact of small within-context sample sizes in unbalanced designs merits further exploration. Where possible, researchers should consider whether their conclusions are method- or algorithm dependent.