Back to the multilevel page of Tom Snijders

Snijders, Tom A.B., and Bosker, Roel J.
Multilevel Analysis: An Introduction to Basic and Advanced Multilevel Modeling, second edition.
London etc.: Sage Publishers, 2012

ISBN 9781849202008 (hardcover), ISBN 9781849202015 (pbk). xii + 368 p.

The second edition of the textbook, Multilevel Analysis: An introduction to basic and advanced multilevel modeling, written by Tom A.B. Snijders and Roel J. Bosker, appeared November 2011 at Sage Publishers. The official publication year, however, is 2012. The Sage announcement of this book is here, and here is the table of contents. The book was totally updated compared to the first edition, with new chapters on missing data, survey weights in multilevel analysis, and miscellaneous methods (Bayesian estimation, sandwich standard errors, latent class models). Each chapter (from 2 to 17) ends with a glommary, which is a combination of a glossary and a summary, giving the main terms and an overview of the chapter. Here is a set of slides which can be used for teaching. This webpage contains: Data sets used in the book. Macros / scripts that may be useful in general. Software setups for reproducing examples in the book. Additional remarks and references. Corrections.

Data Sets

• The main data set, used in Chapters 4-10 (where missings are excluded):
  • mlbook2_r_dat.zip, zipped file containing the file named mlbook2_r.dat, an ascii file with 11 variables, top line gives the variable names, may be read into R by a command such as

    > mlbook_red <- read.table("mlbook2_r.dat", header=TRUE)

    For reading this file into other software, delete the top line but note that it contains the variable names.
  • mlbook2_r_obe.zip, zipped file containing the file named mlbook2_r.obe, the same, but formatted as a macro file to be read directly by MLwiN:

    obey mlbook2_r.obe

  • mlbook2_r.zip, zipped file containing the file named mlbook2_r.dta, a Stata data file used in most of the Stata do-files below.
  • mlbook2_mm.zip, zipped file containing the file named mlbook2_mm.dat, an ascii file with 23 variables, top line gives the variable names. This is the data file used in Example 9.3, containing 4016 cases. The subset of 3758 cases without any missings in the variables being analyzed is used in examples in other chapters. For example, 114 of these 4016 have missing values on the dependent variable langPOST. For the construction of this file and meaning of the variable names, consult script mlbook2_prepare_data.r contained in the zipped file mlbook2_data_preparations.zip which is also mentioned below.
    After unzipping, this file may be read into R by a command such as

    > mlbook_mm <- read.table("mlbook2_mm.dat", header=TRUE)

    For reading this file into other software, delete the top line but note that it contains the variable names.
  • The original data set and an r script transforming it to the derived data sets used here, is in the zipped file mlbook2_data_preparations.zip.
    This is not directly needed, but is included just to show the data transformations that were carried out.

• Data set used in example 3.7 in Chapter 3:
  • mlbook2_bb_obe.zip, zipped file containing the file named mlbook2_bb.obe; a data set with a few more cases than mlbook2_r (obtained by also including those that had a missing value on aritPOST but not on any other variables) formatted as a macro file to be read directly by MLwiN:

    obey mlbook2_bb.obe

    If you wish to use this data file in other software, just delete the top and bottom text lines that contain the information for MLwiN.

• Stata data set used in Chapter 9:
  • mlbook_2nded_withmissingvalues.zip, zipped file containing the file named mlbook_2nded_withmissingvalues.dta, a Stata data file used in chap9.do.

• Data set used in Chapter 14:
  • DataPISA.zip, zipped file containing the file combineusa-999_c.sav.
    
  • pisacc.zip, an excerpt from this file in ascii format, as written in script pisa_b.r below.
  • grades.txt, a data file use in script pisa_b.r below.
    For how to use this: see the MLwiN and r scripts below.

• Data set used in Section 15.1.
  • soep_5560_21.zip, zipped file containing the file soep_5560_21.dat, to be read in when using R.
    
  • soep5560_21.zip, zipped file containing the file soep5560_21.dat, to be read in when using MLwiN.
    For how to use this: see the MLwiN and R scripts below.
  • Data_Ex15_12.zip, zipped file containing the data for Example 15.12.
    For how to use this: see the MLwiN script below.

• Data set used in the examples in Chapter 16:
  • mlbook2_b_obe.zip, zipped file containing the file named mlbook2_b.obe; a data set with a few more cases than mlbook2_r (obtained by also including those that had a missing value on aritPOST or langPOST but not on any other variables) formatted as a macro file to be read directly by MLwiN:

    obey mlbook2_b.obe

    If you wish to use this data file in other software, just delete the top and bottom text lines that contain the information for MLwiN.
  • mlbook2_b_dat.zip, zipped file containing the file named mlbook2_b.dat, which basically is the same data file, but now for use by R:

    > mlbook_b <- read.table("mlbook2_b.dat", header=TRUE)

• Data set used in Sections 17.2-3:
  • rel_level12.zip, zipped file containing the files named rel_level1.txt and rel_level2.txt, two ascii files for both of which the top line gives the variable names. These files may be read into R by a command such as

    > level2 <- read.table("rel_level2.txt", header=TRUE)

    For reading this file into other software, delete the top line but note that it contains the variable names.
  • wvs_evs_small.zip, zipped file containing the file named wvs_evs_small.dta, a Stata data file used in chap17.do.

• Data set used in Sections 17.3.4-5:
  • LOOPDIC.zip, zipped file containing the file named LOOPDIC.DAT, an ascii file with 10 variables.

• Data sets for Stata:
  • MA2data_CH03.zip, zipped file containing the file named MA2data_CH03.dta, a Stata data file containing some data of Chapter 3, used in Chap17A_np.do.
  • mlbook2_r.zip, zipped file containing the file named mlbook2_r.dta, a Stata data file used in most of the Stata do-files below.
  • mlbook_2nded_withmissingvalues.zip, zipped file containing the file named mlbook_2nded_withmissingvalues.dta, a Stata data file used in chap9.do (see below).
  • wvs_evs_small.zip, zipped file containing the file named wvs_evs_small.dta, a Stata data file used in chap17.do (see below).

• A complete set in Excel (zipped) of all data sets currently available (9.6MB), kindly put together by Niels Proctor of the School of Forest Resources & Conservation, University of Florida:
  • Multilevel_Analysis_2nd_Ed_data.zip.

Macros / scripts

MlwiN

• hetsced1.obe, a MLwiN macro for testing level-one heteroscedasticity using test (10.5) of Snijders and Bosker (2012), which is the same as test (9.6) in Bryk and Raudenbush (2002). Used in Chapter 10.
• smooth_2.obe, a MLwiN macro for smoothing Y as a function of X, using moving averages. Used in Chapter 10.
• table_2.obe, a MLwiN macro for averaging Y as a function of variable X that should have a limited number of discrete values. Used in Chapter 10.
• dinfl_2.obe, a MLwiN macro for calculating deletion influence statistics for level-two units. It is based on Snijders and Berkhof (2008, Section 3), and implements the methods also explained in Section 10.7 of Snijders and Bosker (2012). Used in Chapter 10.
• explvar.mac, a MLwiN macro for computing the explained variance in a two-level random intercept logistic regression model, according to formula (17.22).

R

• ml2_impute_b.R, a collection of R functions for multiple imputation by two-level random intercept models for continuous and binary variables.
• nlme_utils.R, a couple of nlme utilities used in the R script for Chapter 9.
• Not a macro used in an example, but useful packages:
  The packages reshape and reshape2 can apply various transformations to data that are very useful for multilevel modeling. For data sets of repeated measures (level 1) within individuals (level 2), they can easily transform between multivariate (one line per individual) and disaggregated (one line per measurement) formats. See the cookbook for R for a brief explanation of reshape2. If you wish more information, note that the author Hadley Wickham has explained the transition from reshape to reshape2; and there are an introduction to reshape and a publication.
• Not a macro used in an example, but a useful package:
  The package influence.ME for detecting influential data in mixed effects models ; also see the publication about this package in the R journal.

SAS

• Not used in an example in the book, but useful nevertheless for Section 9.4.2:
  A SAS Macro for Computing Pooled Likelihood Ratio Tests with Multiply Imputed Data by Stephen A. Mistler.

Stata

• predict_margins.do, a do-file to plot marginal effects and predicted probabilities from multilevel logistic regression models, contributed by Tim Mueller.

Software Setups

Note that you can download the files below in many browsers by right-clicking on the file, and choosing something like "save as".

	HLM	MLwiN	R	Mplus	SAS	Stata	Chapter Title
Chapter 3		CH3ex7.obe				Chap03_np.do	Statistical Treatment of Clustered Data
Chapter 4	mlbook_hlm_chapter4.zip	CH4568.obe	ch45.r			chap4.do	The Random Intercept Model
Chapter 5	mlbook_hlm_chapter5.zip	CH4568.obe	ch45.r			chap5.do	The Hierarchical Linear Model
Chapter 6	mlbook_hlm_chapter6.zip	CH4568.obe	ch6.r			chap6.do	Testing and Model Specification
Chapter 7							How much does the model explain?
Chapter 8	mlbook_hlm_chapter8.zip	CH4568.obe	ch8.r			chap8.do	Heteroscedasticity
Chapter 9			ch9.r			chap9.do	Missing Data
Chapter 10		ch10.obe  ch10_infl.obe	ch10.r			chap10.do	Assumptions of the Hierarchical Linear Model
Chapter 11							Designing Multilevel Studies
Chapter 12						chap12.do	Other Models and Methods
Chapter 13						chap13.do	Imperfect Hierarchies
Chapter 14		PISA.obe	pisa_b.R			chap14.do	Survey Weights
Chapter 15		soep5560_21.obe  Example15_12.obe	ch_15.r				Longitudinal Data
Chapter 16		CH16.obe	ch16.r				Multivariate Multilevel Models
Chapter 17			ch17.r  ch17_ex6.r			chap17.do Chap17A_np.do	Discrete Dependent Variables

Additional remarks and references

• A very nice book (open access!) with a wide-ranging treatment of multilevel modeling:

  Alastair H. Leyland, Peter P. Groenewegen, Multilevel Modelling for Public Health and Health Services Research, Cham: Springer, 2020.

  Although framed in the context of health-related research, it is also useful as a general introduction. The book has three extensive MLwiN tutorial chapters.

• A website with nice pointers to various great resources is the multilevel FAQ of the MRC Cognition and Brain Sciences Unit (CBU) in Cambridge (UK).

• A nice general introductory article is
  Markus Brauer and John J. Curtin. "Linear Mixed-Effects Models and the Analysis of Nonindependent Data: A Unified Framework to Analyze Categorical and Continuous Independent Variables that Vary Within-Subjects and/or Within-Items." Psychological Methods (2018), 23, 389-411.
  DOI: http://dx.doi.org/10.1037/met0000159

• A paper contrasting the use of multilevel (hierarchical linear) models with methods that control in a simpler way for clustering of data by population-averaged models:
  Daniel McNeish, Laura M. Stapleton, and Rebecca D. Silverman (2017). "On the unnecessary ubiquity of hierarchical linear modeling." Psychological Methods 22, 114-140.
  DOI: http://dx.doi.org/0.1037/met0000078

• A paper about how to use propensity scores for estimating treatment effects from observational (i.e., non-experimental) multilevel data:
  Walter L. Leite, Francisco Jimenez, Yasemin Kaya, Laura M. Stapleton, Jann W. MacInnes, and Robert Sandbach (2015). "An evaluation of weighting methods based on propensity scores to reduce selection bias in multilevel observational studies". Multivariate Behavioral Research, 50, 265-284.
  DOI: https://dx.doi.org/10.1080/00273171.2014.991018

• Chapter 2
  
  • Section 2.2
    For multilevel models with dependent variables at a higher level, in addition to the papers mentioned on p. 9, also see
    • Margot Bennink, Marcel A. Croon and Jeroen K. Vermunt: "Micro-macro multilevel analysis for discrete data: a latent variable approach and an application on personal network data". Sociological Methods and Research 42 (2013), 431-457.

• Chapter 3
  
  • An article about the ecological fallacy:
    • Michela Gnaldi, Venera Tomaselli, and Antonio Forcina (2018). "Ecological fallacy and covariates: New insights based on multilevel modelling of individual data". International Statistical Review (2018), 86, 119-135.
      DOI: http://dx.doi.org/10.1111/insr.12244

• Chapter 4
  
  • Sections 4.3.1 and 4.6.
    Further and more detailed arguments against the use of the Hausman test as a pretest for deciding about the choice between a random effects and a fixed effects analysis are given in the following paper.
    • Paul Kabaila, Rheanna Mainzer and Davide Farchione (2017). Conditional assessment of the impact of a Hausman pretest on confidence intervals. Statistica Neerlandica, 71, 240-262.
      DOI: http://dx.doi.org/10.1111/stan.12109
  • Section 4.6
    The group average of the explanatory variables can be a quite unreliable estimate of the 'true' group mean. It may be better to treat this mean as a latent variable and use a method expressing this. See
    • Lüdtke, O., Marsh, H.W., Robitzsch, A., Trautwein, U., Asparouhov, T., and Muthén, B. (2008) 'The multilevel latent covariate model: a new, more reliable approach to group-level effects in contextual studies'. Psychological Methods, 13, 203-229.
  • Section 4.8.
    The empirical Bayes estimators (posterior means) have a smaller expected variance than the random intercepts/slopes that they are estimating (predicting). This, and methods to adjust for this, are reviewed in
    • Ghosh, Malay, and Tapabrata Maiti. "Adjusted Bayes estimators with applications to small area estimation." Sankhya: The Indian Journal of Statistics, Series B 61 (1999): 71-90.
    • Judkins, David R., and Jun Liu. "Correcting the bias in the range of a statistic across small areas." Journal of Official Statistics 16.1 (2000): 1-14.
    • Fabrizi, Enrico. "A comparison of adjusted Bayes estimators of an ensemble of small area parameters." Statistica 69.4 (2009): 269-284.

• Chapter 5
  
  • An important paper about the estimation of random slopes in situations where there is a potential correlation between the random slopes and a level-one covariate, is the following.
    • Michael David Bates, Katherine E. Castellano, Sophia Rabe-Hesketh, and Anders Skrondal (2014). Handling Correlations Between Covariates and Random Slopes in Multilevel Models. Journal of Educational and Behavioral Statistics, 39, 524-549.
      DOI: http://dx.doi.org/10.3102/1076998614559420

• Chapter 6
  
  • An important discussion about the specification of the random part of the model, focusing on factorial experiments but of wider relevance, was given by
    • Dale Barr, Random effects structure for testing interactions in linear mixed-effects models. Frontiers in Psychology 4 (2013), 328.
      with a critical response by
    • Douglas Bates, Reinhold Kliegl, Shravan Vasishth, and Harald Baayen (2015). Parsimonious mixed models. arXiv:1506.04967.
  • There now exist non-parametric tests for clustered data: e.g.,
    
    • Somnath Datta and Glen A. Satten (2005). Rank-sum tests for clustered data. Journal of the American Statistical Association 100, 908-915.
    • Somnath Datta and Glen A. Satten (2008). A signed-rank test for clustered data. Biometrics 64, 501-507.
    • Sandipan Dutta and Somnath Datta (2016). A rank-sum test for clustered data when the number of subjects in a group within a cluster is informative. Biometrics, 72, 432-440.
    • Bernard Rosner, Robert J. Glynn, and Mei-Ling T. Lee (2006). The Wilcoxon Signed Rank Test for Paired Comparisons of Clustered Data. Biometrics, 62, 185-192.
    • Bernard Rosner, Robert J. Glynn, and Mei-Ling T. Lee (2003). Incorporation of Clustering Effects for the Wilcoxon Rank Sum Test: A Large-Sample Approach. Biometrics, 59, 1089-1098.
    • Bernard Rosner, Robert J. Glynn, and Mei-Ling T. Lee (2006). Extension of the Rank Sum Test for Clustered Data: Two-Group Comparisons with Group. Biometrics, 62, 1251-1259.

      For these tests, there are the R packages ClusterRankTest and clusrank, both available from CRAN.

    • Jaakko Nevalainen, Denis Larocque, Hannu Oja, and Ilkka Pörsti (2010). "Nonparametric Analysis of Clustered Multivariate Data." Journal of the American Statistical Association 105, 864-872.
    • Joanna H. Shih and Michael P. Fay (2017). "Pearson's chi-square test and rank correlation inferences for clustered data". Biometrics, 73, 822-834.
      DOI: http://dx.doi.org/10.1111/biom.12653

• Chapter 8
  A paper with an interesting example of the substantive importance of heteroscedasticity is
  • M. Nicolaus, J.E. Brommer, R. Ubels, J.M. Tinbergen, and N.J. Dingemans (2013). Exploring patterns of variation in clutch size-density reaction norms in a wild passerine bird. Journal of Evolutionary Biology, 26, 2031-2043.

• Chapter 9
  Papers about treatment of missing data in multilevel studies:
  • James R. Carpenter and Michael G. Kenward (2013). Multilevel Multiple Imputation. Chapter 9 in Multiple Imputation and its Application, New York: Wiley.
    DOI: https://dx.doi.org/10.1002/9781119942283.ch9
  • Craig K. Enders, Han Du, and Brian T. Keller (2019). A model-based imputation procedure for multilevel regression models with random coefficients, interaction effects, and nonlinear terms. Psychological Methods, 25(1), 88–112.
    DOI: http://dx.doi.org/10.1037/met0000228
  • Craig K. Enders, Timothy Hayes, and Han Du (2018). A comparison of multilevel imputation schemes for random coefficient models: Fully conditional specification and joint model imputation with random covariance matrices. Multivariate behavioral research, 53.5: 695-713.
    DOI: http://dx.doi.org/10.1080/00273171.2018.1477040
  • Harvey Goldstein, James R. Carpenter, and William J. Browne (2014). Fitting multilevel multivariate models with missing data in responses and covariates that may include interactions and non-linear terms Journal of the Royal Statistical Society: Series A, 177, 553-564.
    DOI: http://dx.doi.org/10.1111/rssa.12022
  • Simon Grund, Oliver Lüdtke, and Alexander Robitzsch (2018). Multiple imputation of missing data for multilevel models: Simulations and recommendations. Organizational Research Methods, 21.1: 111-149.
    DOI: http://dx.doi.org/10.1177/1094428117703686
  • Simon Grund, Oliver Lüdtke, and Alexander Robitzsch (2019). Missing data in multilevel research. Chapter 16 in S. E. Humphrey and J. M. LeBreton (eds.), The Handbook of Multilevel Theory, Measurement, and Analysis, APA.
    https://simongrund1.github.io/publications/pdf/Grund_etal_2019_APA_Handbook.pdf
  • Oliver Lüdtke, Alexander Robitzsch, and Simon Grund (2017). Multiple imputation of missing data in multilevel designs: A comparison of different strategies. Psychological Methods, 22(1), 141–165.
    DOI: http://dx.doi.org/10.1037/met0000096
  • M. Quartagno, M., J. R. Carpenter, and H. Goldstein (2019). Multiple Imputation with Survey Weights: A Multilevel Approach. Journal of Survey Statistics and Methodology, 2019.
    DOI: http://dx.doi.org/10.1093/jssam/smz036
  Various software packages have appeared:
  • Timothy Hayes, Flexible, Free Software for Multilevel Multiple Imputation: A Review of Blimp and jomo (2019). Journal of Educational and Behavioral Statistics, 44.5: 625-641.
    DOI: http://dx.doi.org/10.3102/1076998619858624
  • Grund, Simon, Oliver Lüdtke, and Alexander Robitzsch (2016). Multiple imputation of multilevel missing data: An introduction to the R package pan. Sage Open , 6.4.
    DOI: http://dx.doi.org/10.1177/2158244016668220
  • Vincent Audigier (2019). micemd: Multiple Imputation by Chained Equations with Multilevel Data.
    https://rdrr.io/cran/micemd/man/micemd-package.html

• Chapter 10
  Interesting papers about checking assumptions in multilevel modeling:
  • Reza Drikvandi, Geert Verbeke and Geert Molenberghs (2017). Diagnosing Misspecification of the Random-Effects Distribution in Mixed Models. Biometrika, 73, 63-71.
    DOI: http://dx.doi.org/10.1111/biom.12551
  • Julio M. Singer, Francisco M.M. Rocha, and Juvencio S. Nobre (2017). Graphical Tools for Detecting Departures from Linear Mixed Model Assumptions and Some Remedial Measures. International Statistical Review, 85, 290-324.

• Chapter 11
  Two books about sample sizes and power in multilevel studies:
  • Mirjam Moerbeek and Steven Teerenstra. Power Analysis of Trials with Multilevel Data (2016). Boca Raton, FL: CRC Press.
  • Chul Ahn, Moonseoung Heo, and Song Zhang (2014). Sample Size Calculations for Clustered and Longitudinal Outcomes in Clinical Research. Boca Raton, FL: CRC Press.
  Interesting papers:
  • Christopher Rhoads (2017). Coherent Power Analysis in Multilevel Studies Using Parameters From Surveys. Journal of Educational and Behavioral Statistics, 42, 166-194.
    DOI: http://dx.doi.org/10.3102/1076998616675607.
  • Anup Amatya and Dulal K. Bhaumik (2018). Sample size determination for multilevel hierarchical designs using generalized linear mixed models. Biometrics, 74, 673-684. DOI: http://dx.doi.org/10.1111/biom.12764.
    Unfortunately, they do not refer to the work by Raudenbush, Rhoads, Moerbeek, and others.

• Chapter 12
  For the sandwich estimator (Section 12.2), two good reviews are:
  • Cameron, A. Colin, and Douglas L. Miller. "Robust inference with clustered data." In: Amon Ullah and David E.A. Giles (Eds.), Handbook of Empirical Economics and Finance, pp. 1-28. Boca Raton: Chapman & Hall / CRC (2016).
  • MacKinnon, James G. "Thirty years of heteroskedasticity-robust inference." In: Recent Advances and Future Directions in Causality, Prediction, and Specification Analysis, pp. 437-461. New York: Springer (2013).

  A paper contrasting the use of multilevel (hierarchical linear) models with methods that control in a simpler way for clustering of data, e.g., the sandwich estimator:

  • Daniel McNeish, Laura M. Stapleton, and Rebecca D. Silverman (2017). "On the unnecessary ubiquity of hierarchical linear modeling." Psychological Methods 22, 114-140.
    DOI: http://dx.doi.org/0.1037/met0000078

  A different approach to latent class modeling:

  • Gerhard Tutz and Margret-Ruth Oelker (2017). Modelling Clustered Heterogeneity: Fixed Effects, Random Effects and Mixtures. International Statistical Review, 85, 204-227.

• Chapter 14
  Some additional papers on weighting in multilevel models are the following.
  • Anderson, Carolyn J., Jee-Seon Kim, and Bryan Keller (2014). Multilevel modeling of categorical response variables. In L. Rutkowski, M. von Davier, and D. Rutkowski (Eds.), Handbook of International Large-scale Assessment: background, technical issues, and methods of data analysis (pp. 481-519). Boca Raton, London, New York: Chapman and Hall/CRC.
  • Chantala, K., Blanchette, D., and Suchindran, C. M. (2011). Software to compute sampling weights for multilevel analysis. Carolina Population Center, University of North Carolina at Chapel Hill.
  • Leite, W. L., Jimenez, F., Kaya, Y., Stapleton, L. M., Macinnes, J. W., and Sandbach, R. (2015). An evaluation of weighting methods based on propensity scores to reduce selection bias in multilevel observational studies. Multivariate Behavioral Research, 50, 265-284.
    DOI: http://dx.doi.org/10.1080/00273171.2014.991018
  • Stapleton, Laura M. (2013). Incorporating Sampling Weights into Single- and Multilevel Analyses. In L. Rutkowski, M. von Davier, and D. Rutkowski (Eds.), Handbook of International Large-scale Assessment: background, technical issues, and methods of data analysis (pp. 363-388). Boca Raton, London, New York: Chapman and Hall/CRC.
  • Stapleton, Laura M., Jeffrey R. Harring, and Danuel Y. Lee (2016). Sampling Weight Considerations for Multilevel Modeling of Survey Data. In Jeffrey R. Harring, Laura M. Stapleton, and S. Natasha Beretvas (Eds.), Advances in Multilevel Modeling for Educational Research (pp. 63-95). Charlotte, NC: Information Age Publishing.
  • Stapleton, Laura M. and Yoonjeong Kang (2016). Design Effects of Multilevel Estimates From National Probability Samples. Sociological Methods & Research, 47, 430-457.
    DOI: http://dx.doi.org/10.1177/0049124116630563
  • West, B. and Gałecki, A. (2011). An overview of current software procedures for fitting linear mixed models. American Statistician, 65, 274-282.

• Chapter 15
  
  • A book about (among other topics) longitudinal data with informative information times:
    Robert Elashoff, Gang Li, Ning Li (2016). Joint Modeling of Longitudinal and Time-to-Event Data. Boca Raton: CRC Press.

• Chapter 17
  
  • The following paper gives a general definition for a coefficient of explained proportion of variance in random intercept hierarchical generalized linear models. They also follow McKelvey and Zavoina's (1975) approach. However, their proposal covers also the case of Poisson distributions although the latent variable is not clearly defined for this case. This seems questionable.
    S. Nakagawa and H. Schielzeth (2013). A general and simple method for obtaining R^2 from generalized linear mixed-effects models. Methods in Ecology and Evolution, 4, 133-142.
  • A review of multilevel event history modeling:
    Peter C. Austin (2017). A Tutorial on Multilevel Survival Analysis: Methods, Models and Applications. International Statistical Review , 85, 185-203.
  • Hierarchical models for Likert scales that separate the neutral category and further use an ordinal model:
    Gerhard Tutz (2021). Hierarchical Models for the Analysis of Likert Scales in Regression and Item Response Analysis. International Statistical Review, 89: 18-35.
    DOI: http://dx.doi.org/10.1111/insr.12396

• Chapter 18
  Also see:
  • West, B. and Gałecki, A. (2011). An overview of current software procedures for fitting linear mixed models. American Statistician, 65, 274-282.

• Section 18.1
  Other software:
  • ASReml , software for fitting linear mixed models specially designed for large data sets. It is optimised for working with genetics data. There is also a version ASReml-R making it available (but not for free!) in the R statistical system.

• Section 18.1.2
  
  • There is a lot of helpful material for MLWiN at the MLwiN website.
  • The book by Leyland and Groenewegen mentioned above has extensive tutorials for general and logistic multilevel modelling with MLwiN.
  • MLwiN can also be run from within R (if MLwiN as well as R are installed), using the R package R2MLwiN.

• Section 18.2.2
  
  • Some websites and resources that can be very helpful for the beginning R user:
    1. General information about R from UCLA.
    2. Impatient R, previously Some Hints for the R Beginner, by Patrick Burns with the memorable quote
       "I asked R users what their biggest stumbling blocks were in learning R. A common answer that I was quite surprised by was that the biggest stumbling block was thinking that R was hard".
    3. icebreakeR by Andrew Robinson.
    4. Quick R website (intended especially for experienced users of other statistical software).
    5. R-bloggers tutorials for learning R by Tal Galili.
    6. The official R intro.

  • There is a Frequently Asked Questions list associated with the r-sig-mixed-models mailing list, maintained by Ben Bolker, which contains a lot of interesting stuff. E.g., a list of R packages available for multilevel modeling.
    Also see their comparison page for multilevel packages.
  • The book
    Andrzej Gałecki and Tomasz Burzykowksi (2013). Linear mixed-effects models using R; A step-by-step approach. New York: Springer
    presents extensively the use of nlme, lme4 and RLRsim for estimating and testing hierarchical linear models.
    There is a package nlmeU which contains all examples in the book.
  • A briefer treatment and user-friendly of the multilevel possibilities in R is given by W. Holmes Finch, Jocelyn E. Bolin, and Ken Kelley, Multilevel Modeling Using R, 2nd ed. (2019). Boca Raton, FL: CRC Press.
  • The group developing lme4 (Douglas Bates, Martin Maechler, Ben Bolker, Steven Walker) has a helpful page with resources (slides, articles, draft chapters of a book with Sweave code, etc.).
  • These authors have an extensive paper Fitting linear mixed-effects models using lme4 (June 2014), Journal of Statistical Software, 67(1), 1-48;
    which has statistical background and also shows a lot of code for using many goodies in lme4.
  • For a blog about the choice between lme4 and nlme, see Mixed Models in R: lme4, nlme, or both? by Altea Lorenzo.

  Other R packages

  • ordinal
    For multilevel ordered regression models, proportional odds models, proportional hazards models for grouped survival times and ordered logit/probit/... models. Estimation is via maximum likelihood and mixed models are fitted with the Laplace approximation and adaptive Gauss-Hermite quadrature. Multiple random effect terms are allowed and they may be nested, crossed or partially nested/crossed. See the vignettes which give a good introduction.
    Examples are also in Finch, Bolin and Kelley (2014), mentioned above.

    A function to calculate the explained variance and residual intraclass-correlation (as treated in Section 17.4) for estimates obtained from ordinal is contained at the end of script ch17.r.

  • robustlmm
    A package to fit linear mixed effects models robustly. The package is described in
    Manuel Koller (2016). "robustlmm: An R Package for Robust Estimation of Linear Mixed-Effects Models", Journal of Statistical Software, 75 (6), 1-24.
    DOI: http://dx.doi.org/10.18637/jss.v075.i06
  • RLRsim
    RLRsim is a package for exact testing of random effect variances and nonparametric regression functions in Additive Mixed Models. See the tutorial introduction.
  • lmerTest
    Different kinds of tests for linear mixed effects models as implemented in the lme4 package are provided. The tests comprise types I - III F tests for fixed effects, LR tests for random effects. The package also provides the calculation of population means for fixed factors with confidence intervals and corresponding plots. Finally the backward elimination of non-significant effects is implemented.
  • nlmeVPC: Visual Model Diagnosis for the Nonlinear Mixed Effect Model
    Various visual and numerical diagnosis methods for the nonlinear mixed effect model, including visual predictive checks, numerical predictive checks, and coverage plots. An overview is given in the R Journal.
  • MCMCglmm
    MCMCglmm is a package for fitting Generalised Linear Mixed Models using Markov chain Monte Carlo techniques. Most commonly used distributions like the normal and the Poisson are supported together with some useful but less popular ones like the zero-inflated Poisson and the multinomial. Missing values and left, right and interval censoring are accommodated for all traits. The package also supports multi-trait models where the multiple responses can follow different types of distribution. The package allows various residual and random-effect variance structures to be specified including heterogeneous variances, unstructured covariance matrices and random regression (e.g. random slope models). Three special types of variance structure that can be specified are those associated with pedigrees (animal models), phylogenies (the comparative method) and measurement error (meta-analysis).
    See:
    Hadfield, J. D. (2010). MCMC methods for multi-response generalized linear mixed models: the MCMCglmm R package. Journal of Statistical Software, 33(2), 1-22.
    
  • glmmAK
    This package implements Bayesian MCMC estimation for the logistic and Poisson regression models with random effects.
  • brms
    Fits Bayesian generalized (non-)linear multilevel models using Stan for full Bayesian inference. A wide range of distributions and link functions are supported, allowing users to fit - among others - linear, robust linear, count data, survival, response times, ordinal, zero-inflated, hurdle, and even self-defined mixture models all in a multilevel context.
    The author has a list of blog posts about brms.
  • TMB: Template Model Builder: A General Random Effect Tool Inspired by ADMB
    TMB (Template Model Builder) is an R package for fitting statistical latent variable models to data. It is strongly inspired by ADMB, a package for automatic differentiation, and uses the Laplace approximation. Unlike most other R packages the model is formulated in C++. This provides great flexibility, but requires some familiarity with the C/C++ programming language.
    There is a TMB wiki.
  • lmeresampler:: Bootstrap Methods for Nested Linear Mixed-Effects Models
    Bootstrap routines for nested linear mixed effects models fit using either 'lme4' or 'nlme'. An overview is given in the R Journal.

    Several packages about imputation of missing data:

  • The package mitml: Tools for Multiple Imputation in Multilevel Modeling provides tools for multiple imputation of missing data in multilevel modeling. Includes a user-friendly interface to the packages 'pan' and 'jomo'.
  • Similarly to Schafer's package pan , jomo is a package for multilevel joint modelling multiple imputation (Carpenter and Kenward, 2013).
  • Vincent Audigier (2019). micemd: Multiple Imputation by Chained Equations with Multilevel Data.
    DOI: https://rdrr.io/cran/micemd/man/micemd-package.html
  • The package blimp by Craig Enders, Brian Keller, and Han Du, offers general-purpose Bayesian estimation for a wide range of single-level and multilevel regression models with two or three levels, with or without missing data. In particular, it offers estimation routines for interactive and polynomial effects that are not yet available in other software packages, including support for binary, ordinal, and nominal variables.

  • ClusterRankTest and clusrank, for non-parametric comparison of clustered data.

• Section 18.2.3
  • Stata 14 has a suite of commands memeglm for the Multilevel mixed-effects generalized linear model. Seven distributions for the response variable are supported (Gaussian, Bernoulli, binomial, gamma, negative binomial, ordinal, and Poisson); and five link functions are possible (identity, log, logit, probit, and complementary log-log). As examples, see models for multilevel ordered logit , multilevel multinomial logit , and multilevel Poisson models.

• Section 18.3
  • For Mplus (see Section 18.3.4), a good guide is
    W. Holmes Finch and Jocelyn E. Bolin, Multilevel Modeling Using Mplus (2017). Boca Raton, FL: CRC Press.
  • The program SPA-ML (Statistical power for multilevel designs) is described in
    Mirjam Moerbeek and Steven Teerenstra, Power Analysis of Trials with Multilevel Data (2016). Boca Raton, FL: CRC Press.

Corrections

• Example 3.1 and 3.2: the observed between variance is 105.6499, which should be rounded to 105.6.
• On p. 24-25, the effective sample sizes indicated by N_effective, N_ts, and N_srs, might more clearly have been denoted by M_effective, M_ts, and M_srs, because further the convention is used to indicate level-two sample sizes by N and total sample sizes by M.
• On p. 35, line 8 from the bottom, the formula in the middle of the line should be hat(rho_b) = 13.77/sqrt(12.45 * 17.86) = 0.923.
• The first part of Example 6.4 (p. 99) is incorrect. It was copied from the first edition without updating...

  The text should be:
  When comparing Tables 4.4 and 5.1, it can be concluded that m1 - m0 = 2 parameters are added and the deviance diminishes by D0 - D1 = 24888.02 - 24864.87 = 23.15. Testing the value of 23.15 in Table 6.2 for p=1 yields p < 0.001. Thus, the significance probability of the random slope for IQ in the model of Table 5.1 is p < 0.001.

• On p. 129, line 11, (8.8) should be (8.7).
• On p. 142, 9 lines from bottom, "K = 5" should be "M = 5"; this is about number of imputations, not sampling frequency.
• On p. 291, line 22, the reference to Section 17.3.2 should be to Section 17.3.3.
• On p. 293, in Example 17.1, the country with the largest number of respondents is Spain (not the Russian Federation) with 7,745; the country with the lowest proportion of attendance is China (not Denmark) with 0.006.
• On p. 298, in Example 17.3, the definition of the educational level should say "minus 17" rather than "minus 14". (The change affects only the intercept of the logistic regression model, and the average country-level educational of 0.82 given at the top of page 299; these are both correct for "minus 17".)
• On p. 299, in Table 17.2, the coefficient for gamma_10, the country average education, should be -0.359. This error comes back in the formula in Example 17.5, but the results there are correct.
• On p.303, the title for Table 17.3 should say "Logistic random slopes model" rather than "Logistic random intercept model". In the script ch17.r, a different centering is applied for Example 17.4, which gives a slightly better stability of the estimation.
• Example 17.7 continues the analysis of Example 17.6, not of 17.5.

Materials Oxford Spring School, April 2012