Analysis of single-case experimental count data using the linear mixed effects model: A simulation study

Lies Declercq, Laleh Jamshidi, Belén Fernández-Castilla, S Natasha Beretvas, Mariola Moeyaert, John M. Ferron, Wim Van den Noortgate

Research output: Contribution to journalArticle

Abstract

When (meta-)analyzing single-case experimental design (SCED) studies by means of hierarchical or multilevel modeling, applied researchers almost exclusively rely on the linear mixed model (LMM). This type of model assumes that the residuals are normally distributed. However, very often SCED studies consider outcomes of a discrete rather than a continuous nature, like counts, percentages or rates. In those cases the normality assumption does not hold. The LMM can be extended into a generalized linear mixed model (GLMM), which can account for the discrete nature of SCED count data. In this simulation study, we look at the effects of misspecifying an LMM for SCED count data simulated according to a GLMM. We compare the performance of a misspecified LMM and of a GLMM in terms of goodness of fit, fixed effect parameter recovery, type I error rate, and power. Because the LMM and the GLMM do not estimate identical fixed effects, we provide a transformation to compare the fixed effect parameter recovery. The results show that, compared to the GLMM, the LMM has worse performance in terms of goodness of fit and power. Performance in terms of fixed effect parameter recovery is equally good for both models, and in terms of type I error rate the LMM performs better than the GLMM. Finally, we provide some guidelines for applied researchers about aspects to consider when using an LMM for analyzing SCED count data.

LanguageEnglish (US)
JournalBehavior Research Methods
DOIs
StateAccepted/In press - Jan 1 2018

Fingerprint

Linear Models
Research Design
Simulation
Mixed Effects Model
Research Personnel
Outcome Assessment (Health Care)
Guidelines
Experimental Design

Keywords

  • Generalized linear mixed model
  • Linear mixed model
  • Monte Carlo simulation
  • Single-case experimental design

ASJC Scopus subject areas

  • Experimental and Cognitive Psychology
  • Developmental and Educational Psychology
  • Arts and Humanities (miscellaneous)
  • Psychology (miscellaneous)
  • Psychology(all)

Cite this

Declercq, L., Jamshidi, L., Fernández-Castilla, B., Beretvas, S. N., Moeyaert, M., Ferron, J. M., & Van den Noortgate, W. (Accepted/In press). Analysis of single-case experimental count data using the linear mixed effects model: A simulation study. Behavior Research Methods. https://doi.org/10.3758/s13428-018-1091-y

Analysis of single-case experimental count data using the linear mixed effects model : A simulation study. / Declercq, Lies; Jamshidi, Laleh; Fernández-Castilla, Belén; Beretvas, S Natasha; Moeyaert, Mariola; Ferron, John M.; Van den Noortgate, Wim.

In: Behavior Research Methods, 01.01.2018.

Research output: Contribution to journalArticle

Declercq, Lies ; Jamshidi, Laleh ; Fernández-Castilla, Belén ; Beretvas, S Natasha ; Moeyaert, Mariola ; Ferron, John M. ; Van den Noortgate, Wim. / Analysis of single-case experimental count data using the linear mixed effects model : A simulation study. In: Behavior Research Methods. 2018.
@article{0c63d4b95802418dbe3adf9fbb0cbdae,
title = "Analysis of single-case experimental count data using the linear mixed effects model: A simulation study",
abstract = "When (meta-)analyzing single-case experimental design (SCED) studies by means of hierarchical or multilevel modeling, applied researchers almost exclusively rely on the linear mixed model (LMM). This type of model assumes that the residuals are normally distributed. However, very often SCED studies consider outcomes of a discrete rather than a continuous nature, like counts, percentages or rates. In those cases the normality assumption does not hold. The LMM can be extended into a generalized linear mixed model (GLMM), which can account for the discrete nature of SCED count data. In this simulation study, we look at the effects of misspecifying an LMM for SCED count data simulated according to a GLMM. We compare the performance of a misspecified LMM and of a GLMM in terms of goodness of fit, fixed effect parameter recovery, type I error rate, and power. Because the LMM and the GLMM do not estimate identical fixed effects, we provide a transformation to compare the fixed effect parameter recovery. The results show that, compared to the GLMM, the LMM has worse performance in terms of goodness of fit and power. Performance in terms of fixed effect parameter recovery is equally good for both models, and in terms of type I error rate the LMM performs better than the GLMM. Finally, we provide some guidelines for applied researchers about aspects to consider when using an LMM for analyzing SCED count data.",
keywords = "Generalized linear mixed model, Linear mixed model, Monte Carlo simulation, Single-case experimental design",
author = "Lies Declercq and Laleh Jamshidi and Bel{\'e}n Fern{\'a}ndez-Castilla and Beretvas, {S Natasha} and Mariola Moeyaert and Ferron, {John M.} and {Van den Noortgate}, Wim",
year = "2018",
month = "1",
day = "1",
doi = "10.3758/s13428-018-1091-y",
language = "English (US)",
journal = "Behavior Research Methods",
issn = "1554-351X",
publisher = "Springer New York",

}

TY - JOUR

T1 - Analysis of single-case experimental count data using the linear mixed effects model

T2 - Behavior Research Methods

AU - Declercq, Lies

AU - Jamshidi, Laleh

AU - Fernández-Castilla, Belén

AU - Beretvas, S Natasha

AU - Moeyaert, Mariola

AU - Ferron, John M.

AU - Van den Noortgate, Wim

PY - 2018/1/1

Y1 - 2018/1/1

N2 - When (meta-)analyzing single-case experimental design (SCED) studies by means of hierarchical or multilevel modeling, applied researchers almost exclusively rely on the linear mixed model (LMM). This type of model assumes that the residuals are normally distributed. However, very often SCED studies consider outcomes of a discrete rather than a continuous nature, like counts, percentages or rates. In those cases the normality assumption does not hold. The LMM can be extended into a generalized linear mixed model (GLMM), which can account for the discrete nature of SCED count data. In this simulation study, we look at the effects of misspecifying an LMM for SCED count data simulated according to a GLMM. We compare the performance of a misspecified LMM and of a GLMM in terms of goodness of fit, fixed effect parameter recovery, type I error rate, and power. Because the LMM and the GLMM do not estimate identical fixed effects, we provide a transformation to compare the fixed effect parameter recovery. The results show that, compared to the GLMM, the LMM has worse performance in terms of goodness of fit and power. Performance in terms of fixed effect parameter recovery is equally good for both models, and in terms of type I error rate the LMM performs better than the GLMM. Finally, we provide some guidelines for applied researchers about aspects to consider when using an LMM for analyzing SCED count data.

AB - When (meta-)analyzing single-case experimental design (SCED) studies by means of hierarchical or multilevel modeling, applied researchers almost exclusively rely on the linear mixed model (LMM). This type of model assumes that the residuals are normally distributed. However, very often SCED studies consider outcomes of a discrete rather than a continuous nature, like counts, percentages or rates. In those cases the normality assumption does not hold. The LMM can be extended into a generalized linear mixed model (GLMM), which can account for the discrete nature of SCED count data. In this simulation study, we look at the effects of misspecifying an LMM for SCED count data simulated according to a GLMM. We compare the performance of a misspecified LMM and of a GLMM in terms of goodness of fit, fixed effect parameter recovery, type I error rate, and power. Because the LMM and the GLMM do not estimate identical fixed effects, we provide a transformation to compare the fixed effect parameter recovery. The results show that, compared to the GLMM, the LMM has worse performance in terms of goodness of fit and power. Performance in terms of fixed effect parameter recovery is equally good for both models, and in terms of type I error rate the LMM performs better than the GLMM. Finally, we provide some guidelines for applied researchers about aspects to consider when using an LMM for analyzing SCED count data.

KW - Generalized linear mixed model

KW - Linear mixed model

KW - Monte Carlo simulation

KW - Single-case experimental design

UR - http://www.scopus.com/inward/record.url?scp=85052213100&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85052213100&partnerID=8YFLogxK

U2 - 10.3758/s13428-018-1091-y

DO - 10.3758/s13428-018-1091-y

M3 - Article

JO - Behavior Research Methods

JF - Behavior Research Methods

SN - 1554-351X

ER -