How to Model and Interpret Cross-Lagged Effects in Psychotherapy Mechanisms of Change Research : A Comparison of Multilevel and Structural Equation Models

• Objective: Modeling cross-lagged effects in psychotherapy mechanisms of change studies is complex and requires careful attention to model selection and interpretation. However, there is a lack of field-specific guidelines. We aimed to (a) describe the estimation and interpretation of cross lagged effects using multilevel models (MLM) and random-intercept cross lagged panel model (RI-CLPM); (b) compare these models' performance and risk of bias using simulations and an applied research example to formulate recommendations for practice. Method: Part 1 is a tutorial focused on introducing/describing dynamic effects in the form of autoregression and bidirectionality. In Part 2, we compare the estimation of cross-lagged effects in RI-CLPM, which takes dynamic effects into account, with three commonly used MLMs that cannot accommodate dynamics. In Part 3, we describe a Monte Carlo simulation study testing model performance of RI-CLPM and MLM under realistic conditions for psychotherapy mechanisms of change studies. Results: Our findings suggested that all three MLMs resulted in severely biased estimates of cross-lagged effects when dynamic effects were present in the data, with some experimental conditions generating statistically significant estimates in the wrong direction. MLMs performed comparably well only in conditions which are conceptually unrealistic for psychotherapy mechanisms of change research (i.e., no inertia in variables and no bidirectional effects). Discussion: Based on conceptual fit and our simulation results, we strongly recommend using fully dynamic structural equation modeling models, such as the RI-CLPM, rather than static, unidirectional regression models (e.g., MLM) to study cross-lagged effects in mechanisms of change research. What is the public health significance of this article? We describe the differences between multilevel and structural equation modeling in the study of mechanisms of change in psychotherapy research. We argue that the common application of multilevel modeling assumes that there is no within-patient inertia in predictor or outcome variable, and the outcome variable does not impact the predictor, both of which seem highly unrealistic in psychotherapy research. Moreover, we demonstrate that violations of these assumptions may lead to severe bias in estimated coefficients, resulting in inaccurate recommendations for clinical practice. Thus, we recommend researchers to use structural equation modeling to estimate the effects of proposed change mechanisms over time.

Ämnesord

SAMHÄLLSVETENSKAP  -- Psykologi (hsv//swe)

SOCIAL SCIENCES  -- Psychology (hsv//eng)

SAMHÄLLSVETENSKAP  -- Psykologi -- Tillämpad psykologi (hsv//swe)

SOCIAL SCIENCES  -- Psychology -- Applied Psychology (hsv//eng)

Nyckelord

process research

statistical methods

mechanisms of change

outcome research

prediction models

Psychology

Psykologi

Publikations- och innehållstyp

ref (ämneskategori)

art (ämneskategori)