Rasch Analyses with Polytomous Data: Do RUMM2030 and Jamovi Produce the Same Results?

Details

Supervisors

Penta, Massimo

Faculty

Faculté de psychologie, logopédie, sexologie et des sciences de la famille

Degree label

Master [120] en sciences psychologiques, à finalité approfondie

Abstract

The Rasch model, developed in the 1960s, offers a rigorous framework for converting ordinal scores—where responses can be ranked but distances between ranks are unequal—into interval-level measures, where score differences are consistent and meaningful across the scale. It also enables verification of key statistical requirements for objective measurement, ensuring consistency of results across individuals and instruments. This study presents the first comprehensive, real-data comparison of two platforms implementing the polytomous Rasch model: RUMM2030, a long-established commercial software widely recognized for its validity, and Jamovi, a free and open-source platform gaining traction in applied research. The aim was to determine whether both tools produce comparable results and conclusions when applied to the same dataset. The dataset comprised responses from 316 first-year psychology students at the University of Leuven to 24 polytomous items on a verbal aggression scale, scored on a three-point Likert scale. Ten analytical components were compared, most directly assessing criteria for objective measurement: (1) model selection, (2) overall model fit, (3) reliability, (4) individual item and person fit, (5) item and person locations, (6) Bond–Fox plots for items, (7) ordered thresholds, (8) local item independence, (9) unidimensionality, and (10) differential item functioning (DIF). Findings revealed a high degree of concordance. Both platforms selected the partial credit model, reported similar overall fit and reliability, and produced closely aligned item fit results. Item and person location estimates were highly consistent, with only minor deviations at the extremes; Jamovi displayed a slightly broader measurement range. Bond–Fox plots confirmed this agreement for items. Category thresholds were identically ordered, local dependence detection was identical, unidimensionality tests supported a single latent dimension, and DIF results showed only limited discrepancies. The main divergence was in person fit, with Jamovi identifying substantially more misfitting cases. Overall, Jamovi can, in many respects, produce results comparable to RUMM2030. However, to achieve even closer alignment, it must address the limitations identified in this study—enhancements that could establish it as a practical, accessible, and scientifically robust alternative for Rasch analysis, and expand its adoption across diverse research contexts.