Introducing Concerto, an open-source platform designed to realise the potential of modern measurement theories

Abstract

Item response theory (IRT) models and algorithms for computer adaptive testing were originally developed in the 1960s. However, their widespread use was restricted by available computer processing power, lack of suitable software for conducting IRT analyses and, until recently; the absence of any tool for administering questionnaires within an IRT framework. In 2011, the open-source Concerto platform was released to allow psychologists to develop and administer questionnaires and create flexible computer adaptive tests which include automatic scoring and tailored feedback. Computer adaptive testing (CAT) allows measurement to be better targeted, more efficient (shorter) and more accurate (reliable) than standard paper-based measurement. The open-source accessibility of Concerto means that CATs are readily available for any researcher in an accessible and easy-to-use system, which still maintains the capacity to apply advanced measurement theories. The talk will introduce three recent implementations of Concerto in health-care settings. Concerto hosts a computer adaptive version of the World Health Organisation Quality of Life -100 scale, which is significantly shorter than the paper-based version and provides tailored graphical and text feedback. In the USA, researchers have created the Movement Ability Measure, an adaptive test which assesses the disparity between people’s current and ideal functional capacity, with clear feedback. In higher-stakes assessment, Concerto is being used by the University of Exeter as the platform for a PROM-based clinical intervention that combines standard and adaptive assessment with feedback linked clinical practice guidelines.