Developing an operational definition and a self-report measure of engagement with e-/mHealth interventions: critical reflections

Abstract

Background: The plethora of existing definitions and related measures of engagement with digital behaviour change interventions hinders the aggregation of data from multiple studies to address the question as to what mechanisms link engagement with successful behaviour change. The aims of this study were to develop 1) an operational definition of engagement with e-/mHealth interventions and 2) a self-report scale to measure the construct of interest. Methods: Informed by classical test theory, the operational definition of engagement was developed using a combination of a deductive (i.e. a systematic review) and an inductive (i.e. ‘think aloud’ and interviews) approach. A pragmatic approach to item generation and scaling was taken. A “sorting taskâ€, asking participants to assign items to one of a finite number of categories, was designed to assess the content validity of initial scale items. Findings: Based on the two different approaches to construct development, engagement was conceptualised as a state-like construct, manifested by the following subdimensions: attention, interest, enjoyment, ‘amount of use’ and ‘depth of use’. Eighteen initial items were generated, scored using 7-point Likert scales where possible. Results from the first round of the “sorting task†(N = 51) suggested that task instructions may not have been sufficiently clear, as a low proportion of participants correctly classified reverse-worded items. Discussion: Key decisions made during the development of an operational definition and measure of engagement with e-/mHealth interventions are reflected on critically, highlighting issues of construct development, item generation, item scaling and content validity.