Are your questionnaires good enough? Reassessing scale validity, reliability and generating population norms using meta-analysis

Abstract

Background: Standardised measures are the cornerstone of quantitative health psychology research. After the initial validation efforts, the psychometric properties and construct validity of such questionnaires are rarely reassessed. This is problematic as scales can soon become ‘gold standard’ after being assessed on a sample that is restricted in range and size. Method This paper demonstrates how meta-analysis may be used to assess the quality of a questionnaire. This is achieved by examining all papers citing a scale that publish at least one of the following; scale mean, standard deviation, reliability coefficient or correlation with another measure. These are weighted by sample size, assessed for risk of bias and meta-analysed to produce new scale norms and reliability information. Published correlation coefficients are used to generate a qualitative commentary of a scale’s construct validity. This approach is illustrated by an analysis conducted on 89 studies utilising the Short Health Anxiety Inventory (SHAI). Findings: SHAI mean scores were highest in hypochondriacal populations, followed by the medically unwell and university students. Internal consistency coefficients ranged from 0.76-0.97. As predicted the SHAI was correlated most highly with other measures of health anxiety, followed by general anxiety measures. Discussion: SHAI scores vary consistently in different populations; the measure has dependable psychometric properties including internal consistency. A narrative review of correlations with other measures indicated the SHAI has good construct validity. This paper argues this method of analysis is useful for health psychology researchers and can be easily applied to other standardised measures used in the field.