Research Articles
Validation of self-reported chronic conditions and health services in a managed care population1

https://doi.org/10.1016/S0749-3797(99)00158-0Get rights and content

Abstract

Background: Self-reported data are commonly used to estimate the prevalence of health conditions and the use of preventive health services in the population, but the validity of such data is often questioned.

Methods: The Behavioral Risk Factor Survey (BRFS) was administered by telephone to a stratified, random sample of health maintenance organization (HMO) subscribers in Colorado in 1993, and self-reports were compared with HMO medical records for 599 adults aged >21. Sensitivity and specificity were calculated for three chronic conditions and use of six preventive services.

Results: Sensitivity was highest for hypertension (83%), moderate for diabetes (73%), and lowest for hypercholesterolemia (59%); specificity was >80% for all three conditions. Sensitivity ranged from 86% to 99% for influenza immunization, clinical breast examination, blood cholesterol screening, mammography, Pap test, and blood pressure screening; specificity was <75% for all preventive services.

Conclusions: Self-reports are reasonably accurate for certain chronic conditions and for routine screening exams and can provide a useful estimate for broad measures of population prevalence.

Introduction

T elephone surveys are widely used to collect information about the health characteristics of a population. Although the method is cost-effective and efficient, the validity of the information collected is often questioned. Overestimates of some screening procedures and underreporting of medical conditions and risk behaviors have been attributed to survey data.1, 2, 3, 4, 5, 6 However, studies also have shown reasonable estimates of disease prevalence7 and trends over time3 based on self-reports. Much of the validation research on self-reported data for use of screening procedures has focused on mammography and Pap tests.2, 3, 8, 9, 10, 11, 12, 13, 14 Some studies have shown that self-reported data for these screening tests are generally valid.8, 9 Reporting the date of the screening examination is more problematic,3, 8, 10, 11, 12 and self-reports may significantly overestimate the use of screening,2 especially among some minority populations.12, 13, 14

The Behavioral Risk Factor Survey (BRFS) is a population-based telephone survey that provides state health departments with data for estimating prevalence of disease risk factors and for use in preventive clinical services.15 Interview information from the BRFS and direct patient examination have been used in a few studies to validate self-reported risk factors for cardiovascular disease.4, 16, 17 However, a comprehensive comparison of BRFS data with medical record data has never been undertaken. Information about the validity of self-reports from a standardized survey like BRFS would be especially useful for health plans in developing performance and accountability measures for use in plan evaluation systems such as the Health Plan Employer Data and Information Set (HEDIS).18

For this study, we examined the validity of self-reported data for the prevalence of three chronic health conditions (hypertension, hypercholesterolemia, and diabetes) and for the use of six preventive health services (blood pressure screening, blood cholesterol screening, clinical breast examination, mammography, Pap test, and influenza immunization) by comparing telephone survey findings with respondents’ medical records in a managed care population.

Section snippets

Methods

In 1993, the Colorado Department of Public Health and Environment collaborated with Kaiser Permanente of Colorado (KPC) to conduct a special BRFS. The 1993 BRFS instrument, including questions regarding whether each preventive service received was performed at KPC or at some other service delivery site, was administered by telephone to a random sample of the KPC managed care population aged >21 years who had been members of KPC for at least 1 year. The sample was stratified by six age and

Results

For the three health conditions, sensitivity was highest for hypertension (83% of respondents with a medical record of hypertension reported the condition on the survey), moderate for diabetes (73%), and lowest for hypercholesterolemia (59%) (Table 1). One third of the respondents who were not sure whether their blood cholesterol had ever been checked had a medical record of hypercholesterolemia (data not shown). Specificity was high for all three conditions, especially for diabetes (99% of

Discussion

We found that sensitivity and specificity for self-reported chronic conditions were more consistent than for self-reported use of preventive clinical services. Self-reports were reasonably accurate for several routine screening tests: blood pressure screening, blood cholesterol screening, mammography, and Pap test. Although sensitivity for preventive services was high, specificity was quite low for most of the measures. However, the low specificity estimates are partially explained by the small

Acknowledgements

This project was made possible in part through grants from the Centers for Disease Control and Prevention and the Kaiser Foundation Health Plan of Colorado.

Analytic assistance and support from the Council of State and Territorial Epidemiologists is acknowledged.

References (22)

  • M.B. Mittlemark et al.

    Prevalence of cardiovascular diseases among older adultsthe Cardiovascular Health Study

    Am J Epidemiol

    (1993)
  • Cited by (0)

    1

    The full text of this article is available at http://www.elsevier.com/locate/ajpmonline.

    View full text