presented by Karen L. McCulloch
This course provides an overview of psychometric properties important for selection of outcome measures, and characteristics of measures that aid in their interpretability and clinical usefulness. The approach of item response theory will be described as a tool to refine developing measures so that they can be shorter and represent a hierarchy of item difficulty. This approach also may facilitate the transition to computer assisted measurement. Consensus-oriented processes and resources will be reviewed that provide therapists with information about ideal measures for different adult neurologic conditions. The application of these principles to cases or practice settings is addressed via knowledge check examples.
Karen L. McCulloch, PhD, PT, MS, NCS, is a Professor in Physical Therapy in the Division of Physical Therapy, Department of Allied Health Sciences, School of Medicine at University of North Carolina – Chapel Hill, where she has taught entry-level and advanced-level students in neurorehabilitation since 1993. She has served in multiple roles within the Academy of Neurologic Physical Therapy, including the inaugural Director of Education, and has been honored with the Service to the Section Award and the APTA Lucy Blair Service Award. Karen has cared about individuals with traumatic brain injury since beginning as a PT in clinical practice, extending from moderate to severe brain injury to a recent focus on concussion. Her research has focused on developing outcome measures and interventions to improve active movement, balance, and functional mobility, with the aim to improve quality of life. She developed the Arm Motor Ability Test (for upper limb recovery following stroke) and the Walking and Remembering Test (for dual-task performance in older adults and individuals with acquired brain injury). She served as an ORISE Fellow with the Army Office of the Surgeon General, addressing TBI issues that affect individuals in military service. Her current research efforts are focused on wounded warriors with mild traumatic brain injury as part of a team that developed the Assessment of Military Multitask Performance, a test battery of challenging dual- and multi-task activities. She is currently leading a group writing a clinical practice guideline for physical therapy management of concussion, and is involved in intervention studies that address treatment for sports and military concussion. Funding support for her research has come from the Foundation for Physical Therapy, Centers for Disease Control and Prevention, National Institutes of Health, National Football League, and Department of Defense.
This chapter addresses what we should measure and why, from the standpoint of the ICF framework; purposes for which measures may be chosen; the types of measures that are used in practice; and the challenge of identifying the right measure for the patient and setting.
The types of data used in clinical measures will be reviewed with implications for what types of analyses may be appropriate with this data, and scaling properties that affect psychometrics.
Different types of reliability are reviewed that relate to stability of measurement between and within raters and internal consistency, with review of statistical tests that are used to illustrate these relationships. Minimal detectable change and minimal clinical important difference terms are defined and contrasted as they relate to interpretation of change with a repeated test.
Types of validity are reviewed. More critically, consideration of choosing the appropriate measure for the patient in that setting and at that level of function is emphasized. Common statistical analyses used to analyze validity are reviewed. Aspects of reliability and validity are illustrated through various studies conducted in the development of the High-Level Mobility Assessment Tool.
An overview of the aims of item response theory is provided, with an illustration of its use to refine a measure in development and an example of its use in computer assisted testing from the PROMIS Physical Function subscale. Consensus-based and computer-assisted testing resources that are available for use with neurologic patients are highlighted, including the Common Data Elements website, PROMIS, and NeuroQOL.
Consensus-based outcome measurement projects conducted by the Academy of Neurologic Physical Therapy EDGE groups and other multidisciplinary, evidence-based projects (StrokEngine, COMBI, ABI-EBR, SCIRE) are shared as resources for considering outcome measure selection.