Tracking and Evaluation
The Center for Translational Medicine (CTM) relies on systematic evaluation of programs, resources and services, and translational activities to assess its progress toward improving efficiency and effectiveness along the translational science spectrum: moving research concepts from basic science to clinical studies and population health outcomes. The objective of CTM Tracking and Evaluation is to assess the overall impact and accomplishments of the Center, building a robust composite of performance indicators to continuously improve research translation.
Methods and Best Practices
Due to the variety and complexity of processes CTM components employ to achieve specific aims, the Tracking and Evaluation team utilizes mixed methods to monitor success: from data surveillance to surveys, logic models, metrics tables, specific aim reports (key performance indicators), a balanced score card, and program dashboards. In addition, we collect qualitative feedback from program faculty, staff, and investigators to identify areas needing immediate or drastic improvement.
The Tracking and Evaluation team is committed to involving each CTM program in its own development, promoting user-defined specific aims, logic models, and metrics to determine success. In addition, Tracking and Evaluation seeks to bridge internal gaps between components to facilitate dynamic improvement, analyzing performance data from different sources to inform cross-functional metrics. Tracking and Evaluation employs classic methodologies to assess CTM effects on investigators and trainees (customers), program components (contributors), and the larger system (clinical and translational science as a whole).
Tracking and Evaluation methods and tools used in the Center for Translational Medicine:
- Metrics Tables
- Logic Models
- Process Maps
- Specific Aims Reports (Key Performance Indicators)
- Balanced Score Card
- Evaluation Research
Meaningful measurements and data collection are important steps in program evaluation. Therefore, Tracking and Evaluation works with each CTM component to define measurable metrics toward specific outcomes; these outcomes align with pre-defined specific aims. Outcome indicators are displayed in a variety of ways, including examples shown in Methods and Best Practices. These indicators are then parsed to demonstrate overall CTM outcomes through a balanced score card and formal evaluation reports: important information that CTM leadership and the National CTSA Evaluation Consortium Committee use to make program changes.
Texas Regional CTSA Consortium (TRCC)
The Tracking and Evaluation team actively participates in the Texas Regional CTSA Consortium (TRCC), specifically on two projects:
- Defining a set of Standard Evaluation Metrics for CTSAs in the TRCC
- Defining a set of clinical trial metrics for multi-site clinical research conducted between TRCC locations
Outcomes from these projects will improve translational research infrastructure for all Texas CTSA institutions, providing benchmarks and strategic alignment between CTSAs and multi-site clinical trial infrastructure for translational investigators. Reports for each project will be released as they are made available.
National CTSA Consortium
Tracking and Evaluation is actively involved in National CTSA Consortium efforts, including Evaluation Key Function Committee projects and membership in multiple work groups. Results of these efforts can be found in the documents section of corresponding committees.
CTSA Evaluation Methodology Publications and Important Links
- Factor Analysis of the Clinical Research Appraisal Inventory (CRAI)
- Institute Of Medicine Report - The CTSA Program at NIH: Opportunities for Advancing Clinical and Translational Research
- Evaluation Guidelines for the Clinical and Translational Science Awards (CTSAs)
- Final Report on Field Visits to CTSAs
- Evaluating Translational Research: A Process Marker Model
- Common Metrics to Assess the Efficiency of Clinical Research
- Evaluation Metrics for Biostatistical and Epidemiological Collaborations
- Evaluating and Giving Feedback to Mentors: New Evidence-Based Approaches
- Assessing Research Self-Efficacy in the Physician-Scientists: The Clinical Research Appraisal Inventory
- A Shortened Version of the Clinical Research Appraisal Inventory: CRAI-12
- Google Wave: Have CTSA-Minded Institutions Caught It?
- Using Research Metrics to Improve Timelines: Proceedings from the 2nd Annual CTSA Clinical Research Management Workshop