Outcomes Research and the Need for a New Generation of Scholars
Outcomes research represents a field of scientific inquiry that examines the results of healthcare interventions and policy.1 It is not defined by any particular method, but rather, it leverages multiple methodologies to answer clinically relevant questions in the practice of medicine. Relevant skills and disciplines include epidemiology, health services research, economics and decision analysis, behavioral science, qualitative research, health informatics, advanced statistical methodologies and study design. What principally defines the field of outcomes research is its focus on the end results of healthcare and its disciplined efforts to define the patient characteristics, processes of care, and the structural components of care that are associated with those outcomes. Moreover, the relevant outcomes extend beyond mortality and disease progression to include patients’ health status (symptoms, function and quality of life) and the costs of care. An emerging aspect of outcomes research is its focus on designing new innovations in healthcare delivery to improve the quality of care and patients’ outcomes.
The need for a new generation of scholars to spearhead research into the ‘basic science’ of clinical practice has never been greater. Medicine is facing an unparalleled demand for leaders capable of quantifying patients’ outcomes, defining the determinants of these outcomes, comparing the effectiveness of alternative treatments, translating new knowledge into clinical care, and measuring and improving the quality of care. Training programs dedicated to outcomes and quality of care research help meet the growing national demand for scholars with these specialized research capabilities.
Outcomes Training Program
Within the field of cardiology, Saint Luke’s Mid America Heart Institute (SLMAHI), a primary teaching facility of the University of Missouri-Kansas City (UMKC) School of Medicine, has emerged as one of the preeminent outcomes research programs in the country. Today, SLMAHI is part of the integrated Saint Luke’s Health System with 10 hospitals in the Kansas City metropolitan area and surrounding region, but its legacy of innovation in cardiology began in the early 1970s when SLMAHI opened as the nation’s first heart hospital. Over 30 years ago, SLMAHI created the Cardiovascular Research Center to manage and analyze clinical data. Since then, the center has earned a world-wide reputation for excellence in outcomes and health economics research. John Spertus, MD, MPH, Missouri/Lauer Endowed Chair, Professor at UMKC and Clinical Director of Outcomes Research at SLMAHI, and David Cohen, MD, MSc, Endowed Chair in Cardiovascular Research, Professor at UMKC and Director of Cardiovascular Research at SLMAHI, have built a program that actively conducts extramurally funded research. The Outcomes Research Group includes researchers, statisticians, PhDs in Nursing and Psychology, and support staff with expertise in patient-centered outcomes research, observational registry research, quality assessment/improvement, and implementation science. The Health Economics and Technology Assessment Team provides decision analysis, technology assessment, cost effectiveness, and quality of life research support for numerous ongoing NIH- and industry-funded clinical trials. The unique expertise of the SLMAHI research program led to its selection as an analytic center for the American College of Cardiology’s (ACC) National Cardiovascular Data Registries, including its ICD registry and efforts at implementing the Appropriate Use Criteria. The center also receives grant and contract funding support from the National Institutes of Health (NIH), American Heart Association (AHA), Patient-Centered Outcomes Research Institute (PCORI), and industry. SLMAHI contributes >200 peer-reviewed publications/year. Overall, there are over 60 investigators and support staff that constitute the SLMAHI research program and provide a robust foundation for training.
History of the Training Program & Success of Trainees
Commensurate with its growing research operations, SLMAHI has increased its investment in research training. Since 2001, SLMAHI has had an active and growing outcomes research training program, with the goal to mentor promising trainees in the field. In 2008, the SLMAHI training program evolved substantially after the institution was designated as one of four AHA Outcomes Research Centers. This first ‘center grant’ ever offered for outcomes research in the U.S. included a two-year postdoctoral fellowship training program that sponsored five fellows to complete the SLMAHI outcomes research training program from 2009 to 2012. Building upon the strength of the established training infrastructure, SLMAHI and UMKC were awarded an NIH T32 training program grant that began in July 2012. The Cardiovascular Outcomes Research Fellowship Training Program is funded by the National Heart Lung and Blood Institute to UMKC and hosted by SLMAHI. Dr. John Spertus is the Program Director, and ~20 training faculty members with varying interests and areas of expertise are available to mentor trainees and provide a rich training environment. Other core faculty members include Suzanne Arnold, MD, MPH; Donna M. Buchanan, PhD; Paul Chan, MD, MSc; David Cohen, MD, MSc; Carole Decker, RN, PhD; Edward Ellerbeck, MD, MPH; Phil Jones, MS; Mikhail Kosiborod, MD; and Elizabeth Magnuson, ScD.
What really inspired Dr. Spertus and the training faculty to continue to build the program and seek funding was the extraordinary success of the outcomes research fellows. While a fellow might typically be expected to complete two to five research manuscripts during their fellowship, recent graduates of the SLMAHI program published >15 peer-reviewed manuscripts stemming from their fellowship work. Accomplishments of recent graduates and current fellows include unprecedented scientific productivity with numerous peer-reviewed journal publications (including JAMA, Circulation, JACC and Circulation: Cardiovascular Quality and Outcomes); numerous abstract and invited presentations at national conferences (including AHA Scientific Sessions, AHA Quality of Care and Outcomes Research [QCOR] Scientific Sessions, ACC Scientific Sessions); honors of special distinction at national conferences (e.g., winners and finalists in AHA QCOR Young Investigator Award competitions, ACC ‘Rising Star Fellow-In-Training Award’, ‘Best of AHA Specialty Conferences’, ACC ‘Best Posters’ and ‘Best Fellow-In-Training Posters’); academic institution awards for excellence in academics and research; and funding awards as principal and co-investigators on Patient Center and Outcomes Research Institute (PCORI) grants, AHA’s Grant-in-Aid, post-fellowship career development awards, and industry-sponsored projects.
Training Program Objectives & Formula for Success
The training program’s objective is to recruit and mentor outstanding potential researchers who are motivated to learn and apply outcomes research principles and techniques in performing comparative effectiveness research, translating existing knowledge to clinical practice, and to assess and improve quality. The two-year training program is specifically designed to help fellows demonstrate core competencies in: formulating relevant research questions (including the perspective – whether it is from patients, clinicians or policy makers), designing ideal studies to address the questions (including registries, clinical trials, or systematic reviews), executing the relevant studies (including protection of human subjects), design of proper outcomes assessments – and the strengths and weaknesses of alternative approaches, using the appropriate statistical analyses to answer the questions (emphasizing methods to address confounding, selection biases, missing data and the heterogeneity of treatment benefit), and disseminating the results of their research (publication and oral presentations for various audiences). Successful trainees are equipped to conduct independent academic research, work with the pharmaceutical/device industry, serve as leaders in quality within hospitals and health systems, and to pursue careers in public health and health policy. The need for these individuals has grown markedly in the past decade and they are sought after in every region of the country.
SLMAHI’s approach to training was published as an exemplar for training cardiovascular outcomes researchers.2 The program’s highly proven success in supporting the ability of fellows to make significant contributions to the scientific literature and to embark on successful research careers is based on an approach comprised of three synergistic components: 1) education, 2) mentorship, and 3) hands-on research.
To ensure a basic foundation of general skills for clinical research, fellows are enrolled in the UMKC’s Masters of Science Degree Program in Bioinformatics if they do not already have an advanced research-based degree. UMKC’s Bioinformatics Program emphasizes patient-oriented research in the context of state-of-the-art medical informatics and offers three areas of emphasis: clinical research, computational and genomics (http://www.med.umkc.edu/msb). Courses are also available at the University of Kansas, just 2 miles from SLMAHI, for specialized training (e.g., pharmaco-epidemiology and advanced statistical methods).
In conjunction with the organized coursework of the Masters Degree Program, fellows participate in a one-hour weekly seminar of specialized methods in outcomes research and academic ‘survival skills’. This course is a series of seminars presented by researchers who are actively engaged in health outcomes studies and explores multiple topics that are uniquely relevant to clinical investigation. Key concepts include health economics, cost-effectiveness of care, statistical methods for outcomes research, large dataset analyses, health decision aids, healthcare disparities, psychosocial factors related to health outcomes, prospective registries, translational research and advanced statistical methods. Specifically, the course is designed to help fellows understand statistical approaches for leveraging observational data to generate scientifically valid knowledge; select methods of outcomes assessments, including quality of life and economic outcomes; appraise current approaches to quality assessment, including how best to quantify the structures, processes and outcomes of care; plan for operational aspects of clinical research, including team dynamics, budgets and medical writing; describe the strengths and weaknesses of alternative data sources, including administrative and prospectively collected observational data; and utilize qualitative research techniques to generate novel research hypotheses.
Ongoing activities of SLMAHI’s research team provide additional training opportunities to supplement the fellows’ education and include weekly work-in-progress meetings of the Outcomes Research and the Health Economics Teams, Cardiovascular Grand Rounds, and Outcomes Research Fellows’ Journal Club meetings. Fellows also have opportunities to participate in research and training activities through the Heartland Institute for Clinical and Translational Research (NIH Clinical and Translational Science Award [CTSA] funded to the University of Kansas), for which the SLMAHI program leads the personalized medicine and outcomes research core. Edward Ellerbeck, MD, MPH, Associate Program Director of this Outcomes Research Fellowship and Education Director for the CTSA, facilitates fellow involvement in CTSA training activities. Attending and presenting original research at national conferences also provides fellows with ideal opportunities to network with other scholars in the field and to learn the state-of-the-art in outcomes research. Fellows attend the annual AHA QCOR Scientific Sessions and are encouraged to submit their work for QCOR’s Young Investigator Award competition. When presenting original research, attendance of fellows is supported for the AHA and ACC Scientific Sessions and other conferences, depending on the individual fellows’ needs and interests.
A particular strength of the SLMAHI Cardiovascular Research Program is its intimate and highly collaborative culture. The program has a number of excellent mentors in diverse disciplines. Investigators work closely together and interact on a daily basis, providing a mutually supportive environment for fellows and staff alike. Because the program excels in multidisciplinary research and collaboration, it is common for fellows to have multidisciplinary mentorship teams. There are biweekly group mentoring sessions in which the faculty and fellows meet together to review each of the fellows’ ongoing activities. By the end of an initial two-month orientation period, fellows are also paired with a primary mentor and fellows submit a career development plan that is regularly reviewed and updated. Each fellow meets with his/her primary mentor no less than every two weeks to review their educational curriculum, to evaluate additional training opportunities, and to support their research. Graduates from the training program maintain close research collaborations with fellow trainees and mentors.
To stimulate the next generation of researchers, there is no better enticement than the challenge and thrill of participating in research projects from conception to publication and, ultimately, translation into clinical care. Hence, the primary objective of the fellowship program is to provide a rich research experience. SLMAHI offers an abundance of ongoing research activity dedicated to quality of care, health outcomes, clinical research and trials, decision and economic analyses. Fellows not only have the opportunity to engage in any of these ongoing projects, but have ready access to highly accomplished investigators and analysts who can help fellows with their own projects.
Each fellow works collaboratively with his/her mentor to identify and develop a mentored research project within their first two months of the program. It is common for fellows to complete at least two to three mentored projects, which often entail secondary data analysis, in their first year of our program. Fellows are encouraged to work with several faculty members on different types of projects during their fellowship so that they can better define their primary research interest. After completing the first year of fellowship, during which a much more solid foundation in study design and analysis will have been acquired, fellows are expected to define an independent project(s). Often this will capitalize on their emerging interests, and may involve prospective data collection, clinical implementation/quality improvement activities, or other intellectually creative projects. For those enrolled in the Masters Degree Program, this project serves as their thesis.
In outcomes research, the ‘laboratory’ refers to the data resources from which important clinical questions can be answered and the clinical environment in which novel interventions can be implemented and pilot tested. Hallmarks of the SLMAHI/UMKC training program include the critical infrastructure of access to numerous existing data sources and highly experienced statistical support. The extraordinary array of unique databases include both multicenter outcome registries (e.g., PREMIER, TRIUMPH, PORTRAIT, PRISM), clinical trial databases (e.g., EPHESUS, PARTNER, TAVR, FREEDOM, SYNTAX), and national quality databases (e.g., ACC’s National Cardiovascular Data Registries, AHA’s National Registry of Cardiopulmonary Resuscitation). The abundant data resources are continually analyzed by the Biostatistics Core within SLMAHI’s Cardiovascular Research Center, which currently includes 12 statisticians who have extensive experience in the analysis of clinical and health services data. Statisticians are highly versed in issues pertaining to observational data, including selection bias, missing data, propensity and instrumental variable models, latent class and variable analyses, Bayesian analyses, multivariable and hierarchical modeling, multiple imputation, etc. SLMAHI staff are also highly experienced in costing health services, decision and cost-effectiveness analysis, simulation models, and estimation of life expectancy. The statisticians stay abreast of current methodological research and have a strong track record of applying new techniques to ‘real world’ problems. The statisticians have and use state-of-the-art statistical packages (e.g., SAS, SPSS, WinBUGS, R, S-Plus, M-Plus, PASS, STATA, TreeAge). Finally, the statisticians have a strong commitment to working with trainees to enable them to refine their data management and analytic skills.
Eligibility Criteria and Application Process
The SLMAHI/UMKC Cardiovascular Outcomes Research Program is currently seeking applications for the two-year postdoctoral fellowship program. Applications are due November 30, 2013 for fellowship positions beginning July 1, 2014. In addition to providing a multifaceted curriculum, rich mentorship, and hands-on research experience, the full-time training program offers salaries based on NIH postdoctoral stipend scales and benefits that include health, dental, disability, and life insurance; vacation time; tuition for Masters Degree coursework; and funds for travel expenses to professional conferences.
Eligible candidates must possess an MD, PhD, DO, or equivalent doctoral degree. In addition, physician candidates must have also finished their residency training, if applicable. This NIH T32 funded training program requires that candidates be U.S. citizens or permanent residents. In addition to cardiologists or cardiology-bound physicians, applicants are welcome from other health service professions, including internal medicine, public health, neurology, nursing, pharmacy, psychology, and other related health fields. Minority candidates are encouraged to apply, given SLMAHI’s strong research activity in studying disparities and developing strategies to overcome them.
The clinical training programs at SLMAHI/UMKC also provide valuable opportunities to integrate outcomes research training with fellows’ clinical training. Currently, SLMAHI/UMKC has ACGME-accredited programs in general cardiology, interventional cardiology, electrophysiology, and advanced heart failure/transplantation, and non-accredited programs in imaging and preventive cardiology. This is an ideal opportunity for trainees interested in both clinical and outcomes research training.
For more information on the SLMAHI/
UMKC Cardiovascular Outcomes Research Training Program, visit http://www.saintlukeshealthsystem.org/OutcomesResearchFellowship and contact Donna Buchanan, PhD, Assistant Program Director, at MAHIResearch@saint-lukes.org.
Disclosures: The authors report that the SLMAHI/UMKC Cardiovascular Outcomes Research Training Program is funded by an NIH T32 Training Grant (NHLBI T32 HL110837). Outside the submitted work, Dr. Spertus reports honoraria from UnitedHealthcare and the AHA, and other grants/grants pending to his institution.
- Krumholz HM. Outcomes research: generating evidence for best practice and policies. Circulation. 2008;118:309-318.
- Kosiborod M, Spertus JA. Careers in cardiovascular outcomes research. Circulation. 2009;120:76-81.