A formative mixed methods evaluation of a new Ross program: why context matters

Abstract

Background

The Ross procedure is a technically complex operation to address the pathology of the aortic valve. Prior attempts at widespread implementation have been plagued with variations in effectiveness. We report our initial programmatic outcomes using dissemination and implementation (D&I) science frameworks, with an aim to define both programmatic efficacy and effectiveness. The study design was a single center, explanatory sequential mixed methods evaluation of a Ross surgery program from June 2020 to April 2023. Quantitative measures for baseline patient characteristics and postoperative outcomes were summarized. Qualitative measures were obtained using semi-structured interviews and characterized using implementation science frameworks.

Results

A total of 71 patients underwent the Ross procedure at a single academic center. Mean age was 40 years (± 13). There were no perioperative deaths or valve-related reinterventions (0%). During the follow-up period, 2 (2.8%) patients required re-operation and 2 (2.8%) required balloon dilation of the pulmonary homograft. Qualitative measures noted communication and prospective self-audit facilitated measures of effectiveness, as dichotomy within expected vs. observed outcomes led to program modifications. Collaboration across hospital systems promoted adoption and implementation, which led to the incorporation of the Ross procedure into accepted organizational practice.

Conclusions

In this formative evaluation, limited-efficacy outcomes demonstrated similar programmatic success to data from other established centers. With the prerequisite center volume and case selection, the Ross procedure is a safe and efficacious treatment for aortic valvular disease. Implementation science can be used both in real-time and retrospectively to guide organizational efforts to improve outcomes and to improve the translation of results across centers.

Background

Surgical aortic valve replacement (AVR) remains the cornerstone of therapy for non-elderly patients with significant aortic valvular disease [1]. Recent innovation has expanded the cardiac surgical armamentarium to address aortic valvular pathology in this population; options include mechanical valves, biologic/bioprosthetic valves, aortic valve repair, transcatheter aortic valve implantation (TAVI), and the Ross procedure [2]. While some data suggest improved survival amongst non-elderly adults who undergo aortic valve replacement (AVR) with mechanical prostheses compared to biologic prostheses, mechanical valves require life-long anticoagulation and thus expose young patients to a substantial cumulative risk of bleeding and thromboembolic events [2,3,4,5]. Contemporary data from Goldstone et al. additionally note that evidence regarding prosthesis selection had previously assumed equal mortality; however, their results conclude that most prior studies were likely underpowered to detect any clinically relevant differences within populations [5]. Conventional AVR has been shown to result in excess mortality in non-elderly patients compared to healthy controls, which includes patients receiving mechanical valves [6]. This ongoing uncertainty of the ideal valve substitute for non-elderly adult patients has manifested as discordance between the last two guidelines published by the American Heart Association/American College of Cardiology (AHA/ACC) and European Society of Cardiology/European Association for Cardio-Thoracic Surgery (ESC/EACTS), leading to further equipoise within the global community regarding the optimal aortic valve substitute [7,8,9,10,11].

Of the available aortic valve substitutes, the Ross procedure has been the only operation that alleviates the need for life-long anticoagulation, provides long-term viability, and allows for adaptive remodeling with a hemodynamic profile analogous to the native, non-disease aortic valve [12, 13]. Originally described by Ross et al. in the 1960s, a pulmonary autograft is used to replace the diseased native aortic valve [14]. Since inception, the Ross procedure has been slow to gain acceptance within the surgical community, as evidenced by varied and often conflicting results within the literature [3, 11, 12, 15]. This historical trend in Ross acceptance and outcomes illustrates the importance of considering both intervention-specific and system-level factors during procedural and programmatic implementation, as renewed interest in the Ross procedure likely contributed to the dissemination of key information to overcome barriers to success. These barriers can occur during any point of implementation and lead to gaps in patient care, often in the form of less effective outcomes [16].

To address such patient care gaps, the science of dissemination and implementation research (D&I) has emerged as a methodologic approach to characterize factors leading to success and/or failure [17]. While factors leading to improved Ross outcomes have largely been proposed or described on a center- or surgery-specific level, the barriers and facilitators during the implementation of a new Ross program remain largely uncharacterized [18, 19]. To address this gap, we sought to use implementation science to (1) perform a limited-efficacy evaluation of a new Ross program, (2) identify barriers and facilitators contributing to program outcomes, and (3) identify key implementation lessons to facilitate the establishment of new Ross programs or improve outcomes at existing Ross centers.

Methods

Study setting: program overview

Prior to the Ross program, no Ross procedures were performed at the University of Washington. Therefore, patients either received mechanical or bioprosthetic AVR based on the patient’s age, personal preference, and ability to tolerate anticoagulation. Our institution had an existing adult congenital heart disease (ACHD) program with a large population of patients with bicuspid aortic valve (BAV), which served both as a major impetus for a Ross program and also composed the majority of patients referred to our surgical team. Ross program inception involved collaboration between an adult aortic surgeon, a congenital heart surgeon, and adult congenital cardiologists in the ACHD clinic. From the program onset, the “Ross Team” model was conceptualized and embraced. Potential patients were identified and discussed in a multi-disciplinary fashion. This “Ross Team” has grown and evolved to now include two dedicated aortic surgeons, a congenital heart surgeon specializing in ACHD, a group of cardiologists with expertise in ACHD and valvular heart disease, director of the cardiothoracic intensive care unit, pharmacist with cardiovascular expertise, advanced practice providers (APPs) in intensive care and acute care/outpatient settings, and a dedicated nurse care-team coordinator.

Irrespective of the Ross program, there are approximately 50 AVRs performed annually at the University of Washington. Only 5–10 of those cases are performed in patients under the age of 60, which is due to either a patient not being a suitable Ross candidate (i.e., complex congenital history, poor ventricular function) or patient preference for conventional AVR. Additionally, any patient with isolated BAV and aortic regurgitation (AR) is offered first-line valve repair if possible.

Study design

A formative, single-center evaluation of a Ross surgery program was conducted using an explanatory sequential mixed methods approach with concurrent triangulation design as modeled by Creswell and Clark [20]. Quantitive data was collected retrospectively from chart review, while qualitative measures were obtained using semi-structured interviews of Ross program APPs. The University of Washington Institute Institutional Review Board approved this study (STUDY00014540) on 12/6/2021.

Study population and data definitions

Participants eligible for inclusion were all adult patients who underwent the Ross procedure from program inception in June 2020 to April 2023. Baseline patient demographics, procedural details, postoperative outcome data, and echocardiographic follow-up were collected and analyzed. Qualitative study participants were identified using purposeful sampling of key stakeholders during the Ross program implementation. Two APPs were identified for qualitative interviews, one with extensive experience in the acute care and outpatient setting, and another with extensive ICU experience. Both led implementation efforts of the Ross program. Semi-structured interviews of participants were conducted by videoconferencing after obtaining verbal consent. The interview guide was adapted from the Reach, Effectiveness, Adoption, Implementation, and Maintenance (RE-AIM) QuEST framework for program evaluation as described by Forman et al. and modified by the research team (Additional file 1) to identify barriers and facilitators to implementation of the Ross program, as well as identify key programmatic lessons [21]. All interviews were fully audio-recorded and transcribed verbatim.

Evaluation framework overview

The D&I frameworks used in this study were the RE-AIM QuEST framework and Consolidated Framework for Implementation Research (CFIR) [22, 23]. The RE-AIM framework was originally published by Glasgow et al., to improve the translation of evidence-based interventions into practice, and later modified into the RE-AIM QuEST framework by Forman et al. to include open-ended questions for qualitative interview [21, 22]. A major limitation of RE-AIM QuEST is that it does not adequately explain the conditions influencing a program’s success and/or failure. Given this, CFIR was used as a second framework to characterize the barriers and facilitators during implementation. The CFIR framework was originally published by Damschroder et al. to provide a pragmatic structure to guide formative evaluations in the real world, highlighting that utilization outcomes alone are inadequate, as researchers and practitioners must recognize that the success of an intervention is also dependent on its implementation and optimization within a given system [23, 24]. This framework combination has been previously defined and used for formative program evaluation [21, 24].

Data analysis

Quantitative data organization and transformations were performed using Stata/SE 17.0 for Mac (StataCorp, College Station, TX, USA). Analyses were performed using RStudio 4.2.2 statistical software (R Core Team (2022). R Foundation for Statistical Computing, Vienna, Austria. URL https://www.R-project.org/). We descriptively characterized patients in the overall sample. We report mean and standard deviations (SD) for continuous variables and numbers and percentages for categorical variables. Postoperative transthoracic echocardiogram (TTE) and computed tomography (CT) results were recorded at the time of patient discharge and during follow-up at 3 months, 1 year, and 2 years. TTE and CT data were compared using Fisher’s exact test and the Kruskal–Wallis rank sum test. Changes in aortic root diameter at discharge and during follow-up were visualized using LOESS regression. A p-value of less than 0.05 was used as a cut-off to define statistical significance.

For qualitative data, direct content analysis was performed using the RE-AIM QuEST framework and CFIR [21, 23]. Key quotes were characterized using RE-AIM QuEST domains and evaluated using CFIR to identify contextual factors and processes contributing to program success. Transcripts were analyzed by a research team of 2 (KB, FY) coders using Dedoose software (Version 9.0.17, cloud application for managing, analyzing, and presenting qualitative and mixed method research data (2021). Los Angeles, CA: SocioCultural Research Consultants, LLC www.dedoose.com).

Results

Reach

Seventy-one patients underwent the Ross procedure from June 2020 through April 2023. Baseline preoperative patient characteristics are summarized in Table 1. A majority of patients were male (n =  50, 70%), with an average age of 40 years (± 13), and white race (n =  57, 80%). Most patients presented with severe aortic stenosis (AS) (67%) and of those patients with a native valve, most had a BAV (93%). A primary AR indication for surgery was present in only 11 (15%) patients. Nearly a quarter (n =  17, 24%) had undergone prior sternotomy, and 18.3% (n =  13) had undergone prior AVR.

Table 1 Baseline patient characteristics prior to Ross procedure

The main contributors to the program reach were physician referral and patient-performed research. Providers reported that patients commonly performed research to either facilitate referral or improve knowledge of available treatments (Table 2). They noted that having a program website with broad information on heart valve surgery and the Ross procedure served to both facilitate program reach and address patient needs. Providers also noted that an initial barrier to patient referral was the COVID-19 pandemic; however, as the program became more established a theme of cosmopolitanism arose, where the degree to which the existing aortic surgery program was networked with other external organizations contributed to regional referral patterns and improved patient access.

Table 2 Notable quotations from providers during qualitative interviews

Effectiveness

Short- and long-term outcomes after the Ross procedure are summarized in Table 3. There were no in-hospital deaths; one death occurred during study follow-up (n =  1, 1.4%). The autopsy was declined and the cause of death was unclear. The average intensive care unit (ICU) length of stay was 3 days (± 2). During ICU admission, 12 patients developed atrial fibrillation requiring intervention (17%), 5 required inotropic support greater than 48 h (7.0%), 5 developed persistent brady-dysrhythmias requiring permanent pacemaker insertion (7.0%), 1 developed acute renal failure requiring continuous veno-venous hemofiltration/hemodialysis (1.4%), and 4 required postoperative transfusion (5.6%). Four patients (5.6%) underwent interventions during the study follow-up period. Two patients required re-operation at the autograft-aorta anastomosis, one due to early bleeding and another for pseudoaneurysm at 6 months post-Ross. Two patients were noted to have increased gradients through the pulmonary homograft at 1-year follow-up and underwent balloon dilation with a significant drop in their gradients and resolution of symptoms. There were no valve-related reinterventions (0%) and two patients had normal, uncomplicated pregnancies (2.8%). Postoperative TTE at discharge and during follow-up is summarized in Table 4. On the most recent follow-up TTE, 98.5% of patients had < 2 + aortic regurgitation (AR). Longitudinally, the aortic valve mean gradient (MG) was noted to decrease during follow-up, from a mean gradient of 5.76 mmHg (± 2.83) to 2.00 (NA) at 2 years postoperatively (p = 0.002), whereas the pulmonic valve MG was noted to increase from 4.4 mmHg (± 2.6) to 8.0 mmHg (± 5.5) at 2 years postoperatively (p < 0.001). There were no differences between average aortic root diameters at discharge and during follow-up (p = 0.4) despite a trend toward increase which plateaued after the 1-year follow-up (Fig. 1).

Table 3 Short- and long-term outcomes after Ross procedure

Table 4 Comparison of postoperative transthoracic echocardiogram data at discharge and follow-up

Fig. 1

Change in aortic root diameter from computed tomography at discharge and during outpatient follow-up

From a provider’s perspective, patient experience and assessment of program effectiveness were positive in both acute care and outpatient settings (Table 2). One provider relayed that patients perceived the Ross to be advantageous compared to alternative therapies to maintain important lifestyle considerations, as the population of patients served by the Ross program tended to strongly value an active lifestyle. Another theme that arose was reflecting and evaluating. Providers relayed that for the Ross procedure to be effective, strict blood pressure management was a challenging but integral component of postoperative care. They cited that having a strong and cohesive network of providers across multiple departments contributed to program effectiveness and that specifically having a group of providers who received education on program goals from core Ross faculty enabled providers to reflect and evaluate the progress and quality of implementation efforts (Table 2).

Adoption

To facilitate the adoption and success of the Ross program, attention to the implementation process and input from key stakeholders were imperative. Providers cited that not only was it critical to align all stakeholders to facilitate implementation, but the creation of a working group enabled open communication and the establishment of a network amongst key opinion leaders within the organization (Table 2). Opinion leaders, such as Ross surgeons, are able to influence the attitudes and beliefs of colleagues, thereby aligning the organizational “change valence” through discussion of provider concerns, identify key stakeholders who demonstrate self-efficacy in their ability to execute the goals of implementation, and then formally appoint implementation leaders who become champions to “drive through” implementation [25]. The creation of this robust internal network strongly contributed to programmatic adoption by motivating providers irrespective of discipline to value the Ross procedure and program.

Implementation

In addition to having a strong and established network of providers composing a “Ross team,” having clearly articulated program goals and access to knowledge and information emerged as a strong factor in implementation success. Providers relayed that investment by program leadership, particularly in education and clear programmatic and perioperative goals, empowered “implementation champions,” who became drivers through the implementation process and ensured the use of the previously established perioperative care pathway. This re-highlighted the theme of reflecting and evaluating, as implementation champions prospectively monitored progress and performed regular, informal debriefing sessions amongst other providers that ultimately led to key program modifications (Table 5).

Table 5 Key program modifications by “Ross Team” to overcome implementation barriers

One of the largest barriers to implementation success was postoperative blood pressure control and ensuring protocol adherence by both nursing and providers. A significant change from normal perioperative care was altering the frequency of vital sign checks from every 8 h to every 4 h, enabling providers to make necessary adjustments to medication regimens and ensure readiness for discharge. This placed an increased demand on acute care nursing and required significant provider involvement to ensure adherence to the protocol. One provider cited having a favorable implementation climate, where the culture with surgical attendings enabled honest conversation that created tension for change, and the use of the existing stakeholder communication helped overcome these barriers (Table 2). They relayed surgical attending availability and support were integral during the implementation process. An additional theme that arose and benefited implementation success was cosmopolitanism (the degree to which an organization is networked with other external organizations), which improved the translation of experiences from other Ross centers and facilitated complex decision-making. During a period of implementation where frequent changes and uncertainty are an inherent part of the process, this external network creates collective intelligence, where a group is capable of decision-making that extends beyond the knowledge of any one individual [23, 26].

Maintenance

Providers relayed that the Ross program has been fully integrated into routine care and that given the degree of specialization and large referral network, they did not experience any substantial challenges with referring providers once patients were in the outpatient phase of care. Two factors were reported to contribute to successfully navigating the transition to outpatient care, one being discharge notes with clear instructions and the other being patients who were highly educated on their medical care. Providers noted that patients desired clear discharge parameters and used outpatient resources readily to address questions or concerns, which substantially increased the demands on outpatient resources and may pose a barrier to ongoing maintenance as the program continues to increase in size (Table 2).

Discussion

In this formative evaluation of a new Ross program, limited-efficacy outcomes demonstrated similar programmatic success to data from other established centers, thus re-demonstrating the efficacy of the Ross procedure in non-elderly adult patients [1, 2, 4, 18, 19]. While contemporary data has mainly focused on efficacy-centric evaluations, insight into factors contributing to program effectiveness remains uncharacterized. To address this gap, we sought to use implementation science to identify barriers and facilitators contributing to program outcomes and identify key implementation lessons to improve the translation of results across new and existing centers.

Key implementation lessons learned while establishing a Ross program are summarized in Table 6. Understanding locoregional practice patterns is an essential consideration, as there not only is a well-established volume to outcome relationship, but the relative advantage of the Ross procedure compared to alternative therapies, namely mechanical or bioprosthetic substitutes, must be determined as this will dictate referral patterns. Our experience has been congruent with data by Bouhout et al., as the procedural and perioperative learning curve for the Ross procedure has been ~ 75 cases [6]. This emphasizes the need for planning and attention to the locoregional patient population, namely, what is the proportion of young, relatively healthy patients with bicuspid aortic valves who may be ideal candidates for the Ross procedure and maximally benefit from the establishment of a program. There must not only be the correct population to benefit, but also a sustained number of prerequisite cases to justify resource utilization.

Table 6 Key implementation lessons learned from establishing a Ross program

This compounds with lesson two, which is the composition of an effective team. The multi-disciplinary team that most effectively contributed to patient outcomes at our institution were the following: surgeon specializing in complex adult aortic (root) surgery, congenital/adult congenital heart disease surgeon, cardiologist with expertise in valvular heart disease, director of the cardiothoracic intensive care unit, pharmacist with cardiovascular expertise, APPs in ICU and acute care/outpatient settings, and a dedicated nurse care-team coordinator. The backbone of this team must be a surgeon with extensive aortic root experience and if able, another with adult-congenital experience. This expertise translates into an institutional opinion leader, who can not only create an implementation climate and culture favorable for programmatic formation but also engage other providers and identify necessary physical and human resources to help drive through implementation. This became evident during our implementation efforts, as attending surgeon identification and support of key APP “champions” facilitated identification of barriers to the implementation effort and ensured timely modifications to the perioperative care pathway/protocol. The biggest barriers to implementation were encountered during transitions of care. Both ICU to acute-care, and acute-care to outpatient transitions required increased demands on nursing staff, nurse-care coordinators, and APPs. Prospective consideration of staffing, establishment of perioperative care pathways, and resources for outpatient referring providers and patients facilitated program effectiveness.

The next lessons encompass the acquisition of experience, mentorship, and appreciation of the complexity of the Ross procedure. Establishing a supportive external network is imperative during program formation. The Ross procedure is inherently complex, and data has shown that the procedure itself has benefitted from modifications and adaptations over time [4, 11, 18, 27]. We additionally found that prospective self-audit was an important part of the implementation process. The establishment of an internal Ross-specific database in combination with a methodologic evaluation framework can provide a means of monitoring procedural efficacy within a given population as well as yield insights into program effectiveness. Implementation science provides a common taxonomy which can be used between organizations and experts to communicate factors and experience, thereby improving the generalizability and translatability within and between centers. Something of particular importance in a procedure such as the Ross, where identification of “ideal” vs. “good” candidates can have profound differences with respect to outcomes, and complex operative candidates require complex decision-making and resource utilization within programs. For example, early in our experience, we noted two separate issues with the autograft-aortic anastomosis, requiring reoperation. This led to the routine use of a Dacron sinotubular junction (STJ) ring at the distal end of the autograft on all Ross cases. The rationale behind the STJ ring is twofold: (1) providing support to the fragile autograft suture line and (2) prevention of STJ dilation in the future, a known mechanism of recurrent aortic regurgitation.

The final lesson has been regarding the dissemination of knowledge and experience. As illustrated by the study by Reece et al., data suggest that low Ross center volumes can have sub-optimal results [15]. We owe it to our patients to create programs that are not only efficacious and effective, but also sustainable. It is imperative that close data and outcomes tracking be performed at a multi-institution level. A significant gap exists within the aortic and surgical community, as no Ross-specific database presently exists. Our center experience has illuminated the benefit of multi-center collaboration, noting that strong external networks accelerated learning and contributed to our overall program success. A future consideration should be the establishment of a multi-institutional Ross-specific database or consortium, where programmatic and observational data can be used to improve our understanding of how patient and program-level factors interact and contribute to outcomes.

Strengths and limitations

Findings from this study should be interpreted considering several strengths and limitations. Strengths include a well-maintained, Ross-specific internal database to track patient-level outcomes, the use of methodologic implementation science frameworks to provide a taxonomy for communication of measures of effectiveness, and a study design which integrated program-specific determinants to improve the interpretation of reported outcomes. Limitations include data from a single center, overall program experience of 71 operative cases over a 3-year period, and a small number of performed qualitative interviews amongst the implementation team. Our study may have been underpowered to detect differences in outcomes and may suffer from a time-period bias, as our program may not have had adequate longitudinal follow-up to detect differences in outcomes. Lastly, while qualitative interviews were conducted with key members of the implementation team who agreed to participate in this study, the inclusion of other providers or patients may have provided additional insights into measures of effectiveness and improve the translatability of this study.

Conclusions

With the prerequisite center volume and case selection, the Ross procedure is a safe and efficacious treatment of aortic valvular disease within non-elderly adult patients. Implementation science can be used both in real-time and retrospectively, to guide organizational efforts to improve outcomes and to improve the translation of results across centers.