The Aligning Forces for Quality Initiative: Background and Evolution From 2005 to 2015

Objective: The Robert Wood Johnson Foundation’s (RWJF’s) Aligning Forces for Quality (AF4Q) program was the largest privately funded, community-based quality improvement initiative to date, providing funds and technical assistance (TA) to 16 multi-stakeholder alliances located throughout the United States. This article describes the AF4Q initiative’s underlying theory of change, its evolution over time, and the key activities undertaken by alliances.

Study Design: Descriptive overview of a multi-site, community-based quality improvement initiative.

Methods: We summarized information from program documents, program meetings, observation of alliance activities, and interviews with RWJF staff, TA providers, and AF4Q alliance stakeholders.

Results: The AF4Q program was a dynamic initiative, expanding and evolving over time. The underlying theory of change was based on the notion that an aligned, multi-stakeholder approach is superior to independent siloed efforts by stakeholders. Participating alliances developed or strengthened programming to varying degrees in 5 main programmatic areas: (1) measurement and public reporting of healthcare quality, patient experience, cost, and efficiency for ambulatory physician practices and hospitals; (2) efforts to engage consumers in health, healthcare, and alliance governance (consumer

engagement); (3) adoption and spread of effective strategies to improve care delivery; (4) advancing healthcare equity; and (5) integration of alliance activities with payment reform initiatives.

Conclusion: The AF4Q initiative was an ambitious program affecting multiple leverage points in the healthcare system. AF4Q alliances were provided a similar set of expectations, and given financial support and access to substantial TA. There was considerable variation in how alliances addressed the AF4Q programmatic areas, given differences in their composition, market structure, and history.

Am J Manag Care. 2016;22:S346-S359

Proponents of using multi-stakeholder alliances and regional coalitions to coordinate efforts to improve the quality of healthcare hypothesize that the coordinated efforts of health plans, purchasers, providers, and consumers will yield better and more sustainable outcomes than activities undertaken independently.^1-3 The Robert Wood Johnson Foundation’s (RWJF’s) Aligning Forces for Quality (AF4Q) initiative was based on the premise that “no single person, group, or profession can improve health and healthcare throughout a community without the support of others. AF4Q asks these critical stakeholders to work toward common, fundamental objectives that…will lead to better care.”⁴ AF4Q was the largest privately funded, community-based quality improvement (QI) initiative to date, with an investment of more than $300 million over the life of the project.⁵

In 2006, RWJF began providing grants and technical assistance (TA) to 4 alliances (multi-stakeholder partnerships in each AF4Q community), launching a program that expanded over time, and ultimately included 16 communities across the country. Funding for the alliances ended in the middle of 2015. The AF4Q alliances were either preexisting not-for-profits or established specifically for the AF4Q grant, with payer, provider, and consumer representatives. While RWJF played an active role in program strategy development and oversight (ie, site selection, development of requests for proposals [RFPs], and funding decisions), it delegated the day-to-day program implementation to a national program office (NPO).

The authors of this article (an update of a previously published article describing the background and evolution of the program through 2012)⁶ are a team of investigators from Penn State University, the University of Michigan, the University of Minnesota, Northwestern University, and George Washington University, contracted by RWJF to conduct an independent program evaluation. This article describes the program’s evolution and the range of AF4Q-related activities undertaken by participating alliances. Information was obtained through program document review, meeting participation (eg, AF4Q national meetings, NPO-hosted teleconferences with project directors, and alliance-hosted meetings), observation of alliance activities, and interviews with RWJF staff, TA providers, and alliance stakeholders. Readers interested in additional description about our research design and data sources may refer to an article previously published by Scanlon et al that describes the design of the formative phase of the evaluation⁷ and an article by Scanlon et al in this supplement which focuses on the design of the summative phase.⁸

We begin by describing the theory of change underlying the initiative, which we summarized graphically in a logic model. Subsequently, we discuss the evolution of the program from its inception through conclusion, covering programmatic expectations for the alliances, and the guidance and TA provided.

Theory of Change: the AF4Q Logic Model

A key step in program evaluation is to articulate the initiative’s theory of change—that is, the underlying assumptions and expectations regarding how program interventions will lead to the expected outcomes, and in what timeframe. This theory of change is often graphically depicted through a logic model, which “helps to focus an evaluation by making a program’s assumptions and expectations explicit, and increases stakeholders’ understanding about the program or initiative.”⁹ The Figure depicts the logic model developed by our evaluation team early in the program’s implementation with input from RWJF staff, the NPO, TA providers, and key alliance stakeholders.

As depicted on the right-hand side of the model, the objective of the AF4Q initiative was improvement in key community and population health outcomes, such as health status and quality of care received. The outcomes were broad and ambitious, and were envisioned to take time to realize. More proximate outcomes, such as increased transparency about provider quality and cost, and improved care coordination, are depicted as intermediate outcomes. The left-hand side of the Figure illustrates how the program was envisioned to achieve these outcomes. Specifically, within the community, the AF4Q initiative started with a multi-stakeholder community alliance. The alliance was responsible for establishing a leadership team and organizational structure to support program activities. Leadership was responsible for formulating the alliance’s vision and strategy within their community. To achieve this vision, the alliance developed and implemented interventions, which were activities targeted at facilitating changes in the programmatic areas germane to the AF4Q initiative. Alliances may have sponsored these activities directly, or in collaboration with other community organizations.

At a minimum, the alliances had to develop or advance efforts in 5 main programmatic areas: (1) measurement and public reporting of healthcare quality, patient experience, cost, and efficiency for ambulatory physician practices and hospitals; (2) efforts to engage consumers in their health, healthcare, and alliance governance (consumer engagement); (3) the adoption and spread of strategies to improve care delivery; (4) advancing healthcare equity; and (5) integration of alliance activities with payment reform initiatives. In addition to aligning stakeholders around a common vision, the AF4Q initiative targeted alignment of programmatic areas, depicted by connectors in the interventions box.

Across the top of the Figure, we indicated that RWJF provided TA through multiple organizations or individuals with expertise in key programmatic areas to assist the alliances in strategy development and implementation. The model also reflects that the alliances varied significantly in terms of history and market structure and were influenced by factors in the external environment not directly related to the AF4Q initiative. Because RWJF’s objective was to sustain the alliances’ activities beyond the conclusion of the grants in 2015, an important long-term program goal was to build collaborative capacity within the community. This could be accomplished through continuation of the alliance or through alternative models, such as continuation of programmatic activities by others in the community. Finally, as noted on the bottom of the Figure, the alliances’ activities and the impact of the AF4Q initiative were expected to evolve over a period of time, with necessary adjustments based on feedback from experiences in program implementation.

While the logic model provides a succinct view of the overall program, it was not sufficiently detailed to guide our evaluation. Accordingly, our team also developed individual models for 4 of the programmatic areas. These programmatic logic models are available in a previously published online eAppendix.¹⁰

Evolution of the AF4Q Initiative

Under the AF4Q initiative, RWJF provided funding and TA to participants; in turn, the alliances were expected to meet specified goals and objectives. While the program’s initial scope was substantial, it expanded through enhancements to existing programmatic areas and the addition of new ones. RWJF made a significant commitment to the provision of TA, investing more than $25 million in TA over the course of the program (see the Table for a list of select TA providers contracted by the NPO).¹¹ In this section, we describe the evolution of the initiative, including the scope, goals, expectations, and TA offerings.

Phase I (July 2006-April 2008)

The overarching goal of the initiative’s first phase was to help communities substantially improve the quality of healthcare provided in ambulatory care settings for persons with chronic diseases. Phase I targeted 3 programmatic areas believed to be key drivers of quality (depicted in the interventions box of the logic model): (1) performance measurement and public reporting of performance data; (2) QI in primary care physician practices; and (3) consumer engagement. RWJF chose to invite 4 communities with a history of stakeholder collaboration on healthcare quality—Detroit, Michigan; Memphis, Tennessee; Minnesota; and Puget Sound, Washington (which expanded to the state of Washington)—to serve as the initial communities for the program. Funding began in July 2006, and the Center for Health Improvement (Sacramento, California) was selected as the NPO. An additional 10 communities (Cincinnati, Ohio; Cleveland, Ohio; Humboldt County, California; Kansas City, Missouri/Kansas; Maine; south central Pennsylvania; West Michigan; Western New York; Willamette Valley, Oregon [which expanded to the state of Oregon]; and Wisconsin) were added in February 2007 through a competitive grant process. Short overviews of each alliance, which include information on their founding organizational structure, date of entry into the AF4Q program, descriptions of their focus as of spring 2016, and geographic scope, can be found in eAppendix A.

Alliances were to publicly report ambulatory care performance information for the community’s primary care providers within 3 years, using local multi-payer data and nationally endorsed quality measures. They were given a broad goal for consumer engagement, which was to get consumers to “create an overall demand for high-quality care, in their choice of provider and health plan, in their choice of treatment options, as advocates for change, and in managing their own health conditions.” Rather than offering a specific model for consumer engagement, RWJF chose to encourage the alliances to innovate and “let a thousand flowers bloom.” In addition, the alliances were required to have “substantial and credible consumer representation” on their leadership teams.¹² While RWJF identified QI as a core component of the initiative, no formal requirements were established in this phase. However, alliances were encouraged to engage in activities associated with patient-centered medical homes (PCMHs) and other ambulatory QI programs.

Early TA was built around webinars on topics related to the core programmatic areas, but largely focused on performance measurement and public reporting. All 14 AF4Q alliances became part of RWJF’s Consumer Engagement Learning Collaborative (CELC), the primary vehicle through which consumer engagement TA was provided. The CELC provided a structured framework, using a combination of TA consultants and meetings of the alliances, to assist in the development of consumer engagement strategies in 2 areas—self-management and consumer use of public reports of provider quality.¹³ RWJF and the NPO also began sponsoring semiannual national meetings of the AF4Q alliances, providing educational sessions and workshops on programmatic areas, and offering opportunities for shared learning among the alliances. Specific progress measures used by RWJF and the NPO were limited during this phase; instead, each alliance developed its own work plan based on general guidance from RWJF and the Center for Health Improvement, which included a list of goals, a timeline, specific strategies, and status update for each goal identified.

Phase II (May 2008-April 2011)

In May 2008, all 14 alliances were funded for the second phase of the program. During this phase, RWJF more clearly defined its expectations for the alliances in the programmatic areas established in phase I. It also added new programmatic areas: (1) assisting hospitals and other inpatient healthcare facilities with improving quality; (2) focusing and strengthening the role of nurse leaders and frontline nurses in QI initiatives; and (3) using performance measures to capture patient experience of care and reduce disparities in healthcare quality by race, ethnicity, or primary language (REL) spoken. During phase II, RWJF chose George Washington University’s Center for Health Care Quality as the new NPO. RWJF and the new NPO instituted a number of administrative and programmatic changes in this phase, including the introduction of new alliance reporting processes and programwide systems for measuring and reporting alliance progress in each programmatic area, and reorganization and expansion of TA to move from a one-size-fits-all approach toward one that was more tailored to the specific needs of individual alliances. Three additional communities—Albuquerque, New Mexico (which expanded to the state of New Mexico); Central Indiana; and Boston, Massachusetts—were added during this phase to bring the total number of communities to 17.

During phase II, RWJF focused the alliances’ consumer engagement work on facilitating consumer awareness and use of quality performance reports of health providers and involving consumers in alliance governance.¹⁴ Self-management education was no longer emphasized, because the foundation viewed it as less aligned with the rest of the AF4Q program as it “matured,” and because alliances were challenged by the breadth of consumer engagement work.^13-15

Performance measurement and public reporting requirements were expanded to include reporting of nationally recognized measures of quality, patient experience, efficiency, and hospital pricing.¹⁴ Alliances also had to demonstrate a plan for achieving the standardized collection of self-reported data about patients’ REL from all healthcare providers and integrate this information into their measurement, reporting, and QI activities.

To address the new inpatient QI expectations, the NPO and TA providers offered 3 QI learning collaboratives, which were open to hospitals in AF4Q communities. The collaboratives included (1) Transforming Care at the Bedside, led by the American Organization of Nurse Executives, which was a pre-existing initiative developed by RWJF in collaboration with the Institute for Health Improvement (IHI), to target the development of patient-centered care in nursing units;¹⁶ (2) the Equity Quality Improvement Collaborative, which focused on improving the quality of care delivered to cardiac patients, while reducing racial and ethnic disparities; and (3) the Language Quality Improvement Collaborative, which aimed to improve care to non-English-speaking patients. The collaboratives employed a combination of in-person meetings, webinars, and monthly conference calls. Alliances were expected to assist in recruiting hospitals within their communities for participation in the collaboratives. In 2010, the NPO modified the collaboratives to be less dependent on travel, and to include more peer-to-peer learning and cover additional topics to facilitate the alliances’ ongoing QI efforts. Three virtual collaboratives focused on the following topics: reducing readmissions, increasing patient throughput, and improving language services. Over 100 hospitals participated in the NPO-led online hospital quality networks.

Alliances were required to inventory regional QI needs and resources for ambulatory care and incorporate this information into plans for sustainable ambulatory QI infrastructures in their communities.¹⁷ The NPO also established 2 ambulatory peer-to-peer quality networks in 2010; 1 was focused on the PCMH, and the other concentrated on helping alliances implement a regional learning collaborative. Both the inpatient and ambulatory peer-to-peer networks afforded alliances opportunities to share and learn from each other’s experiences and provided access to online resources and toolkits, consultation with QI leaders, and data-driven feedback.

The NPO introduced new systems for measuring and reporting alliance progress in meeting grant expectations for each programmatic area. Starting with 14 quality/equality indicators in 2009, the list expanded to 33 indicators in 2010. In light of the programmatic changes and additions, TA offerings were greatly expanded, and the way TA was implemented changed, as well. Rather than employing the one-size-fits-all TA approach used in phase I, alliances were given latitude to select from a wide-ranging list of AF4Q-sponsored TA providers covering the main programmatic areas, alliance governance issues, and communication strategies. TA was delivered through a combination of webinars, telephone conference calls, learning collaboratives, workshops, special reports, and direct consulting with TA vendors. Topics addressed in AF4Q programmatic areas included: making the alliances’ websites more consumer friendly; physician benchmarking and attribution; capturing and reporting patient experience data; communicating with physicians about performance measurement; standardized collection of self-reported data on patient REL; and the use of consumer-decision aides in healthcare. Governance topics focused primarily on alliance leadership development and formulating strategies to sustain alliance activities beyond the conclusion of the grant.

Phase III (May 2011-April 2013)

In 2010, all 17 AF4Q alliances applied for funding under phase III of the project; 16 alliances were selected to continue as part of this phase, which began in May 2011 and ended in April 2013. The Central Indiana alliance did not receive continued funding because, according to RWJF, the alliance’s stakeholders were unable to agree on a strategy for publicly reporting their performance data, a mandatory AF4Q program requirement. Under phase III, new program expectations for the alliances included (1) setting and achieving explicit, measurable goals around specific clinical conditions (eg, diabetes and acute myocardial infarction) and sites of care (eg, physician practices and emergency departments); (2) focusing efforts on the selected conditions and sites of care to improve quality, cost, and value; (3) experimenting with payment reform; and (4) leveraging federal, state, and local health information technology (HIT) efforts. The new goals reflected RWJF’s desire to expand the initiative’s focus beyond quality to include cost and efficiency.¹⁸ The individualized goals set by alliances, known as quality and cost goals, were also used by RWJF and the NPO to monitor alliances’ success in the program. Examples of 1 alliance’s individualized quality and cost goals included: (1) providers with specialties related to diabetes care achieve national benchmarks across 3 of the 4 diabetes performance metrics, as measured by the regional average; and (2) avoidable emergency department visits reduced region wide, as measured by the regional average.

Program expectations under this phase recognized the potential for the alliances to capitalize on opportunities stemming from recent legislation, including the Health Information Technology for Economic and Clinical Health (HITECH) Act and the Patient Protection and Affordable Care Act (ACA). Several alliances applied for and received funding through the Center for Medicare & Medicaid Innovation (CMS Innovation Center), established through the ACA. For example, the Cincinnati alliance was chosen to participate in the CMS Innovation Center’s Comprehensive Primary Care (CPC) initiative.¹⁹ A key objective for this initiative is to establish a system in which Medicare, commercial, and state health insurance plans pay bonuses to primary care doctors who better coordinate patient care.

RWJF also made additional funding opportunities available specifically to AF4Q alliances during this phase of the program. In 2011, RWJF granted supplemental funding to 5 AF4Q alliances (Memphis, Western New York, Oregon, Minnesota, and West Michigan) to connect AF4Q efforts with public health stakeholders to implement policies and systems change efforts to improve social determinants of health. In 2012, RWJF granted funding to 6 AF4Q alliances (Boston, Cincinnati, Cleveland, Humboldt County, Maine, and West Michigan) to work on a 2-year project with the Camden Coalition for Healthcare Providers to reduce emergency department visits and hospitalizations for super-utilizers of healthcare. Also, in 2011 and 2012, RWJF partnered with Consumers Union to provide support to 3 AF4Q alliances (Boston, Minnesota, and Wisconsin) in their release of special inserts of provider performance measures in Consumer Reports.^20-22

The NPO continued to offer a menu of TA opportunities that could be tailored to alliances’ needs. In addition, alliances were afforded the opportunity to create direct peer exchanges with fellow grantees. In 2010 and 2011, 12 alliances received funding for the peer-to-peer exchange program to visit and learn from other AF4Q alliances. For example, leaders from the Cleveland alliance visited Maine to learn about the Maine alliance’s success in integrating consumers into their activities.

During this phase, RWJF convened an expert panel to review the program and provide guidance on how to refine the program for the final phase. The 2012 review resulted in 3 primary recommendations for RWJF: (1) help alliances demonstrate their work’s impact on healthcare costs to make a business case for local stakeholders to sustain alliance efforts beyond 2015; (2) allow more flexibility in program requirements to help alliances emphasize parts of the program that are most strategic for them; and (3) share early results and lessons learned from the alliances with policy makers and healthcare leaders. Subsequently, RWJF and the NPO developed a framework to address these recommendations and implemented changes during phase IV of the program, including allowing alliances to emphasize parts of the program that were most strategic for them, and emphasizing a storytelling approach through a series of program articles. These components are described in more detail below in the description of phase IV.²³

Formal alliance sustainability planning also began during phase III. In April 2012, approximately a year before phase IV funding was to start, the NPO released a memo signaling that alliances would need to go through a sustainability exercise as part of the 4.0 proposal development period. In the memo, alliances were directed that “this strategic plan should describe the traditional elements...strengths, weaknesses, opportunities and threats...of current alliance activities, how these elements will be addressed during the 4.0 implementation phase, and their implications for successful pursuit of alliance cost and quality priorities after funding is complete in 2015.”²⁴

Phase IV (May 2013-April 2015)

In May 2012, RWJF issued an RFP to the participating alliances for the final phase of the program. All 16 alliances applied for funding in October 2012 and all 16 received funding to continue with the final phase, which ran from May 2013 to April 2015.

In contrast to previous phases that expanded expectations around the programmatic activities, phase IV emphasized the development of long-term strategies for sustaining the alliances and/or their activities after the AF4Q program. Alliances were charged with continuing the sustainability planning they began during phase III by developing a “strategic plan for sustainable high-value care” for their respective communities.

Per the phase IV RFP, the alliances were instructed to “recalibrate, expand, and spread the scope of their third phase [quality and cost] goals.”²⁵ Alliances were given autonomy to select programmatic areas (eg, performance measurement and reporting, consumer engagement, and payment reform) to support their quality and cost goals, rather than implement all of the prescribed program interventions as was expected in the prior phases. However, the final phase required a continued focus on QI and disparities reduction, with RWJF broadening its definition of disparities to allow alliances to include socioeconomic, geographic, or other disparities that were relevant to their particular contexts. While RWJF continued to monitor aggregate program progress using the quality/equality indicators developed in previous phases, these indicators were no longer used to monitor individual community progress; rather, alliances’ individualized quality and cost goals became the standard for determining their success.

During this final phase, with an emphasis on storytelling, RWJF and the NPO asked alliances to identify and describe their successes which would be highlighted in a collection of program articles called “Bright Spots.” For example, a “Bright Spot” about Memphis featured that alliance’s effort to gather REL data to identify gaps in healthcare among different populations.²⁶ Additionally, in 2013 and 2014, RWJF contracted with project managers, data organizations (eg, Freedman HealthCare and Shaller Consulting Group), and a communications firm on the Gathering Evidence project to identify quantifiable evidence of how the AF4Q program affected health and healthcare in the communities.²⁷ Alliances provided data to support and inform this work.

There was a decline in the number of TA providers that worked with alliances during the final phase, with the exception of TA provided around sustainability. In 2013, the NPO contracted with a consulting firm specializing in leadership development, strategy implementation, and community collaboration, to provide sustainability TA for all alliances. The NPO also held 2 sustainability workshops for alliances in 2013 and 2014. Peer-to-peer TA between alliances continued. For example, a team of consumers and consumer engagement staff from Kansas City traveled to south central Pennsylvania to attend Patient Partner Program training to learn new strategies to engage patients and providers, and to understand the details of a patient partner program. Alliances also were offered up to $200,000 through mini-grants which allowed alliances access to a TA provider or expertise that augmented the TA provided through the NPO; all 16 alliances applied for and received the mini-grants. For example, 1 alliance hired a data aggregation organization to enhance its provider portal with the addition of new validation capabilities for providers. Another alliance engaged consultants to facilitate organization development sessions with its staff.

In 2013, RWJF hosted a meeting for the alliances on population health. It focused on making the connection between healthcare and population health work for patients. Also during this phase, alliances had the opportunity to attend several payment reform summits, hosted by AcademyHealth and RWJF, to learn about each other’s payment reform efforts and those of other grantees working on payment reform through RWJF.

In late 2013, RWJF asked the alliance leaders to provide feedback related to its new focus on building a “Culture of Health,” which is described as a vision that “enables all Americans to live longer, healthier lives” through “comprehensive communitywide integration” of healthcare, public health, and other sectors (see the RWJF perspective article in this supplement).⁵ Additionally, as RWJF turned its focus to the Culture of Health vision, it began to frame the AF4Q program to fit its future agenda.

Program Wrap-up

Throughout the program, the Foundation made it clear that it planned to end its formal involvement with the alliances in April 2015. However, RWJF had the expectation that alliances would sustain their activities and community collaboration following the AF4Q program, and serve as models to help propel national reform. In support of this, much of the focus during the last 2 phases of the program was around sustainability planning. RWJF and the NPO organized program wrap-up activities, including the final AF4Q national meeting in May 2014, which focused on striving for sustainability in a changing healthcare market, and an ALIGN Summit in November 2014, which was designed to share lessons learned from the AF4Q program and discuss how they could inform future work.

A number of alliances held community events near the program closure date (April 2015) and incorporated AF4Q program content as a feature of those events. For example, the alliance in Minnesota held a final leadership team meeting focused on AF4Q program reflections and potential future collaboration in the state. During phase IV, the NPO also began to close out its activities. In May 2015, it archived the AF4Q website, creating a static resource to serve as a repository for the program, and ceased operations at the end of September 2015. After the AF4Q program, RWJF supported the Network for Regional Healthcare Improvement to lead an online learning community—the Collaborative Health Network (CHN).²⁸ The CHN’s purpose is to support peer-to-peer learnings about the multi-stakeholder approach to improving healthcare. It will archive important AF4Q program documents and facilitate ongoing learning for all regional health improvement collaboratives, including the AF4Q alliances.

AF4Q Programmatic Areas

In the following sections, we describe the range of activities undertaken by the alliances in each of the main programmatic areas. More detailed descriptions of program activities, and their impacts and lessons learned, can be found in other papers in this supplement.^15,29-31 In terms of the logic model, the activities described would appear in the interventions box.

Performance Measurement and Public Reporting Activities

Transparency through public reporting of healthcare providers’ performance was a central theme of the AF4Q initiative. It was hoped that this strategy would encourage consumers to use performance information in making healthcare decisions, such as selecting a healthcare provider or preparing for a physician visit, and motivate providers’ QI efforts through comparisons of their performance with that of peers and other benchmarks. Early in the program, alliance leaders noted that performance measurement and reporting were given more emphasis than other AF4Q programmatic areas.

Prior to the AF4Q initiative, 4 of 16 AF4Q communities were reporting physician quality measures and 3 were reporting inpatient quality measures. During the initiative, all 16 AF4Q alliances released at least 1 report with physician quality measures, with 15 releasing multiple iterations. Fourteen alliances released at least 1 report on inpatient quality, with content consisting mostly of reformatted performance measures obtained from Medicare’s Hospital Compare program.

Indicators used for reporting physician quality were based predominantly on the Healthcare Effectiveness Data and Information Set performance measures modified for the ambulatory practice setting. The principal medium for reporting performance information was the alliances’ websites, with alliances receiving TA from expert providers and communications firms, and guidance from RWJF to help them make their websites consumer friendly. Some alliances expanded both the number of conditions and the number of measures in their reports as the AF4Q program progressed.

Alliances also were encouraged to include measures of patient experience in their reports. Only 1 alliance reported this information before the AF4Q program, but 12 alliances released at least 1 report on patient experience in physician office practices by the end of the program.

To construct report measures, over half of the alliances used aggregated insurance claims data across multiple payers and purchasers, with the remaining alliances using data gathered directly from providers or a combination of claims and provider-acquired data. Per a program requirement issued in 2008, alliances also worked to incorporate cost or efficiency measures into their public reports, such as appropriate use of back pain imaging, generic medication prescribing rates, average procedure costs, and total cost of care. At the end of the initiative, 8 alliances reported cost or utilization ambulatory measures such as these, and 6 alliances chose to report hospital readmissions measures.

Following the AF4Q program, 6 alliances had plans in place to sustain public reporting, while 6 did not plan to continue any public reporting. The future of public reporting in 4 alliances remains uncertain. The evolution of public reporting by alliances is described in detail by Christianson et al in this supplement.²⁹

Consumer Engagement Activities

RWJF targeted consumer engagement as 1 of the 3 initial key drivers critical to improving the quality of healthcare in communities. While RWJF viewed consumer engagement as essential to the AF4Q program, its vision for what consumer engagement would look like was less than clear initially. The consumer engagement goals outlined in the initial call for proposals were broad, requiring alliances to have “substantial and credible” consumer representation in alliance leadership and to support consumers in using healthcare information to make decisions.

Because alliances typically had little or no consumer engagement experience, and evidence-based consumer engagement interventions were limited, in 2007, RWJF convened the CELC. Through this learning community, alliances received TA from experts and alliance leaders shared ideas across communities. With this support, alliances were expected to concentrate work on self-management education and consumer use of public reports of provider quality; alliances began to focus on these 2 areas through a variety of approaches and with varying levels of embrace.¹³

Developing consumer engagement strategies was challenging and time consuming in almost all alliances, so RWJF’s expectations of consumer engagement were changed from broad to more specific.¹³ In a 2009 memo to the alliances, RWJF refined the “new” consumer engagement goal as: “Consumers will access and use health and comparative performance information to make healthcare decisions at key points.” The memo further emphasized the expectation that: “Local consumers and consumer advocacy groups (including those with diverse populations) would be represented on core leadership teams and have a voice in all aspects of AF4Q work.”¹⁴ As long as alliances engaged in these 2 areas, they could continue to build additional consumer engagement activities.

From the AF4Q program’s start to finish, alliances were required to develop efforts to: (1) encourage consumers to “shop” for high quality health providers; and (2) involve consumers in alliance governance. Self-management, which was initially encouraged by RWJF, gained traction with a number of alliances, as did involving consumers in ambulatory care QI teams. Alliances’ participation in these 4 areas of consumer engagement varied, and their level of investment ranged from enthusiastically embracing and prioritizing the work; making a concerted, but limited effort; to “checking the box.” The following are examples of alliance efforts in each area, which are further detailed by Greene et al in this supplement:¹⁵

• Self-management training included the evidence-based Stanford Chronic Disease Self-Management Program, community-based patient empowerment training, weekly newspaper articles on managing chronic conditions, health-related emails, and Internet-based educational information.
• Shopping for high-quality healthcare providers meant that alliances worked to develop consumer-friendly online public reports and disseminated the reports through traditional and social media, worked with partner organizations, and presented to consumers.
• Consumer involvement in alliance governance resulted in the addition of consumers to alliance boards, and the integration of consumers into various work groups and committees.
• Involvement of patients in QI included the introduction of consumers as “patient partners” integrated into QI teams and the creation of patient advisory councils.

While the type, scale, and scope of consumer engagement activities varied substantially across the AF4Q alliances, there seemed to be a consensus among alliance leaders that participating in the AF4Q program, at a minimum, elevated the awareness and discussion about the importance of the role of consumers within their organizations. Further, a few alliances developed substantial expertise in consumer engagement as a result of AF4Q program participation.

QI Activities

The AF4Q alliances were charged with facilitating communitywide QI. Throughout the program, the alliances participated in TA and learning collaboratives provided by the NPO. In addition, the alliances implemented, often in partnership with other local organizations, QI initiatives within their own communities.

All alliances participated in collaborative approaches to promote shared learning and diffusion of best practices. For example, since 2007, the Better Health Partnership in Cleveland has led 17 learning collaborative sessions modeled after the IHI’s Breakthrough Series³² and the Chronic Care Model.³³ To promote the diffusions of best practices, time was spent during each learning collaborative for providers who have successfully improved outcomes in the publicly reported data to share experiences, resources, and ideas.³⁴

Initiatives to develop and implement local PCMHs were among the most popular QI strategies across alliances. For example, the Maine alliance led or supported a number of PCMH activities since 2008. Maine Quality Counts expanded the number of Maine PCMHs through the Centers for Medicare & Medicaid Services’ Multi-Payer Advanced Primary Care Practice demonstration, State Innovation Model award, and the Health Homes initiative. The alliance continues to provide support to practices under each of these projects. In addition, the alliance leads a learning collaborative that includes sessions specifically targeted to PCMH practice teams.

Several alliances implemented practice coaching, providing individualized support to practices to help improve processes, overcome barriers, and offer training in QI methods. The Western New York alliance hired and trained Practice Enhancement Associates, who worked with more than 850 providers to make process improvements, utilize electronic health records (EHRs), obtain PCMH status, and implement best practices.³⁵

Alliances also focused on enhancement of HIT, including the diffusion and meaningful use of EHRs, and promoting the sharing of health information electronically across providers. Nearly all of the alliances educated local physician practices about HIT and its meaningful use through learning collaboratives or webinars. Other popular approaches included focusing on health information exchange to promote EHR adoption, meaningful use, and secure messaging with hospitals.³⁶ The majority of alliances also promoted the regional extension center (REC) activities to alliance members. The REC’s are located in multiple regions across the county and provide support and resources for providers to implement EHRs. Three alliances served as the local REC, supporting and assisting providers in EHR implementation and other HIT needs.³⁷

The legacy of the alliances’ QI activities are described in detail in the article by McHugh et al in this supplement.³⁰

Advancing Healthcare Equity

During phases II and III, RWJF directed the alliances to work toward advancing the collection of standardized data on patient REL by hospitals, ambulatory care providers, and health plans; using this data to stratify healthcare system performance measures; and identifying targets for disparity reduction. The NPO and TA providers devoted most of their early efforts in helping alliances engage providers and health plans in the standardized and routine collection of REL data. In the final phase of the AF4Q program, RWJF broadened the definition of disparities to include disparities by socioeconomic status (SES), geographic region (eg, rural vs urban), or other characteristics pertinent to a particular community. In addition, RWJF emphasized that alliances should incorporate disparities into their quality and cost goals. RWJF also encouraged alliances to move beyond collecting data, analyzing it, and identifying disparities, to creating interventions to target those disparities. Alliances were provided flexibility in the way they did this, and the NPO offered TA around designing and implementing practice-level interventions to target disparities.

All 16 alliances stratified some portion of their hospital or ambulatory care quality measures by REL and/or by SES. Most alliances chose to privately report the results to participating hospitals and ambulatory practices to inform local QI initiatives. Only 1 alliance released stratified data by REL via their alliance-sponsored public reporting website. Several other alliances released information about local disparities to the public through separate equity reports. These reports usually examined disparities by race or SES at the regional or state level rather than the medical group level. Barriers to the public reporting of disparities at more granular levels included: (1) concerns regarding adequate sample sizes of minority populations within medical groups to allow for stable performance estimates; and (2) apprehension regarding public perception of data on disparities.

As alliances worked to target disparities through QI and consumer engagement, they usually partnered with federally qualified health centers (FQHCs) or local organizations that served predominantly minority communities. For example, south central Pennsylvania’s alliance established an FQHC learning collaborative to improve staff QI capacity and share best practices to improve diabetes control and cancer screening. Also, Kansas City’s alliance teamed up with African American churches for a series of “Diabetes Sundays” to promote diabetes screening and linkage to primary care. For more details on alliances’ equity-focused activities during the AF4Q program, please see Jean-Jacques et al in this supplement.³¹

Payment Reform Activities

Alliances were directed by RWJF in May 2011 to engage in the payment reform arena. Most alliances responded by hosting local multi-stakeholder “payment summits” to discuss potential approaches to payment reform in their communities. Some alliances moved beyond this stage and implemented specific pilot payment programs. For example, the Washington alliance co-led a multi-payer PCMH reimbursement pilot program with the Washington State Health Care Authority in conjunction with 7 health plans and 8 medical groups. In Cincinnati, the alliance began a series of multi-payer PCMH pilot programs and was selected to participate in the CMS Innovation Center’s CPC initiative, which helped the alliance expand the number of payers and practices involved in their pilot.

Alignment

In addition to a focus on activities in the individual programmatic areas, as the name of the initiative suggests, alignment was hypothesized to be a key component. Alignment can be examined from at least 2 perspectives. The first relates to the degree to which stakeholders align around a common vision for an improved healthcare system in their community. In our midpoint assessment of the initiative, we saw early evidence that stakeholders shared a common vision, even if they were not always in agreement about the best strategies for achieving their goals.³⁸ Survey data collected near the conclusion of the initiative suggest that only about half of the participants across all of the alliances perceived their respective alliance as having the right set of strategies and programs in place for the future. Similarly, in interviews conducted during the same period, only 9 of the 16 alliance leaders were able to clearly communicate a strategic direction for their organization at the conclusion of the program.³⁹

Second, alignment can be viewed as the degree to which the program components of the AF4Q initiative (eg, consumer engagement and QI) are co-integrated. In designing the initiative, RWJF believed that the main program components were interrelated and mutually important for achieving an improved healthcare system. For example, RWJF believed that transparency related to ambulatory and inpatient care quality was essential for provider QI at the community level, and for meaningful consumer engagement and payment reform. When conceptualizing the AF4Q initiative, RWJF considered other important levers for achieving improvements in quality, such as providers’ adoption of HIT and expansion in insurance coverage. However, RWJF decided to focus on the levers that their resources could influence most, leaving other important levers to be addressed via public policy changes or the market. As it turns out, several of these other levers changed as a result of the passage of the ACA (approximately 3 years after the AF4Q initiative launched).

Our midterm evidence suggested that programmatic alignment was slow to materialize. Most of the Foundation’s requirements for the alliances were grouped by program areas (eg, publicly reporting results of ambulatory quality measures). TA was also largely targeted at individual program areas. At the start of the initiative, most alliances had limited, or no prior experience in many of the AF4Q program areas, presenting a steep learning curve. Typically, alliances established program-specific subcommittees comprised of persons with relevant expertise to address strategies and interventions. Consequently, most program components were initially addressed in silos, with minimal integration. While none of the alliances developed fully integrated strategies encompassing all AF4Q program areas, over time (and to varying degrees), they were able to integrate aspects of their strategies across program lines. For example, when the alliances were directed by RWJF during phase II to focus their consumer engagement activities on facilitating patient selection of providers, most utilized TA to make their performance measurement websites “more consumer friendly.” Also, with the encouragement of RWJF, alliances added consumer representatives to their governance bodies and/or program committees to interject the consumer perspective in planning and implementing interventions. Some of the more frequently utilized program alignment strategies included using performance measurement data for motivating and measuring the progress of physician QI activities; providing performance data for payment reform activities; stratification of performance data by race and ethnicity to identify opportunities for reducing disparities; and incorporation of consumer and family perspectives in physician QI efforts.

Discussion

The AF4Q initiative was an ambitious community-based initiative. Sixteen alliances were given similar levels of funding, a common set of expectations, and access to substantial TA. The initiative was dynamic, evolving in response to emerging trends in the healthcare field and lessons learned from the alliances’ early implementation efforts. In its nearly 10 years of existence, the AF4Q program expanded in scope, adding new programmatic areas in all but the final phase, when a substantial focus of the program shifted to sustaining existing efforts.

While all alliances worked within the same overarching framework of the program, there was substantial variation in how the program was implemented across the AF4Q communities. Several factors influenced the approaches taken by alliances, including differences in interpretation of the AF4Q initiative’s expectations; variation in alliance histories and stakeholder composition; the unique characteristics of each alliance’s geographic service area including demographics, market structure, and political environment; and alliance responses to large-scale events, such as the national economic recession and federal changes in healthcare policy.

Because the program was designed to provide some degree of latitude based on local context, no 2 alliances approached the program in exactly the same way. Additionally, in some cases, alliance leaders made successful arguments for taking a greatly different approach in 1 or more areas because of community characteristics (eg, a separate organization in place that already provided hospital quality reports to the community).

Variation in alliance history, composition, and leadership also played a role in program implementation. Some alliances were new organizations formed specifically around the grant, while others were existing organizations that had to incorporate grant-related activities into their organizational structure and operations. The pre-existing alliances varied in their prior experience with the AF4Q initiative’s programmatic areas. For example, the Minnesota alliance had experience in performance measurement and public reporting. Other alliances, such as Maine and Humboldt County, entered the program with experience in QI. While all alliances were able to involve stakeholders representing purchasers, providers, and consumers, there were significant differences in their relative balances across alliances and the program years. Additionally, the tenure and characteristics of alliance staff leaders also varied among AF4Q alliances, with some alliances experiencing numerous changes in leadership during the program and a handful of others retaining the same staff leaders for all phases of the program.

The AF4Q alliances’ geographic service areas differed in dramatic ways as well. Some alliances focused on entire states, some on multi-county metropolitan regions, while others focused on 1- or 2-county regions anchored by small cities or towns. Additionally, the demographics (eg, race and ethnicity) and healthcare market characteristics (eg, number of hospitals/health systems, major insurers, and typical size of ambulatory practices) of the AF4Q regions varied greatly.

Finally, national-level events that occurred during the program period also affected program implementation. The economic recession impacted all communities, some more than others, and ultimately may have influenced the time and resources stakeholders had to participate in alliance activities. In addition, 2 important pieces of federal healthcare reform legislation, the ACA and HITECH Act, were passed during the AF4Q program and led to the funding of several large-scale and national initiatives. These initiatives were diverse, and many complemented or overlapped with the AF4Q programmatic areas. Most AF4Q alliances leveraged their AF4Q program status and experiences to secure additional funding and roles through these initiatives and/or other initiatives that were launched by their respective states. While allowing the alliances to extend their reach, the number of major simultaneous projects also created challenges for some alliances, including needing to ramp up activity across multiple major areas, managing a diffuse set of activities, and creating activities that had unclear paths for sustainability and affected healthcare markets through such avenues as Medicaid expansion and the establishment of the Health Insurance Marketplace. (See eAppendix B for an overview of the large-scale and national initiatives that targeted or included a role for local multi-stakeholder healthcare collaboratives during the AF4Q program.)

As previously noted, one of RWJF’s goals for this initiative was to develop models to be utilized in other communities based on lessons learned during the program. There are important features of the AF4Q program that make it relevant to other initiatives, and the valuable lessons learned can inform current and future programs. This article describes the complex evolution of the AF4Q program and intends to provide the background needed for other articles in this supplement, which focus on detailed aspects of the program and our summative findings. Collectively, the evolution and history of the AF4Q program, combined with the midterm findings presented in 2012, and the summative findings presented in this supplement, represent one of the largest, most formal program evaluations in history.

Author affiliations: School of Public Health, The University of Michigan Ann Arbor, MI (JAA); Jeff Beich Consulting, Grand Island, NY (JB); School of Public Health, University of Minnesota, Minneapolis, MN (JBC); Center for Health Care and Policy Research, Penn State University, University Park, PA (DCF, BL, DPS, BWS, LJW); School of Nursing, George Washington University, Washington, DC (JG); Northwestern University, Feinberg School of Medicine, Division of General Internal Medicine and Geriatrics, Chicago, IL (MJJ); Center for Healthcare Studies, Northwestern University, Feinberg School of Medicine, Chicago, IL (MM); Health Policy and Administration, Penn State University University Park, PA (DPS).

Funding source: This supplement was supported by the Robert Wood Johnson Foundation (RWJF). The Aligning Forces for Quality evaluation is funded by a grant from the RWJF.

Author disclosures: Dr Alexander, Dr Beich, Dr Christianson, Dr Jean-Jacques, Dr McHugh, Dr Scanlon, Ms Farley, Ms Leitzell, Ms Shaw, and Ms Wolf report receipt of grants from RWJF. Dr Scanlon also reports meeting or conference attendance for RWJF. Dr Greene reports meeting or conference attendance for Insignia Health.

Authorship information: Concept and design (JAA, JB, JBC, MJJ, DPS); acquisition of data (JB, DCF, BL, MM, DPS, BWS, LJW); analysis and interpretation of data (JAA, JG, MJJ, BL, MM, DPS, BWS); drafting of the manuscript (JB, JBC, DCF, JG, BL, DPS, BWS, LJW); critical revision of the manuscript for important intellectual content (JAA, JB, JBC, DCF, JG, MJJ, MM, DPS, LJW); obtaining funding (DPS); and administrative, technical, or logistic support (BL, BWS).

Address correspondence to: bjh156@psu.edu.

REFERENCES

1. Delmarva Foundation. Regional coalition collaboration guide. http://archive.ahrq.gov/professionals/quality-patient-safety/quality-resources/tools/collabguide/collabguide.pdf. Published April 2008. Accessed May 19, 2016.

2. U.S. Department of Health and Human Services. 2011 report to Congress: national strategy for quality improvement in health care. www.ahrq.gov/workingforquality/nqs/nqs2011annlrpt.htm. Published March 2011. Accessed May 19, 2016.

3. Beich J, Scanlon DP, Boyce PS. A community-level effort to motivate physician participation in the National Committee for Quality Assurance Diabetes Physician Recognition Program. Popul Health Manag. 2010;13(3):131-138. doi: 10.1089/pop.2009.0035.

4. Aligning Forces for Quality: The Regional Market Project. Robert Wood Johnson Foundation website. www.rwjf.org/en/library/programs-and-initiatives/A/aligning-forces-for-quality--the-regional-market-project.html. Accessed May 19, 2016.

5. Miller CE, Weiss AF. The view from Aligning Forces to a Culture of Health. Am J Manag Care. 2016:22(suppl 12):S333-S336.

6. Scanlon DP, Beich J, Alexander JA, et al. The Aligning Forces for Quality initiative: background and evolution from 2005 to 2012. Am J Manag Care. 2012;18(suppl 6):S115-S125.

7. Scanlon DP, Alexander JA, Beich J, et al. Evaluating a community-based program to improve healthcare quality: research design for the Aligning Forces for Quality initiative. Am J Manag Care.

2012;18(suppl 6):S165-S176.

8. Scanlon DP, Wolf LJ, Alexander JA, et al. Evaluating a complex, multi-site, community-based program to improve healthcare quality: the summative research design for the Aligning Forces for Quality initiative. Am J Manag Care. 2016:22(suppl 12):eS8-eS16.

9. Preskill H, Jones N. A practical guide for engaging stakeholders in developing evaluation questions. www.rwjf.org/en/library/research/2009/12/a-practical-guide-for-engaging-stakeholders-in-developing-evalua.html. Published January 1, 2009. Accessed May 19, 2016.

10. The Aligning Forces for Quality initiative: background and evolution from 2005 to 2012-eAppendix. The American Journal of Managed Care website. www.ajmc.com/journals/supplement/2012/af4q/a413_12sep_scanlon_s126_eapdx. Published September 22, 2012. Accessed May 19, 2016.

11. Robert Wood Johnson Foundation. Lessons learned: Aligning Forces for Quality. Technical assistance investment allocation. http://forces4quality.org/af4q/download-document/12021/Resource-af4q_ta_allocation.pdf. Accessed May 19, 2016.

12. Robert Wood Johnson Foundation. Aligning Forces for Quality: The Regional Market Project. Call for Proposals. Released 2006.

13. Hurley RE, Keenan PS, Martsolf GR, Maeng DD, Scanlon DP. Early experiences with consumer engagement initiatives to improve chronic care. Health Aff (Millwood). 2009;28(1):277-283. doi: 10.1377/hlthaff.28.1.277.

14. The Robert Wood Johnson Foundation. Aligning Forces for Quality: consumer engagement activities (memorandum). March 2009.

15. Greene J, Farley DC, Christianson JB, Scanlon DP, Shi Y. From rhetoric to reality: consumer engagement in 16 multi-stakeholder alliances. Am J Manag Care. 2016:22(suppl 12):S403-S412.

16. Transforming care at the bedside. Institute for Healthcare Improvement website. www.ihi.org/engage/initiatives/completed/TCAB/Pages/default.aspx. Accessed May 19, 2016.

17. Harvey JB, Beich, J, Alexander JA, Scanlon DP. Building the scaffold to improve health care quality in Western New York. Health Aff (Millwood). 2012;31(3):636-641. doi: 10.1377/hlthaff.2011.0761.

18. Painter MW, Chernew ME. Counting change: measuring health care prices, costs and spending. Robert Wood Johnson Foundation website. www.rwjf.org/en/library/research/2012/03/counting-change.html. Published March 1, 2012. Accessed May 19, 2016.

19. Comprehensive Primary Care Initiative. Centers for Medicare & Medicaid Services website. https://innovation.cms.gov/initiatives/Comprehensive-Primary-Care-Initiative/. Updated May 13, 2016. Accessed May 19, 2016.

20. Consumer Reports Health. Consumer Reports/Massachusetts Health Quality Partners Health Insert. Special report for Massachusetts residents: how does your doctor compare? http://c354183.r83.cf1.rackcdn.com/MHQP%20Consumer%20Reports%20Insert%202012.pdf. July 2012. Released May 31, 2012. Accessed May 19, 2016.

21. Consumer Reports Health. Consumer Reports/Minnesota Community Measurement Health Insert. Special report for Minnesota residents: how does your doctor compare? http://consumerhealthchoices.org/wp-content/uploads/2012/09/Minnesota2012.pdf. October 2012. Released July 31, 2012. Accessed July 14, 2016.

22. Consumer Reports Health. Consumer Reports/Wisconsin Collaborative for Healthcare Quality Health Insert. Special report for Wisconsin residents: how does your doctor compare? http://consumerhealthchoices.org/wp-content/uploads/2013/01/Wisconsin2012.pdf. February 2013. Released December 27, 2012. Accessed July 14, 2016.

23. Robert Wood Johnson Foundation. Expert panel review of Aligning Forces for Quality. Robert Wood Johnson Foundation website. www.rwjf.org/content/dam/farm/reports/reports/2012/rwjf403190. Published November 2012. Accessed May 19, 2016.

24. The AF4Q National Program Office (George Washington University). Aligning Forces for Quality: Technical assistance for strategic planning and organizational development (memorandum). April 10, 2012.

25. Robert Wood Johnson Foundation. Aligning Forces for Quality: the fourth phase. Call for Proposals. http://forces4quality.org/node/5023.html. Released 2012. Accessed July 14, 2016.

26. Impact: Addressing health disparities. Aligning Forces for Quality website. http://forces4quality.org/node/6609.html. Published August 7, 2013. Accessed May 19, 2016.

27. Freedman Healthcare, LLC; Zema C, Shaller D, Stopansky J. Aligning Forces for Quality Gathering Evidence Project Summary; 2014.

28. Collaborative Health Network. The Network for Regional Healthcare Improvement website. www.nrhi.org/collaborative-health-network/. Accessed May 19, 2016.

29. Christianson JB, Shaw BW, Greene J, Scanlon DP. Reporting provider performance: what can be learned from the experience of multi-stakeholder community coalitions? Am J Manag Care. 2016:22(suppl 12):S382-S392.

30. McHugh M, Harvey JB, Hamil J, Scanlon DP. Improving care delivery at the community level: an examination of the AF4Q legacy. Am J Manag Care. 2016:22(suppl 12):S393-S402.

31. Jean-Jacques M, Mahmud Y, Hamil J, Kang R, Duckett P, Yonek JC. Lessons learned about advancing healthcare equity from the Aligning Forces for Quality initiative. Am J Manag Care. 2016:22(suppl 12):S413-S422.

32. Changes for improvement. Institute for Healthcare Improvement website. www.ihi.org/resources/pages/changes/changestoimprovechroniccare.aspx. Accessed May 19, 2016.

33. Our approach. Improving Chronic Illness Care website. www.improvingchroniccare.org/. Accessed May 19, 2016.

34. Learning collaborative summits. Better Health Partnership website. www.betterhealthpartnership.org/lcsummits_detail.asp. Accessed May 19, 2016.

35. Spotlight on Western New York. Aligning Forces for Quality website. http://forces4quality.org/spotlight-western-new-york.html. Published January 26, 2015. Accessed May 19, 2016.

36. Health Information Exchange (HIE). HealthIT.gov website. https://www.healthit.gov/providers-professionals/health-information-exchange/what-hie. Updated May 12, 2014. Accessed May 19, 2016.

37. Regional Extension Centers. HealthIT.gov website. https://www.healthit.gov/providers-professionals/regional-extension-centers-recs. Updated November 5, 2015. Accessed May 19, 2016.

38. Alexander JA, Scanlon DP, McHugh MC, et al. Midterm observations and recommendations from the evaluation of the AF4Q initiative. Am J Manag Care. 2012;18(suppl 6):S126-S132.

39. Alexander JA, Hearld LR, Wolf LJ, Vanderbrink JM. Aligning Forces for Quality multi-stakeholder healthcare alliances: do they have a sustainable future? Am J Manag Care. 2016:22(suppl 12):S423-

S436.

Download PDF: The Aligning Forces for Quality Initiative: Background and Evolution From 2005 to 2015