Midterm Observations and Recommendations From the Evaluation of the AF4Q Initiative

Objective: To offer midterm observations and recommendations based on how Aligning Forces for Quality (AF4Q) alliances are faring in their journey toward improving healthcare quality at the community level.

Study Design: This study used a mixed method design.

Methods: Longitudinal evaluation data to date were analyzed, including results from multiple surveys, qualitative analysis of key informant interviews, review of secondary documents and analysis of secondary data, and ongoing tracking of the activities of the 16 participating alliances. The observations and recommendations are based on consensus achieved by the AF4Q evaluation team investigators after in-depth iterative discussions.

Results: Six formative observations are identified and discussed: (1) stakeholder support and participation has been maintained despite changes in economic and political environments; (2) progress on program goals has been slow; (3) the “alignment” in the AF4Q initiative has been slow to materialize; (4) the AF4Q initiative has established a productive network of peer communities; (5) the impact of the AF4Q initiative, and the time to observe impact, vary by community, based on history and context; and (6) sustainability is the major future challenge for the AF4Q initiative.

Conclusions: Multi-stakeholder alliances’ efforts to improve quality should be viewed as “pieces of the health reform puzzle” rather than stand-alone solutions. As healthcare reform is challenged politically, alliances can practice the bipartisanship that focuses conversation on what is good for the community and how best to achieve community goals amid a potential sea of change in both federal and state policy and funding.

(Am J Manag Care. 2012;18:S126-S132) The Robert Wood Johnson Foundation’s (RWJF’s) Aligning Forces for Quality (AF4Q) initiative is arguably the most ambitious attempt in American history to improve the quality of healthcare in communities using a collaborative, multi-stakeholder approach. Details about the history and evolution of the AF4Q initiative are provided in the article by Scanlon et al in this supplement.¹ In general, membership in the AF4Q initiative provides the 16 participating alliances (ie, the generic term used for the multi-stakeholder partnership in each community) with 3 complementary components: (1) a set of program expectations and interventions to achieve those goals; (2) technical assistance (TA) to help alliances develop meaningful population-based interventions; and (3) grant funds to support the staffing and infrastructure required to start and maintain the work of the alliance over a protracted period until the alliance and/or its activities become self-sustainable. While each alliance customizes its work to fit the local context and needs of its community, the program requires that all alliances be actively involved in 5 main programmatic areas: (1) public reporting of quality measures; (2) efforts to engage healthcare consumers and patients in their health and the care received in communities (ie, consumer engagement); (3) systems-level quality improvement; (4) healthcare equity and disparities reduction; and (5) payment reform. Additionally, alliances must focus on the sustainability of these activities.

The AF4Q initiative evolved from the RWJF’s observation that documented quality problems persisted across communities despite isolated examples of successfully addressing those problems with existing tools. The RWJF hypothesized that the missing link was local implementation and the support of a diverse set of committed stakeholders to address subpar quality in communities. Acknowledging that much of the work of the 16 AF4Q alliances would be groundbreaking, the RWJF intended for these alliances to serve as “learning laboratories” for other communities, policy makers, and others interested in local solutions to the nation’s most pressing healthcare quality and value problems. Thus, the RWJF funded a long-term independent formative and summative evaluation of the AF4Q initiative.

The summative component of the evaluation is designed to measure and report on the AF4Q initiative’s effect at the conclusion of the RWJF’s formal support in 2015. The formative component of the evaluation is ongoing, and it is designed to document the initiative’s evolution and highlight important lessons and observations along the way, including examples of barriers, facilitators, and progress on intermediate outcomes. At this intermediate juncture, we can issue important formative observations to assess how these alliances are faring on their journeys toward improving healthcare quality at the community level.

As described in this supplement, the AF4Q initiative evaluation employs qualitative, quantitative, and mixed research methods. More details about the AF4Q initiative evaluation design and associated data collection and analysis can be found in the article by Scanlon et al in this supplement.² The formative observations we present in this article, and the associated recommendations, are based on analysis of longitudinal evaluation data to date, including results from multiple surveys, qualitative analysis of key informant interviews, review of secondary documents and analysis of secondary data, and ongoing tracking of the activities of the 16 participating alliances. The observations and recommendations presented in this paper are based on consensus achieved by the evaluation team investigators after iterative in-depth discussions.

The formative observations and recommendations that we present should be of interest to those involved in multi-stakeholder quality improvement work in communities, regardless of AF4Q affiliation, those funding this type of community-level work, policy makers, and others interested in seeing these efforts expanded to other communities. Our research is particularly relevant, as community-based approaches to improving healthcare have become a prominent component of national and local health reform efforts.³

Formative Observations From the AF4Q Evaluation to Date

1. Stakeholder support and participation has been maintained despite changes in economic and political environments

Healthcare plans, provider organizations, purchasers, and consumers are not always natural allies in healthcare improvement, and some would argue that the historical separation of interests among these stakeholders is the source of our fragmented healthcare delivery system and a major reason for documented poor quality.⁴ While communities were selected for the AF4Q initiative based on their perceived ability and willingness to collaborate, the ability to establish and maintain the legitimacy and neutrality of the alliance is something that should not be taken for granted. Our tracking of alliance efforts since the launch of the AF4Q initiative suggests that alliances have been able to maintain the interest, support, and commitment of a diverse set of stakeholders in their respective communities. Importantly, this commitment has been maintained over approximately 5 years, and during a time in which the nation has experienced a deep recession and the passage and implementation of major healthcare reform. Evidence of this sustained participation and commitment comes from our key informant interview data and results from 3 waves of a survey given to those who participated in the alliances’ activities. This alliance survey, which asks participants about the value of the alliance for the community, individual, or organization the respondent represents, is one way we measure if support for the alliance wanes over time.⁵ For example, results pooled from all alliances, for all participating stakeholders, suggest that: (1) respondents view the alliances’ participants as having a clear and shared vision of health in their communities (89%); (2) alliances are taking meaningful actions (83%); and (3) the benefits of participating in the alliance outweigh the costs of participation for individuals and organizations (88%). Importantly, the level of agreement on these questions has remained steady or increased across the 3 waves of the survey, for both the recurring panel respondents and the entire sample of respondents.

It is no small feat that the AF4Q alliances have been able to keep a diverse set of stakeholders at the table, as prior research has shown that mistrust, competing interests, cultural differences, and personal and organizational histories often conspire to make collaboration a challenge.⁶ Further, the personal ties and informal networks found in alliances often increase the vulnerability of these organizations when turnover, burnout, and attrition among key participants disrupt momentum and cohesion. A key factor that has kept AF4Q alliance members working toward a common goal of improved quality of care is the perception that the alliances were neutral players with no hidden agendas or implicit arrangements with particular stakeholders. As a credible, neutral entity, most alliances have been able to offer a level playing field for the discussion of hard issues and provide a context for leaders to work with different stakeholders without raising questions about credibility or ulterior motives. As one informant observed: “[The] AF4Q [initiative]…really…built the credibility of [the alliance] because what that project did was to say ‘Yes we can work together, yes we can agree on these common measurements’… Because of the AF4Q program…it’s built capacity, credibility, and I think with the medical groups, the health plans, and the state. And I don’t think they could have done it without that grant.”

Our ongoing research has found that there are 2 factors, in particular, that have helped to build the trust and neutrality that we have observed: the leadership of the alliances and the decision-making processes employed by the alliances. Alliance leadership plays a critical role in establishing and sustaining neutrality and legitimacy and, by extension, member participation and commitment, because these organizations cannot rely on the formal structure and authority, such as ownership, that facilitate action in other organizations. Credible leadership has been established through interpersonal skills and effective communication, and it is closely tied to whether the alliance itself is viewed by participants as a body where various stakeholders can “let their defenses down” and where critical, open discussions can occur without the threat of negative consequences.

Similarly, alliances that utilize open and inclusive decision-making processes (eg, all issues and options are identified before making a decision and members feel like they have a voice in the direction eventually taken) appear to be more successful at building consensus around vision, mission, and strategy. Open decision-making processes promote trust among members and cultivate a climate in which members can freely and safely exchange ideas.

These efforts require time, leadership, and patience. We have observed that progress on AF4Q programmatic efforts is often built on the time-consuming and ongoing processes of trust building and managing cultural differences among stakeholders—critical but often underestimated areas in community-based efforts to improve healthcare quality.

2. Progress on broad program goals has been slower

While support for the alliances and their broad vision of improving community health remains strong, progress toward measurable population-level change, such as improved quality, efficiency, and reductions in health disparities, has been slow for 2 reasons. First, as suggested above, it took longer to establish the governance structures and processes than originally anticipated. Second, the evidence base surrounding the key programmatic areas in the AF4Q initiative was not as well established as originally assumed, requiring alliances to engage in defining and developing the evidence base before developing and implementing their strategies.

The underdeveloped evidence base meant that many early conversations in AF4Q communities were centered on defining these major program areas (eg, consumer engagement, quality infrastructure) and discussing the merits of potential interventions. The work of the alliances in the early stages of the AF4Q initiative focused primarily on developing pilot programs or local evidence to determine what works, rather than on achieving broad, community-level changes in quality of care. For example, Hurley et al determined that the alliances spent a significant amount of time trying to define what consumer engagement meant to various stakeholders in order to broker agreement about which aspects of consumer engagement should be prioritized and pursued.⁷ The overall dearth of evidence about effective interventions to improve consumer engagement, especially at a communitywide level, contributed to initial strategies that emphasized smaller pilot initiatives.

While pilot projects and small-scale interventions can provide an opportunity to develop programs, test theories, and gain experience, this approach will only achieve communitywide improvement if pilot efforts are effective and diffused to the larger community of healthcare providers, consumers, and other stakeholders. The tipping point for achieving community-level improvements in quality of care has yet to be realized in the AF4Q initiative. Such efforts require at least 3 conditions: effective interventions, development of sufficient financial and human resource capacity to expand the pilots, and collaborative effort over the long term to sustain the expansion. We have observed that alliances have only recently come to grips with the interrelated issues of sustainability and capacity.

3. The “alignment” in AF4Q has been slow to materialize

A key hypothesis of the AF4Q initiative is: if programmatic activities such as public reporting, consumer engagement, quality improvement, disparities reduction, and payment reform operate synchronously, they can improve the quality of care at the community level. For example, the physician quality performance measures contained in the communities’ public reports are hypothesized to influence consumers to select providers and physician practices who perform well on those measures. These measures are also hypothesized to incentivize physicians and practices to initiate processes to improve on measures that the report suggests are underperforming. By extension, if such synchronicity does not occur and programmatic efforts are loosely coordinated, the impact may be significantly less. We observe that alignment of program elements to date has been relatively modest. Several explanations for the lack of significant programmatic alignment may offer direction for corrective action. First, alliances pay selective but strategic attention to some program areas over others. Alliances emphasize what they know and do best. This selective, strategic attention also is driven by the economic reality that RWJF funding is intended to provide a catalyst for community-based efforts to improve quality, rather than sufficient operating capital to sustain all programmatic activities over the long term. Choices must be made, and confronted with those choices, alliances place their bets on activities they feel will build on existing strengths and yield the biggest payoff.

Second, the organization of alliance programmatic activities does not always lend itself to coordinated efforts across programmatic lines. Alliances typically have organized their work around task forces or work groups which focus on single programmatic areas. While this form of functional organization, or siloing, offers the advantages of in-depth specialization, efficiency, and participant recruitment, it also comes at the price of increasing difficulty in coordination across work groups. For example, our interviews revealed that stakeholders representing a particular programmatic area often have limited knowledge about activities occurring in the other core areas. While some alliances have cross-representation on committees (most notably, placing consumers on performance measurement and public reporting and/or quality improvement committees), it is not clear if this has produced the intended level of programmatic alignment. The result is a set of activities that have often proceeded along parallel paths, but not in such a way as to create true synergies or leverage in the overall effort to improve quality.

Third, both the timing and specificity of expectations from the RWJF differed across program areas, resulting in uneven progress and further complicating efforts to coordinate. For example, public reporting in general has experienced the greatest progress, while progress in the other areas varies by community. This is largely attributable to greater clarity about the expectations for public reporting at the outset of the initiative and more specific direction from the RWJF. Key informants have noted that other areas, by contrast, have received less clear direction from the RWJF, and are perhaps more inherently ambiguous. This unevenness in program development makes integration of the different components challenging. For example, although public reporting is well under way in all 16 AF4Q communities, broad community awareness of the alliances and their activities remains a persistent issue. This necessarily limits the ability of the alliances to widely engage consumers in using the reports to manage their healthcare and more actively engage with their providers. Early evidence suggests that the reports have had greater penetration and influence with providers, particularly in increasing awareness of quality problems and promoting the use of quality improvement processes to address those problems.

4. The AF4Q initiative has established a productive network of peer communities resulting in increased learning opportunities, diffusion of innovations and best practices, and a common voice representing community alliances

An early positive impact of the AF4Q initiative was the creation of a network of communities that provided support to one another in the development and implementation of programmatic interventions designed to improve the quality and value of healthcare. As a means of acquiring more grounded support and assistance for their efforts, alliances often looked to one another for practical, targeted help and TA on issues ranging from measurement of quality to consumer engagement strategies. Such learning often begins with an alliance reaching out to another with greater experience or expertise in an area of need. Because of the multifaceted nature of the AF4Q initiative, and variation in alliance approaches, expertise in specific programmatic areas naturally resides with different alliances. Typically, alliances had specialties based on prior work or areas given priority in the AF4Q initiative. This resulted in some alliances developing a reputation as the “go to” organizations for, say, measurement and public reporting, while others established a reputation in quality improvement or consumer engagement strategies.

For example, alliances have worked closely together to develop their public reports and several have employed the Community Checkup approach, first used by the Puget Sound Health Alliance (PSHA), an early adopter of public reporting. The PSHA’s early decision to emphasize aggregate community quality variation rather than individual physician variation in the dissemination and messaging of its initial report, called the Community Checkup, was a way to gain acceptance and support for its physician practice report from the provider community. Thus, the PSHA was able to initially steer discussion of the Community Checkup toward opportunities for improvement in the community rather than defending the methodology used to produce the scores and ratings of individual physicians.⁸ Many alliances, most notably Better Health Greater Cleveland, learned from the PSHA’s experience and structured their public report dissemination plans using lessons from the PSHA.

The short-term result of such networking was an increase in learning opportunities, and the diffusion of innovations and best practices that can be adapted to local needs and conditions. As one alliance participant noted: “I find it invaluable to talk to the different communities, talk to the project directors and the people that are doing the actual work.” The AF4Q initiative facilitates this cross-community sharing in many ways, but most notably by sponsoring semi-annual in-person meetings of alliance project directors and their broader teams, including a representation of involved stakeholders. In addition, funding has been made available for alliances to take teams to travel to other alliances to learn about activities in particular areas. Furthermore, many AF4Q alliances are also members of other national learning communities, such as the Department of Health and Human Services’ Chartered Value Exchange Program and the Office of the National Coordinator for Health Information Technology’s Beacon Community Program, allowing alliances to share information across content silos and envision opportunities for synergies across many similar, yet distinct, programs.

AF4Q alliances also have used their collective influence to weigh in on key areas related to national health policy, send common messages to key stakeholder groups, and benefit from the efficiencies that come from numbers. For example, the opinions of AF4Q alliances are now routinely sought out regarding measurement development and implementation by the National Quality Forum, the non-profit organization that uses a consensus process to endorse performance measures and set priorities for quality improvement and measure development.

Many alliances have had challenges securing the local commitment and participation of certain large national health plans, particularly in their work related to supplying data for public reporting efforts and in conversations related to payment reform models. In these circumstances, alliances have collectively used the fact that these plans operate in multiple AF4Q markets to send a unified message to the health plans’ national offices about the importance of local plan participation. Finally, alliances have pursued potential economies of scale associated with conducting common core alliance activities, such as receiving vendor estimates for survey work on assessing patient experience, selecting and contracting with third-party claims data aggregators, or engaging the assistance of experts on website design.

Alliance networking arose partly in response to gaps in the formal TA offered early in the AF4Q initiative. This networking has provided insight into the type of formal TA that the alliances find most valuable. First, it appears that formal TA is most useful when customized to the local needs and schedules of the alliances and developed in cooperation with alliance leadership. This type of customized TA contrasts with typical one-size-fits-all approaches, such as educational webinars or consultants with canned products. An important lesson for those developing programs to support community initiatives is that customized TA is more effective, requires flexibility, and is a process to match the time frame and needs of the alliances with the availability of the contracted experts. In addition, alliances clearly prefer on-site TA experts who understand and appreciate the local context and can engage in specific issues facing their communities, including direct interaction with key stakeholders representing the alliance when necessary.

5. The impact of the AF4Q initiative, and the time to observe impact, will vary by community, based on history and context

AF4Q alliances cannot be isolated from the context in which they operate. Local community history and context has facilitated and impeded the AF4Q initiative through local factors, such as the existence of electronic health records; delivery system fragmentation or consolidation; past efforts of public reporting; market share of local and national health plans; and the existence of a valued quality improvement organization or similar entity with a strong quality improvement focus. In addition, the history of collaboration or conflict during past initiatives, and the relationships—or lack thereof—among key stakeholder groups, availability of resources, and the characteristics and personalities of key leaders and organizations, will impact the pace of alliance progress. Because each community starts from a different point, flexibility in program content and expectations for progress may be required. While it is natural to want to estimate an aggregate program effect after a certain period of time, it is more realistic and often more useful to recognize that the time needed to observe and measure the hypothesized program effects will vary by community.

6. Sustainability is the major future challenge for the AF4Q initiative

Although development of a sound business model and securing monetary resources are intuitively obvious pieces in the sustainability puzzle, alliance leaders cannot afford to focus solely on money. Effective management of stakeholder interests and other capacity-building issues such as sustaining participation, recruiting new stakeholders, balancing long-term strategic objectives with short-term wins, and establishing collaborative capacity are equally important pieces. As stated by an alliance participant, an effective alliance takes full advantage of the power of collaboration: “The goals are huge and the resources are small, yet the ability to leverage these multiple stakeholders to use resources that are within their reach—that’s a huge potential. So I think that’s the power of this [collaborative].”

An important challenge to spread and sustainability is that AF4Q alliances are often providing a public good. As such, “free rider” benefits that accrue to organizations may dissuade them from contributing actual dollars or time in ongoing support of the work of the alliance. Further, what is good for the community is not always good for individual participants—particularly those who stand to lose from the public solution. This may also erode support for alliances as they progress beyond the pilot stage. Absent any direct incentives or requirements to participate (eg, tax incentives or penalties), the AF4Q initiative will achieve only what voluntary participation can accomplish. Depending on aspirations and the ability of alliances to influence free riders or reluctant participants, this may fall short of more ambitious, long-term goals and objectives.

Despite the challenges associated with establishing and sustaining multi-stakeholder alliances, several contemporaneous trends are supportive of multi-sector collaboration to improve community healthcare delivery. These include increased national and state focus on community benefit, changing financial policies and programs that tie payment to quality outcomes and payment based on providers’ risk profiles, growing emphasis on managing care (not merely cost) for the chronically ill, and better understanding of the multiple determinants of health and healthcare quality. Although much of the initial AF4Q initiative work has successfully reached small constituencies—for example, a small number of providers or patients/consumers within a community—conversations within alliance work groups have moved from “What are we going to do?” to “How are we going to spread/sustain the work?” As the RWJF plans to end its financial support for the AF4Q alliances in mid 2015, it is encouraging alliances to strategically plan for their continued sustainability. In many respects, one important measure of the AF4Q alliances’ value will be the degree to which community stakeholders are willing to provide support for this work to continue. Our research will continue to track and monitor these efforts and identify important lessons from them.

Recommendations

Based on the formative observations discussed above, we offer 4 general recommendations in the spirit of making our findings useful to key stakeholders. It is important to acknowledge that because of differences in the local context of each community, the applicability of each recommendation to any given community may vary.

1. Increase the evidence for effective and scalable community interventions

The progress and speed of community reform efforts is directly tied to whether community stakeholders can implement proven strategies or whether they have to invent the wheel. Experience is a valuable teacher, but it is frequently time consuming and inefficient. The AF4Q initiative was based on an assumption that there was evidence for most of the programmatic interventions to improve quality at the community level, and that these strategies just needed to be implemented and scaled by alliances. As discussed above, this has proved not to be the case in many program areas, and the lack of an evidence base for community-based health reform has significantly affected the content, implementation, and trajectory of the AF4Q initiative.

While the National Institutes of Health (NIH) has long funded research in specific clinical areas, it has provided little funding on many of the focal areas related to the AF4Q initiative, including research related to how multi-stakeholder efforts can facilitate improvements in the quality of care. Some of the needed evidence can be found in disciplines such as systems engineering and operations research, individual and organizational behavior change and response, the science of diffusion and scalability, and the decision sciences. An obvious home for this type of research within the NIH is in the area of dissemination and implementation research, a new area of emphasis for the NIH that is garnering increased attention.⁹ NIH leadership, including the leadership of its respective divisions, should look to the potential value of multi-stakeholder efforts for solving important health problems and consider funding more research on the impact of these efforts.

2. Integrate federal and state health reform efforts with multi-stakeholder alliances

As the federal and state governments continue to implement healthcare reform efforts, including new methods of payment, health insurance exchanges, and the development of new quality measures, these efforts should better integrate with the AF4Q initiative and similar local initiatives. Local multi-stakeholder alliances such as those operating under the AF4Q initiative are natural allies and potentially productive partners in reform efforts, increasing the potential for synergy and quicker and better acceptance and implementation in the community. For example, alliances’ efforts in public reporting and measurement could benefit payment reform initiatives. The Centers for Medicare & Medicaid Services (CMS) and state Medicaid programs may join efforts with alliances by providing their claims data for all-payer performance measurement databases, allowing alliances to better paint the broad picture of the community and to stimulate more effective population-level improvement. Recognition and support for community-based reform provides alliances with legitimacy, enabling them to effectively partner with and complement other initiatives in a coordinated, non-duplicative fashion.

3. Account for the time necessary to establish effective alliance governance and leadership

Although stakeholders in multi-stakeholder alliances are often assumed to cooperate and align out of a common desire to “fix a broken system,” sponsors underestimate the importance of governing and managing a voluntary entity comprising organizations or groups with historically competing interests and varying world views. Significantly different decision-making styles, expectations, and perspectives on healthcare quality, including expectations regarding time frame to observe expected effects, often create conflict and impede progress at the outset of the alliances’ work. Sustaining these organizations requires ongoing effort and constant monitoring by alliance leadership, both of which demand time and resources that many sponsors should, but typically do not, account for in their timelines and funding commitments.

4. Temper expectations about the likely short-term effects of community-based multi-stakeholder initiatives

Despite a strong case for investing in local solutions to healthcare quality problems, healthcare remains competitive. Community initiatives such as the AF4Q initiative are trying to appeal to the common good when individual organizations or entities may have conflicting motives. Absent any direct incentives or requirements to participate (eg, tax penalties for health plans that don’t provide data to community all-payer data repositories), community-based reform efforts face the limits of what voluntary participation can accomplish. Depending on aspirations, this may fall short of stated goals and objectives, and there may be a limit to what can be accomplished with voluntary participation in the context of directly competing alliance participants in many markets. Thus, funders and policy makers might consider allowing for incremental progress toward quality improvement in community-based reform efforts, rather than expecting short-term transformation or easily implemented silver bullets.

Conclusion

Our mid-term observations of the AF4Q initiative suggest that, while there is reason to be hopeful that multi-stakeholder alliances focused on improving healthcare quality can move communities toward better healthcare outcomes, their efforts should be viewed as pieces of the healthcare reform puzzle rather than stand-alone solutions. For example, multi-stakeholder alliances like the AF4Q initiative can provide “boots on the ground” support to other complementary efforts, such as those of the CMS-contracted quality improvement organizations or the recent activities related to healthcare reform (including programs sponsored by the CMS Innovation Center or the Office of the National Coordinator for Health Information Technology). In addition, as healthcare reform is challenged politically, alliances can practice the bipartisanship that focuses conversation on what is good for the community and how best to achieve community goals amid a veritable sea of change in both federal and state policy and funding.

Author affiliations: School of Public Health, University of Michigan, Ann Arbor, MI (JAA); Penn State University, University Park, PA, and Jeff Beich Consulting, Grand Island, NY (JB); Division of Health Policy and Management, University of Minnesota School of Public Health, Minneapolis, MN (JBC); Center for Healthcare Equity and Institute for Healthcare Studies, Division of General Internal Medicine, Northwestern University, Feinberg School of Medicine, Chicago, IL (RH-W); Institute for Healthcare Studies and Department of Emergency Medicine, Northwestern University, Feinberg School of Medicine, Chicago, IL (MCM); Department of Health Policy and Administration, Penn State University, University Park, PA (JNM, DPS); Center for Health Care and Policy Research, Penn State University, University Park, PA (DPS).

Funding source: This supplement was supported by the Robert Wood Johnson Foundation (RWJF). The Aligning Forces for Quality evaluation is funded by a grant from the RWJF.

Author disclosures: Drs Alexander, Beich, Christianson, Hasnain-Wynia, McHugh, Mittler, and Scanlon report no relationship or financial interest with any entity that would pose a conflict of interest with the subject matter of this article.

Authorship information: Concept and design (JAA, JBC, RH-W, MCM, JNM, DPS); acquisition of data (JB, JBC, RH-W, MCM, JNM, DPS); analysis and interpretation of data (JAA, JB, JBC, MCM, JNM, DPS); drafting of the manuscript (JAA, JBC, DPS); critical revision of the manuscript for important intellectual content (JAA, JB, JBC, RH-W, MCM, JNM, DPS); and obtaining funding (JAA, DPS).

Address correspondence to: Jeffrey A. Alexander, PhD, School of Public Health, 1415 Washington Heights, Ann Arbor, MI 48109. E-mail: jalexand@umich.edu.

1. Scanlon DP, Beich J, Alexander JA, et al. The Aligning Forces for Quality initiative: background and evolution from 2005 to 2012. Am J Manag Care. 2012;18(6):S115-S125.
2. Scanlon DP, Alexander JA, Beich J, et al. Evaluating a community- based program to improve healthcare quality: research design for the Aligning Forces for Quality initiative. Am J Manag Care. 2012;18(6):eS165-eS176.
3. Department of Health and Human Services. Report to Congress. National Strategy for Quality Improvement in Health Care. http://www.healthcare.gov/law/resources/reports/nationalqualitystrategy032011.pdf. Published March 2011. Accessed June 21, 2012.
4. Shih A, Davis K, Schoenbaum S, Gauthier A, Nuzum R, McCarthy D. Organizing the U.S. Health Care Delivery System for High Performance. http://www.commonwealthfund.org/~/media/Files/Publications/Fund%20Report/2008/Aug/Organizing%20the%20U%20S%20%20Health%20Care%20Delivery%20System%20for%20High%20Performance/Shih_organizingushltcaredeliverysys_1155%20pdf. Published August 2008. Accessed June 21, 2012.
5. Hasnain-Wynia R, Sofaer S, Bazzoli GJ, et al. Members’ perceptions of community care network partnerships’ effectiveness. Med Care Res Rev. 2003;60(4)(suppl):40S-62S.
6. Weiner BJ, Alexander JA. The challenges of governing public-private community health partnerships. Health Care Manage Rev.1998;23(2):39-55.
7. Hurley RE, Keenan PS, Martsolf GR, Maeng DD, Scanlon DP. Early experiences with consumer engagement initiatives to improve chronic care. Health Aff (Millwood) 2009;28(1): 277-283.
8. Christianson JB, Volmar K. Aggregating Data for Public Reporting: The Experience of Community Alliances. http://www.hhdev.psu.edu/cms/media/CHCPR/alignforce/files/Data Aggregation_ResearchSummary.pdf. Published April 2009. Accessed June 21, 2012.
9. Office of Behavioral and Social Sciences Research website. Dissemination and implementation. http://obssr.od.nih.gov/scientific_areas/translation/dissemination_and_implementation/index.aspx. Accessed June 21, 2012.

Download PDF: Midterm Observations and Recommendations From the Evaluation of the AF4Q Initiative