Child & Family Service's Evolving Journey to Outcomes

As I mentioned in my last post, measuring outcomes in theory can be quite different from measuring them in practice. Child & Family Service (CFS) is a Hawaii-based social service nonprofit that, in many ways, is “ahead of the curve” locally in its active embrace of performance assessment. Subsequently, CFS offers a unique case example for insights and lessons learned. Howard Garval, President and CEO of CFS, has championed more rigorous evaluation of social service programs for a numbers of years, and has “walked the walk” at his organization, leading a culture shift toward outcomes-based assessment. Recently, via email, I asked Howard about the changes he’s witnessed, the lessons he’s learned, and the insights he’s gained along the way. As he describes below, it’s a process that takes time, patience, and perseverance, but that is ultimately—and rightly—focused on improving outcomes for those served.

 

What specific changes or progress have you experienced in recent years at CFS that you attribute to the organization's embrace of an outcomes-based culture?

We have done two key things to move us to an outcomes-based culture:

(1) We adopted the Results-Based Accountability (RBA) model for program performance measures, developed by Mark Friedman of the Fiscal Policy Studies Institute and author of Trying Hard is Not Good Enough. RBA asks three primary questions: (a) How much did you do?—this is the outputs question; (b) How well did you do it?—this is the quality question; and (c) Is anyone better off?—this is the outcomes question. I have added a fourth question: How can we use the data to get better?

(2) We implemented Efforts to Outcomes (ETO)/Social Solutions electronic record software, which is the system Geoffrey Canada uses for the Harlem Children’s Zone that formed the basis for the Federal Promise Neighborhoods grant funding. Our staff has given ETO/Social Solutions rave reviews, and there has been practically zero resistance to implementation of the electronic record. Staff members have become more comfortable talking about data and outcomes. CFS has developed a common language as a result of RBA. Performance measures continue to evolve, and we keep drilling down to the question: “Is anyone better off?”

What has been most surprising about CFS' journey toward performance measurement over the past few years?

Human service professionals are in this field for their heart; in general, they did not come into this field because they liked data and measurement. However, a smart thing we did was create an agency-wide steering committee and coaches who have been both our cheerleaders and trainers to support our program staff to measure outcomes. I expected more resistance, and even though we have had some pockets of this, overall staff members have moved forward with the journey better than I anticipated. However, we have to keep motivating staff to see how performance measurement can help them improve their services to our clients. Once our direct service staff can see how the data and outcome measurement can help them deliver better services, we think their buy-in will be strengthened. Direct service staff in some programs have stepped up and taken some ownership, breaking down barriers to data collection and actively having a voice.

What has been the greatest challenge in becoming a high performing organization?

I think we have learned that it is better to get a program on ETO/Social Solutions software first, before doing the RBA work. For example, The Institute for Family Enrichment, or TIFFE, fully merged with CFS effective July 1, 2015. Based on what we’ve learned, we will be sure to move TIFFE programs onto ETO/Social Solutions first, then do the RBA work. The greatest challenge is to help human service professionals get comfortable thinking about outcome measures and asking the right questions about how we know a program is effective, and how we know that we are producing a measurable benefit for the people we are serving.

What suggestions would you share with smaller nonprofits that are interested in outcomes assessment, but that may not have the human or financial resources to fully invest in changing organizational culture?              

I think the RBA model has a simplicity to it that makes it accessible to smaller nonprofits. Through our new Institute for Training & Evaluation, CFS is now the only licensed RBA provider in the state of Hawaii. For a reasonable cost, we can assist organizations to learn and implement this model; it also comes with a RBA scorecard developed by the Results Leadership Group. Organizations could potentially pool resources to have us work with them. Also, “train the trainer” models could be a less expensive way to build capacity, as we did with our coaches. I would also say small organizations at least need to ask themselves, What would tell us that one of our programs is actually working and producing a measurable benefit for the people our program serves? It’s a process. It takes time, patience, and perseverance. It certainly helps if resources can be dedicated to move this forward.

What do you think being performance-based makes possible for your organization? For those you serve?

I think performance-based assessment positions us to be a leader in human services, and will enable us to garner additional support from funders and other donors. I think it will also help us sustain funding as funders will increasingly demand outcome measures that show their investment is producing impact. Using data to tell our direct service staff how we are doing and to demonstrate what is working will enable us to really do continuous quality improvement and get better at delivering services that work. In addition, our staff are becoming more aware of the impact they are having on program participants, and that in turn is improving the quality of their work.

For our clients, being performance-based has supported their individual journeys and self-awareness regarding the need for services. For example, one of our program’s PTSD “pre-tests” has led to clients recognizing their own need for counseling services. We can see that one of the benefits of our increased emphasis on outcome measurement is empowerment of our clients, who gain additional insights in tracking their personal growth.

 

Nonprofit leaders: Does your organization have experience with “walking the walk” of a performance-based culture? What successes and challenges have you encountered in creating that culture?

Funders: Is your foundation/philanthropy leading by example when it comes to being an outcomes-based organization? What unique challenges have you encountered in assessing the effectiveness of your philanthropic efforts?

Child & Family Service's "Leap" Toward Outcome-Based Assessment

Nonprofits are often looking for resources to start them on the road toward impact measurement. Moving beyond outputs (the count data, or answers to the "how much" question) to outcomes (the impact data, or answers to the "what meaningful changes did you effect" question) can be overwhelming, especially for smaller organizations. One of my go-to resources for nonprofits seeking guidance on performance assessment is the monograph Leap of Reason, which I describe below. And although I originally published this post in March 2012, it certainly bears revisiting--not only because outcomes-based assessment continues to be a "hot" topic in the nonprofit sector, but also because our understanding of what it takes to be an impactful organization continues to develop.

While talking about measuring outcomes is all well and fine in theory, it's helpful to know what it looks like in practice. Here in Hawaii, I sometimes hear from local nonprofits that few examples exist of organizations that really "get" performance assessment. That's why I'm happy to shine a spotlight on Child & Family Service (CFS), a well-known and respected local social service organization that can serve as a model for others. What is more, CFS has been open about its journey and continuing evolution in fully embracing an outcomes-based culture. I'll be sharing more about CFS' journey in my next post--stay tuned!

****************************************************************************************************************

Have you ever read a book that so clearly, concisely, and compellingly distilled an issue, you just felt the need to share it? Recently, I encountered such a book on outcomes-based management for the nonprofit sector, titled “Leap of Reason: Managing to Outcomes in an Era of Scarcity.” A monograph by Mario Morino, Chair of Venture Philanthropy Partners, Leap of Reason is a call to nonprofits to move toward the rigorous identification and measurement of outcomes to drive the impact of their work. Morino makes the case that in the current climate of tightened budgets and reduced funding from government and philanthropic sources, a paradigm shift toward meaningful, measurable impact is both necessary and desirable throughout the social sector.

I had the pleasure of being introduced to Leap of Reason by Howard Garval, President and CEO of Child and Family Service (CFS). Garval is a veteran nonprofit manager, having served for more than a decade as COO and CEO of The Village for Families and Children in Hartford, Connecticut, before his current tenure at CFS. During his time as an executive at The Village, Garval became familiar with Result-Based Accountability, a framework for producing measurable improvements in the public sector developed by Mark Friedman, Director of the Fiscal Policy Studies Institute. Impact assessment, then, is not a new concept for Garval; he has long embraced the idea of measurable outcomes at the organizations of which he’s been a part. As a result, “Leap of Reason really hit home,” Garval said. Last week, Garval and I had an opportunity to talk about the book, its resonance with his own experience in social services, and the ways in which outcomes-based management is helping shape the management and future direction of CFS.

Garval shared a few key take-aways:

1.    When we talk about outcomes, we are ultimately—and most importantly—talking about creating impact for those we serve. Garval pointed out that for him, the goal of improving CFS, its programs, and its operations is an intermediary step. His ultimate goal in utilizing outcomes-based management is to provide “evidence that we produce measurable benefit, and are truly making a difference [to those we serve].” Likewise, Morino states in Leap of Reason: “The greatest dividends [of managing to outcomes]…accrue to the communities, the families, and the individuals with whom we work. They benefit from stronger schools, smarter clinics, and safer communities—all because of nonprofits’ commitment to becoming better.”

2.     While it’s important for nonprofit leaders to “buy-in” to a performance culture, a top-down approach alone won’t ensure meaningful changes within an organization. Garval noted that in his experience, having direct line staff that subscribe to a culture of measurable impact is just as important as having leadership that does the same. Sometimes, he said, “line staff have the best ideas for producing [measurable] benefits,” precisely because of their direct contact with the individuals and families served. Garval further stated that identifying staff who are “early adopters” of an outcomes approach is helpful in engaging staff overall, since peer-to-peer engagement may be a stronger influence than that exerted by an organization’s leaders. Morino makes this same point as well: “Leaders can’t simply create by edict the organizational cultures they desire.”

3.    Identifying the right questions to ask is challenging, but critical to an organization’s work. Garval described taking part in a recent CFS leadership training in which the group reviewed the organization’s outcomes for five core service areas. Looking critically at the outcomes was “the best part of the training,” said Garval. “We drilled down deeper into our measures to [examine if] we are measuring the right stuff.” The exercise, however, sometimes led to more questions than answers: Are we measuring what’s most important? How are we using the information we collect to continually improve our services? Are we collecting outcome data that will ultimately strengthen programs and, consequently, be most beneficial to those we serve? In Leap of Reason, Morino states the challenge this way: “…With all the rhetoric around mission, scaling, accountability and the like, the reality is that we often have to go back to basics and ask, ‘To what end?’ Defining an organization’s true purpose is absolutely essential to cultivating a performance culture.”

4.    Better outcome measurement may have negative short-term implications, but it’s a crucial investment in long-term improvement. Garval described another CFS leadership training exercise that involved identifying forces that support and restrain the organization’s increasing shift toward a performance culture. The worry that better outcome measurement may initially mean less impressive results for an organization was named as a restraining force, and is certainly a valid concern. Yet, Garval noted this is a concern that must be overcome, because the collection of data on baseline performance and subsequent goals for the organization’s improvement are what will allow CFS to identify and maximize its impact on those it serves. Morino recognizes this challenge—and opportunity—as well: “…The transition to outcomes-oriented management will almost certainly have some negative near-term implications for the organization. These changes, though, will just as certainly have a positive impact for the nonprofit in the long run as it becomes more effective in achieving its mission.”

Nonprofits leaders: Does your organization employ an outcome-oriented approach to its work? How has this approach influenced the management and impact of your organization?

Funders: To what extent have your funding decisions been driven by nonprofits’ outcome performance? How has your funding organization supported nonprofits’ efforts to improve their impact in measurable ways?