As I mentioned in my last post, measuring outcomes in theory can be quite different from measuring them in practice. Child & Family Service (CFS) is a Hawaii-based social service nonprofit that, in many ways, is “ahead of the curve” locally in its active embrace of performance assessment. Subsequently, CFS offers a unique case example for insights and lessons learned. Howard Garval, President and CEO of CFS, has championed more rigorous evaluation of social service programs for a numbers of years, and has “walked the walk” at his organization, leading a culture shift toward outcomes-based assessment. Recently, via email, I asked Howard about the changes he’s witnessed, the lessons he’s learned, and the insights he’s gained along the way. As he describes below, it’s a process that takes time, patience, and perseverance, but that is ultimately—and rightly—focused on improving outcomes for those served.
What specific changes or progress have you experienced in recent years at CFS that you attribute to the organization's embrace of an outcomes-based culture?
We have done two key things to move us to an outcomes-based culture:
(1) We adopted the Results-Based Accountability (RBA) model for program performance measures, developed by Mark Friedman of the Fiscal Policy Studies Institute and author of Trying Hard is Not Good Enough. RBA asks three primary questions: (a) How much did you do?—this is the outputs question; (b) How well did you do it?—this is the quality question; and (c) Is anyone better off?—this is the outcomes question. I have added a fourth question: How can we use the data to get better?
(2) We implemented Efforts to Outcomes (ETO)/Social Solutions electronic record software, which is the system Geoffrey Canada uses for the Harlem Children’s Zone that formed the basis for the Federal Promise Neighborhoods grant funding. Our staff has given ETO/Social Solutions rave reviews, and there has been practically zero resistance to implementation of the electronic record. Staff members have become more comfortable talking about data and outcomes. CFS has developed a common language as a result of RBA. Performance measures continue to evolve, and we keep drilling down to the question: “Is anyone better off?”
What has been most surprising about CFS' journey toward performance measurement over the past few years?
Human service professionals are in this field for their heart; in general, they did not come into this field because they liked data and measurement. However, a smart thing we did was create an agency-wide steering committee and coaches who have been both our cheerleaders and trainers to support our program staff to measure outcomes. I expected more resistance, and even though we have had some pockets of this, overall staff members have moved forward with the journey better than I anticipated. However, we have to keep motivating staff to see how performance measurement can help them improve their services to our clients. Once our direct service staff can see how the data and outcome measurement can help them deliver better services, we think their buy-in will be strengthened. Direct service staff in some programs have stepped up and taken some ownership, breaking down barriers to data collection and actively having a voice.
What has been the greatest challenge in becoming a high performing organization?
I think we have learned that it is better to get a program on ETO/Social Solutions software first, before doing the RBA work. For example, The Institute for Family Enrichment, or TIFFE, fully merged with CFS effective July 1, 2015. Based on what we’ve learned, we will be sure to move TIFFE programs onto ETO/Social Solutions first, then do the RBA work. The greatest challenge is to help human service professionals get comfortable thinking about outcome measures and asking the right questions about how we know a program is effective, and how we know that we are producing a measurable benefit for the people we are serving.
What suggestions would you share with smaller nonprofits that are interested in outcomes assessment, but that may not have the human or financial resources to fully invest in changing organizational culture?
I think the RBA model has a simplicity to it that makes it accessible to smaller nonprofits. Through our new Institute for Training & Evaluation, CFS is now the only licensed RBA provider in the state of Hawaii. For a reasonable cost, we can assist organizations to learn and implement this model; it also comes with a RBA scorecard developed by the Results Leadership Group. Organizations could potentially pool resources to have us work with them. Also, “train the trainer” models could be a less expensive way to build capacity, as we did with our coaches. I would also say small organizations at least need to ask themselves, What would tell us that one of our programs is actually working and producing a measurable benefit for the people our program serves? It’s a process. It takes time, patience, and perseverance. It certainly helps if resources can be dedicated to move this forward.
What do you think being performance-based makes possible for your organization? For those you serve?
I think performance-based assessment positions us to be a leader in human services, and will enable us to garner additional support from funders and other donors. I think it will also help us sustain funding as funders will increasingly demand outcome measures that show their investment is producing impact. Using data to tell our direct service staff how we are doing and to demonstrate what is working will enable us to really do continuous quality improvement and get better at delivering services that work. In addition, our staff are becoming more aware of the impact they are having on program participants, and that in turn is improving the quality of their work.
For our clients, being performance-based has supported their individual journeys and self-awareness regarding the need for services. For example, one of our program’s PTSD “pre-tests” has led to clients recognizing their own need for counseling services. We can see that one of the benefits of our increased emphasis on outcome measurement is empowerment of our clients, who gain additional insights in tracking their personal growth.
Nonprofit leaders: Does your organization have experience with “walking the walk” of a performance-based culture? What successes and challenges have you encountered in creating that culture?
Funders: Is your foundation/philanthropy leading by example when it comes to being an outcomes-based organization? What unique challenges have you encountered in assessing the effectiveness of your philanthropic efforts?