Metrics of the Trade

The art and science of quantifying how The Financial Clinic measures its mission has been a prominent feature of our history. From our inception 10 years ago, when it was critical that we cut through a lot of rhetoric about what it meant to build financial security for our customers, we developed six discrete and measurable outcomes that together, defined our mission.

The advantages were that our six outcomes were organically developed from our own successes and failures, and with minor exceptions, they’ve largely withstood the test of time. Quantifying success served to enlist new stakeholders as well. The drawbacks to the Clinic’s outcomes framework is that after 10 years, we’re feeling that one of the best indicators of success — savings, because it gauges our customers’ future orientation — gets lost in the mix. Another example is credit scores, and how hard it is to attach your mission to what is, basically, a Coca-Cola formula.

It was relatively easy to put aside the inputs and outputs of financial education: the Clinic is much more concerned with being measured by what our customers accomplished with the information. Likewise, we originally rejected purely qualitative measures like customers’ self-reported feelings or attitudes around money. While these were prevalent measures at the time, they can be manipulated by a favorite coach and can’t be standardized. The Clinic also developed a monetary quantification of our mission that we call the cash value of our work, which is helpful in an ROI analysis, but it isn’t a full picture of how we improve our customers’ financial mobility.

Recently I was able to share some of this history with hundreds of practitioners and funders when the Asset Funders Network (AFN) — a membership organization of grant makers focused on promoting economic opportunity — graciously invited the Clinic to be on a panel to discuss a new metric: The Financial Capability Scale (FCS).

Developed by J. Michael Collins from the Center for Financial Security at the University of Wisconsin with the leadership and support of the Annie E. Casey Foundation, the FCS seeks to standardize how we gauge the success of the maturing field of financial coaching.

In 2013, the Clinic enthusiastically and readily jumped right into a pilot to test the FCS. In pursuit of the right measurement for our mission, we’ve been involved in other studies, including NeighborWorks’ Success Measures and CFED/CFPB’s Consumer Financial Well Being Metrics Project. In the end, despite avoiding self-reported qualitative outcomes early on, the FCS demonstrated to us that it strikes the right balance of how the Clinic defines success because it has elements of our outcomes framework combined with self-reported financial well being and other qualitative information.

For a description of how the Clinic operationalized the FCS and standardized it in Change Machine — our financial coaching platform–you are invited to check out AFN’s webinar recording.

The Clinic’s primary conclusion from the pilot was that the FCS is complementary to our mission. First we examined the follow up surveys (6 and 12 months after the original survey), which revealed a 1.4 point average increase after working on customers’ financial security for at least 6 months. Next we observed that the scale confirmed much of what we knew anecdotally to be true of our customers:

  • Greater financial insecurity manifested itself in a low score in almost every area of the scale;
  • Women are more adept at budgeting and savings, but their lower income also meant a lower score; and
  • Average scores increase across demographic areas such as ethnicity, race and education level confirming the Clinic’s belief that our mission is broadly applicable to people in crisis.

metrics

Most critical for the Clinic was documenting a correlation between FCS score changes and outcomes achievement:

  • As the chart shows, changes in the score were positively correlated with the Clinic’s six financial security outcomes, illustrating that as our customers improve their credit scores and lower their debt, it will be reflected in the follow-up FCS survey; and
  • On average, customers improved in all the areas reflected by the question, indicating that the FCS is capturing the breadth of financial security and the coaching process.

These observations have allowed us to conclude that the FCS rounds out the Clinic’s own quantifiable financial metrics, but not in a way that relies solely on qualitative assessments. As with all the measurement projects the Clinic has participated in over the years, we are pleased to be contributing to the discussion about what works.

Just as the Clinic was driven to develop our own outcomes framework a decade ago, I believe it’s important for our growing field to coalesce around common outcomes: united voice holds tremendous promise for conveying to the rest of the world how powerful and impactful this work can be for the communities we serve.

Type of Clinic MetricExampleWhere to Find More Info
Financial Security Action and StrategiesImportance of Emergency SavingsChange Machine
Financial Security OutcomesA Customer Who Makes Three Successive Deposits Into a Savings AccountA History of the Clinic Through How It Defines Its Mission
Faster, Better, Cheaper OutcomesWorkforce Development Customers Who Worked with Career Coaches to Download and Work on Credit Scores Secured 12 Additional Hours of Work Per WeekScaling Financial Development
Cash ValueSavings Increases Trigger Cash Value, So Going from $100 to $200 in an Account, For Example, Results in a Cash Value of $100Cash Value Report