Measure what matters — not just what is easy to measure

Dr Andrew Bastawrous, CEO of Peek Vision and Associate Professor of International Eye Health at London School of Hygiene & Tropical Medicine

Originally published for Devex (March 2020)

What we measure really matters. But what matters most can often be the most difficult to measure. 

In the midst of the COVID-19 pandemic, we are seeing how crucial accurate, reliable, and timely data is to meeting people’s health needs. COVID-19 is set to have a devastating effect in many countries, especially those where health care systems are less well established or vulnerable to sudden shocks. Without concerted global action, there is a risk that there are ripple effects for many years beyond the pandemic itself. The result could be even bigger backlogs of people needing treatment and health services that become further out of reach for the most vulnerable.  

To achieve universal health coverage, we need to make health systems stronger, so that everyone who needs treatment can get it. Doing so will prepare us better for future health crises and ensure that the people most severely affected can access the services they need.

The problem? Measuring progress towards UHC is a major challenge.  

However, there is one critical objective that ultimately aligns the missions of all the organizations, individuals, and people striving to achieve health care for everyone: to increase the proportion of people who have their health care needs met.

Using metrics to improve vision

For example, a staggering number of people worldwide cannot see because they have cataracts. Until they are located by health services and treated, these people have an unmet need for health care. Once they have received surgery and regained their vision, their need has been met.

This may seem simple — obvious, even — but for most global health programs, the metrics which are most often reported only tell part of the story. 

Most programs are held accountable to, and therefore report on, their activities — for example, how many people were screened or treated. A huge amount of work is involved in running large-scale screening or treatment campaigns and that needs to be recognized. But if these are the only numbers we monitor, we could be missing some critical issues.  

Across the world, certain groups of people — such as women, people with disabilities or indigenous people — are particularly vulnerable to being left behind. If we only measure success through the numbers screened or treated, this inequity risks remaining completely invisible. Put simply, we cannot hope to achieve UHC if we don’t know who we’re missing.

Know where the problem is

I know from experience that running and reporting on health programs in low-resource settings is hard, so you might ask how we can justify adding more data collection and reporting.

One solution is to maximise the growing opportunities offered by digital and mobile health.  That is why it is so important to work with eye health NGOs and governments to develop new tools which make understanding progress towards UHC — met need — easy and effective. A recent example from a low-income country Peek Vision have been working in illustrates the point perfectly.  

Local eye health organizations had been running eye health screening programs for years. Thanks to their hard work and skill, they had plenty of good data on how many people had been screened and treated for eye problems. 

However, to understand whether they were truly meeting the eye health needs of their area, the team needed to know the extent of the overall problem and how long it would take to solve with current resources and programming. The answer was another 30 years, in which time the problem would grown faster than it was being solved. 

What looked like a success story — thousands screened and hundreds treated — was in fact masking a very low proportion of people having their eye health needs met.

Working in partnership with CBM international and local eye health teams, we implemented smartphone-based vision screening integrated with digital eye health systems so they could reach more people and capture eye health data from screening to treatment. These data allowed the local eye health teams to understand why some people didn’t have their needs met. They adapted their approach to address the issues they surfaced. The result was that they almost doubled the proportion of people whose eye health needs were met within a few months. Moreover, they now know exactly where they need to focus next to make even bigger improvements.  

Measuring what matters for systemic change

There is a positive and growing movement around equity in health and social care, but recognizing the problem is only the first step. It is essential that everyone working in global health has the tools and data they need to ensure no one is left behind. 

Continuing to neglect this issue will lead to increasing inequity, greater health service instability, and future generations having to grapple with problems that we could have dealt with. If we want to see systemic change, we have to focus on measuring what matters, not what is easy to measure.  

Health providers around the world are working tirelessly to deliver change and bring health care to everyone who needs it. By focusing on what is easiest to measure, however, there is a risk that organizations on the ground are unintentionally being asked to choose between serving the needs of their patients and the needs of their reporting requirements.  

These kinds of digital health revolutions means that global health organizations now have the capability to measure what matters and make their services responsive and data-driven. The result is that those who need our collective efforts the most will reap the benefits. Small steps will soon add up to big changes, provided we can see where we are going.