The US spends billions on foreign aid. But it doesn’t know how much good our money is doing.

In 2021, USAID — the agency of the US government tasked with international development — disbursed $28.3 billion in foreign aid to an assortment of humanitarian causes ranging from hunger programs to medical treatment to education.

But how much good is this money doing? And is that money accomplishing as much as it could be?

The answer, it turns out, is much more difficult to find than you would think — and that’s a problem. USAID is one of the most consequential institutions in the world when it comes to aid for the poor.

Since the world’s resources aren’t unlimited, we have a “moral imperative to use evidence and data to ensure we get the most impact per dollar spent as possible,” says Anne Healy, former head of USAID’s Development Innovation Ventures.

Over the past two decades, researchers have become much better at determining whether a certain idea actually achieves intended goals. The focus on results — evaluating whether a program benefits people cost-effectively — has changed philanthropy and even the US government’s domestic programs.

In theory, USAID recognizes the importance of making sure their programs work. But in practice, it’s largely failing to do so.

Two USAID reviews, one by USAID’s office of the inspector general in 2019 and another commissioned by the agency in 2020, reveal two dismal facts: The agency gives out billions to programs that don’t achieve their intended expectations, and, worse, it’s not even sure of the impact of most of the money it gives in aid. Recent agency moves and statements suggest that USAID wants to fix this problem. Whether it can will determine the fate of billions of dollars — and the health and well-being of many millions around the world.

How USAID works

Since 1961, USAID has poured hundreds of billions of dollars into foreign aid, with the aim of delivering humanitarian assistance to millions globally. Foreign aid accounts for less than 1 percent of the US federal budget — far less than what most Americans think it does. But because of the federal budget’s massive size, even 1 percent is much larger than all of private philanthropy for global development in a given year combined. USAID spends tens of billions of dollars a year on global development programs, the largest categories of which are health, humanitarian assistance, and economic development.

The agency, which works in over 100 countries, usually doesn’t implement programs directly, but partners with different organizations, including NGOs, universities, and faith-based and community groups. Its yearly grants and contracts comprised on average almost $18 billion over the last decade.

When looking into USAID’s effectiveness, it’s important to note that other US government priorities will influence how money is allocated even before USAID itself can make any decisions. As an example, USAID presents its budget requests within five strategic foreign aid objectives developed by the State Department: humanitarian assistance, peace and security, democratic governance, economic growth, and social services.

Ben Curtis/AP

Foreign aid has of course been deployed by US administrations to advance their geopolitical goals. But that complicated record doesn’t mean aid can’t do a lot of good. USAID has been a major contributor to polio eradication in over 100 countries. PEPFAR, the US government’s anti-HIV/AIDS initiative, has led to an estimated 20 percent lower mortality rate in countries that received its aid, and has saved millions of lives. USAID’s Development Innovation Ventures, which funds innovative projects around the world, has funded a handful of highly cost-effective programs in global health and education. USAID has contributed to many other effective global health programs, including developing meningitis vaccines that prevented an estimated 1 million cases.

US foreign aid isn’t working as well as it should

But those success stories can obscure an uncomfortable truth: We don’t really know whether most of US foreign aid is improving anyone’s lives.

To understand why this is, let’s look at USAID’s own evaluations of its programs. In-house reports on the impact of USAID programs abide by the usual academic standards — they need adequate sample sizes and valid control groups, among other criteria. The agency uses a checklist to monitor whether each impact evaluation meets these different criteria, and gives them a quality designation. These evaluations are only one way USAID monitors performance — for other programs, they instead monitor processes or use qualitative work — and they are meant to assess whether or not USAID-funded programs are achieving milestones, such as lowering malnutrition.

But to prove success, the evaluations have to be high-quality, and most of them are not.

“USAID is failing to generate rigorous evidence on which of its programs do or do not work,” wrote three former USAID administrators in an article for the Wilson Center in 2021.

For one thing, USAID’s own efforts to assess its programs’ impact leave a lot to be desired. Most of the agency’s impact evaluation reports are not high or even acceptable quality by the agency’s own standards of rigor.

The aforementioned internal review from 2020 revealed that most USAID-conducted impact evaluations of programs didn’t include one or more key quality elements like sample size, research/evaluation hypotheses, missing data, and other key components to understanding whether the results of an evaluation should be accepted as valid or not.

46 percent of the reports either didn’t have a comparison or control group, or didn’t provide enough statistics on a control group to be accurate.

Only 3 percent met USAID’s highest standards of quality. A bad impact evaluation is a waste of money, and it can even lead to funding going to ineffective programs.

USAID also seemingly keeps paying out contracts to projects that don’t even work at a most basic level. A 2019 study by USAID’s inspector general of 81 USAID grants found that over 40 percent of programs achieved only half of expectations, which meant they self-reported that they didn’t achieve much of what they’d been paid to do by the grant.

The inspector general’s report outlined major concerns with even the awards that did achieve results. For example, one program reported achieving 110 percent of expected results for preventing and managing malnutrition in West Africa. But this was only because they were measuring radio outreach — people who heard about the program on the radio — as a “success”: Most people were not actually receiving malnutrition services, which was the real goal.

According to a USAID spokesperson, the agency has begun “addressing many of the gaps and shortcomings identified” in the 2019 report, as well as some of the recommendations from the 2020 report, including updating its impact evaluation guidance and requiring cost analysis in impact evaluations.

Additionally, USAID is not using outside evidence in the way it could be. While USAID has standards and processes for conducting evaluations, it has fewer processes to ensure evidence from elsewhere is being used, experts told me.

Jekesai Njikizana/AFP via Getty Images

For example, let’s say a university study finds strong evidence that a certain approach to reducing childhood malnutrition is cost-effective. USAID could do more to consider this approach, even if it’s not research it conducted itself.

USAID doesn’t have a monopoly on finding evidence for program effectiveness. There are research institutions, think tanks, and policy organizations in the countries in which USAID works. Having a more systematic way to compile, outsource, and use the evaluations of entities that are already working in relevant areas would help make sure that program and funding decisions at USAID are supported by the best available evidence, said Healy.

USAID is potentially missing out on funding many effective organizations

Zooming out a bit, a systemic problem that likely contributes to USAID’s ineffectiveness is the way it doles out grants.

Unfortunately, the way USAID’s grant structures are set up now means there’s not much incentive for contractors to produce results. The most common form of USAID grants are what’s known as cost-plus grants, which basically means a contractor draws up a list of their expected costs and USAID pays them — regardless of whether they achieve results.

An alternative form of grant, fixed-amount grants, pay contractors when they achieve predetermined milestones and results. These are better, but they’re not yet widely deployed in government grantmaking. USAID deems fixed-amount awards most appropriate when the work has milestones that can be priced with reasonable certainty. USAID might not use them when a project lacks this information, and they also require ceding some direct government oversight of grants.

The other problem with the USAID grants process is that it’s so complicated to navigate that legacy government contractors who know how to write grant applications have a major edge, experts told me. (To be sure, these problems exist across international granting organizations, and both small organizations and USAID administrators have acknowledged the high barriers to entry and importance of greater inclusion.)

Eliya Zulu, executive director of the African Institute for Development Policy, a research and policy organization based in Kenya and Malawi, described the process of putting together a successful USAID bid for his organization as a “huge nightmare.” The process included overtime work, over 150 support documents, and staffing that smaller organizations simply don’t have. A lot of legacy organizations have business development units focused on such tasks, he said, while a worthy but smaller organization might not have the same support.

This leads to a situation where the vast majority of USAID money is going to only 75 organizations, and only 6 percent of grants are given to organizations based in USAID-recipient countries. While legacy contractors aren’t inherently ineffective, the complicated process means smaller organizations, especially those based in the Global South, are often left out of awards, when even a small grant could make a big difference.

This means thousands of innovative Global South-led and -based organizations — groups which may be more effective because they understand local context better and interact with local policy actors to make sure effective programs continue after USAID leaves — are not receiving funding because of bureaucratic issues.

USAID recognizes that this way of doing business is a problem. During USAID’s annual small business conference last year, USAID administrator Samantha Power stated how the limited number of contractors “holds back healthy competition, limits our exposure to new approaches, robs small businesses of the chance to gain valuable experience, and doesn’t make the best possible use of valuable taxpayer dollars.”

Altaf Qadri/AP

The most effective aid, said Zulu, will be evidence-backed and in equitable partnership with governments and organizations that ensure it’s focused on the needs of the people it is going to — and that’s not what’s happening now.

Signs of a shift

The shift that needs to happen at USAID is so simple it seems silly to say out loud: The agency should fund things that are proven to work, and stop funding things that are proven not to work. But saying it is one thing. Doing it is another.

One thing USAID could do is focus on the evidence for different sectors about effective uses of money, said Ruth Levine, CEO of IDinsight, a global development data analytics and advisory organization. “Really importantly, what we have learned about things that absolutely do not work, don’t do those again.” (Disclosure: I worked at IDinsight from 2017 to 2020.)

A good start would be revisiting its process of awarding grants.

Experts told me a way to improve USAID’s record is to give out more fixed-amount awards. These pay contractors when they reach pre-negotiated milestones, meaning they’re more likely to pay for outcomes and results than other types of grant.

Fixed-amount awards currently account for only about 8 percent of USAID’s grants, but could and hopefully will be expanded — in March, a senior official announced plans for more fixed-amount awards and work with more contractors from the Global South. There’s also a lot of flexibility in how they’re implemented. They could potentially, for example, have components that pay for results, but also account for startup costs for a newer organization.

Walter Kerr, the director of Unlock Aid, a global development innovation coalition, noted that in addition to incentivizing based on outcomes and results, these awards are a “great way to mitigate against concerns that some members of Congress have around fraud, waste, and abuse because you only pay for what you get.”

There’s also awarding more money to smaller organizations and those based in the Global South. There have been green shoots here: The New Partnerships Initiative, USAID’s plan to diversify its partners, has awarded hundreds of millions of dollars to “new and underutilized partners” since it began in 2019. Meanwhile, in an exchange at a Senate Foreign Relations Committee meeting in May, Power reiterated USAID’s goals to reduce administrative burden in granting and send 25 percent of foreign assistance to local organizations.

Related to such a reform could be a reorientation toward doling out more direct grants to governments instead of middlemen. Governments have more mechanisms in place than an outside contractor for identifying their problems, finding the people who need help, and continuing programs after USAID leaves. The government actually provides services; it can often procure, for example, health equipment more cost-effectively than USAID buying it.

But less than 4 percent of US foreign aid is channeled through governments. Compare that with a country like Japan, which channels nearly half of its foreign aid this way. In the few cases where bilateral government aid has been tried by the US, it has been effective.

The US has existing grant mechanisms it could expand to increase direct bilateral aid, including the Economic Support Fund, which is used to provide money directly to countries of strategic significance. Experts told me, however, that reforming aid to go directly to governments would be a heavier and longer-term lift than, for example, more fixed-amount awards or support for local NGOs.

Hoshang Hashimi/AFP via Getty Images

Beyond the mechanics of grant-making, USAID could look to the example of the UK’s Foreign, Commonwealth, & Development Office, which has an empowered chief economist and an office that conducts independent reviews of evidence for large spending decisions, and then presents recommendations to senior policymakers. USAID has already shown signs of moving in this direction, such as Power’s announcement last year to start an expanded chief economist office and a behavioral science unit.

And there’s work already happening within the agency that USAID can foreground and scale up. The Development Innovation Ventures office (DIV), for instance, has been a promising testing ground for funding effective programs. DIV invests in potentially high-impact projects, looks for evidence of impact, and pays for results. It has funded new, Global South-based partners, and has funded interventions that have proven to be highly cost-effective at preventing childhood diarrhea, reducing road deaths and injuries, and more.

DIV accounts for only about 0.1 percent of USAID’s budget, said Healy, who was a leader of DIV, but that belies the potential impact of adopting some of its approaches. “The real opportunity for DIV,” Healy told me, “is influencing the 99.9 percent of USAID’s other spending.”

These moves are promising signals of a shift away from business as usual and an embrace of more evidence-based approaches. Which is good, because the time for change is long overdue. USAID has for years identified its own need for reform, but little change has happened. Evidence has told us so much more about how to help the world’s neediest. It’s time for that attitude to sweep through the halls of American diplomacy.

Will you support Vox today?

We believe that everyone deserves to understand the world that they live in. That kind of knowledge helps create better citizens, neighbors, friends, parents, and stewards of this planet. Producing deeply researched, explanatory journalism takes resources. You can support this mission by making a financial gift to Vox today. Will you join us?

One-Time Monthly Annual

$5/month

$10/month

$25/month

$50/month

Other

$

Yes, I'll give $5/month

Yes, I'll give $5/month

We accept credit card, Apple Pay, and Google Pay. You can also contribute via