Foreign Policy Expertise Requires a Culture of Evidence
Foreign Policy Expertise Requires a Culture of Evidence
Fleshing out a vision of what a culture of foreign policy expertise might actually look like.
Thomas Scherer is a key contributor to this article; the text draws heavily from an article we coauthored called Foreign Policy Should be Evidence-Based.
I’ve spent the first handful of this publication focused on understanding the weaknesses of the status quo and laying a foundation for understanding expertise drawn from scholarship. It’s time to lay out a positive vision for building expertise in foreign policy!
As discussed in previous weeks, the two necessary ingredients for the development of expertise are a) receiving feedback about success and failure and b) deliberate practice. Said another way, our institutions need to produce evidence about the efficacy of their actions, and feed that evidence back into the behavior of its decision-makers.
Developing relatively accurate feedback about the impact of one’s actions is a key challenge for the development of expertise. Imagine practicing archery without being able to view where your arrows land. No expertise will form.
Unlike archery, evaluating success in the world of international affairs requires more than simply glancing down the range at your target.
My view is that our ability to cultivate foreign policy expertise is directly correlated with the quality of the evidence we can mobilize to improve our future decision-making. Asking our policymakers to rely entirely on their own instincts to evaluate the impact of their actions offers very weak feedback, undermining the formation of expertise.
For this reason, developing a culture of evidence-based policymaking is necessary for the development of foreign policy expertise. The goal of such a culture must be to improve the quality of expertise in foreign affairs (Reminder that I define expertise as “consistently superior performance,” not simply “knowing a lot of facts”).
What is Evidence-based Foreign Policy?
Evidence-based policymaking is:
“the process of using high-quality information to inform decisions that are made about government policies. It involves the systematic collection of high-quality data and analysis of those data with rigorous research methods, which creates evidence on which decisions can be based. Evidence can provide insights about how policies and programs operate, when and where they work effectively, or trends in performance over time. Evidence-based policymaking requires the use of evidence in the decision-making process, which means such evidence must be generated and available for policymakers to use.” (From the Bipartisan Policy Center)
One must be clear at the outset: Not all evidence is created equal. Arguments based on anecdotes, analogies, or naive statistics can be highly misleading or flat-out wrong. Too often, policymakers rush from the discovery of a few facts to forming a confidently held conclusion, or, worse yet, start with a conclusion and then go hunting for supporting evidence.
A major misconception about evidence-based foreign policy is that it should provide the “right” answers to the problems we face. It will not. It cannot. Anyone who tells you otherwise is trying to sell you something.
Instead, the goal of evidence-based foreign policy is to continually strive for better-informed answers. When policymakers demand transparency about the set of facts used to support recommendations and invite scrutiny about potential sources of bias, they will better understand the world. When decision-makers subject their ideas to evaluation — and have the courage to change their minds in the face of new evidence — they will make better policy.
Evidence-based policymaking demands a high standard. It directs our attention to the quality of the evidence used in the policy process. Evidence-based policymaking relies on the scientific method to subject its evidence and claims to constant scrutiny. The scientific method is straightforward: it asks us to think of our claims as hypotheses that require evaluation rather than as conclusions that need defending. This subtle shift of mindset is the core of evidence-based policymaking.
The Scientific Method is a Powerful Ally
Evidence needs to be curated at every stage of the decision-making process, not simply as a one-time input. Here I adapt the scientific method for the policy process:
The first two steps of the scientific method are things policymakers are already pretty good at: identifying the problem or opportunity and then generating hypotheses for how to respond (hypothesis is just a fancy word for a policy idea). A great way to do this in foreign policy is to read a lot, immerse oneself in a foreign culture, and stay curious about the future. One must carefully consider their values and worldview in order to understand what issues deserve attention.
The third, fourth, and fifth steps require one to sharpen their hypotheses by gathering evidence on all sides of the argument. This gathering of evidence should happen before, during, and after policy implementation. This is where the existing policymaking process so often falls short. Simply relying on intuitive judgment to evaluate the quality of policy ideas is a very weak approach. Overconfidence blinds decision-makers to alternative approaches, contrary evidence, and misplaced assumptions. When policymakers more rigorously evaluate their hypotheses, they will be more likely to achieve their goals and avoid preventable avoidable mistakes.
Great evidence can be garnered using a wide variety of methods and approaches — qualitative or quantitative, experimental or historical, causal or interpretive. Debates about the superiority of one method over another are misguided. A good analyst should pick the right tool for the job. The more interesting question is about how to build the analytical capabilities of our foreign policy workforce. Whereas intelligence analysts receive extensive training, almost nothing analogous is provided for the makers of foreign policy. My view is that methods training should be a lifelong pursuit for everyone in foreign policy.
There are many valid concerns about the shortcomings of evidence-based approaches. When I speak with policymakers, few lament the lack of demand for evidence in the current policy process. Many express valid concerns that existing evidence does not apply to their unique policy challenges, or fails to offer actionable recommendations. Others see evidence-based processes as too slow for the real world.
But skeptics miss the point. The question is not whether evidence should be used in the policy process — all policymakers use evidence — but rather how to continue to raise the bar on the quality of the American foreign policy process.
Coming Next: How to Build an Organization that Cultivates Expertise
Building the conditions for expertise to form in our foreign affairs bureaucracies will require a new culture committed to producing high-quality evidence at every stage of the policy process. This upgrade will require changes at both the individual and organizational levels.
To enhance expertise, the bureaucracy must be greater than the sum of its parts, not less. The process should seek to synthesize knowledge from across units, rather than relying on the fallible judgment of the highest-ranking official. This will not be an easy task — the U.S. State Department alone employs almost 70,000 people.
Yet, during my nearly two decades of work in foreign policy, I have witnessed how lip service paid to evidence-based policymaking is rarely supported by organizational and cultural changes to enable the effective use of evidence.
Oh how I wish there was an easy recipe for how to build stronger organizations! Modernizing government is challenging even under the best circumstances. This Substack is NOT intended to offer conclusive solutions. Instead, I hope this space will be an ongoing exploration of these vital questions and facilitate discussion with all of you about how to achieve our shared goals.