Dan, you may want to look at OIG reports from prior years. In 2014 I led an OIG study for the U.S. Department of State on how to measure and evaluate the effectiveness of diplomacy itself. That is, work done by economic and political officers.
Dan, as always, you have excellent questions. However, I suggest you are confusing (perhaps conflating) "strategy" with "tactics" and "implementation." Unless I missed something, the point of your piece is to help evaluate and improve how the State Departments implement a piece of some so-called "strategy." Except, as your narration indicates, the "strategy" in question is a bunch of tactical efforts that may or may not be synchronized with policy initiatives and actions over time by other elements of the government and work with partners.
You bring up the private sector's tolerance for failure in certain areas is helpful, to a point, but you fail to carry that forward to where you need for this discussion. In your first anecdote, you claimed the "project was well designed," so what? The design does not equal effectiveness, just as smartly designed medicine or software may fail in contact with the real world. Was "flooding" the zone effective? And, by the way, the program you mentioned as "going smoothly" was not, and this wasn't realized until the survey indicated a design failure (or, best, case adversarial activity that undermined the program's potential, which is still a failure since that action wasn't detected if it happened), highlighting the failure, in this part of the narrative, was tactical rather than with strategy.
Strategy and tactics are slippery concepts. A strategy at one level of analysis is a tactic for another level of analysis. Good luck finding an authoritative definition for those terms! Semantics aside, the question at hand is how the State Department can understand the impact of its most important efforts and increase its effectiveness. My point is that this requires focusing on a higher level of intervention --the policy and strategy level. It's not clear from your comments whether you think this is foolish or wise. Would you like to clarify?
No, they aren't slippery concepts. The use of the words is slippery and sloppy. You're talking about a strategy to implement tactical plans, but that doesn't make it a strategy as in "US Foreign Policy Strategy." Semantics shouldn't be put aside because they help inform your query. The State Department's efforts can be tactical solutions lacking strategic support, and if so, should be judged accordingly. Sometimes you will need good tactics to compensate for a bad or absent strategy.
I called out an example in your essay that suggested a tactical failure, or possibly several points of failure. Your narrative jumped to blaming something else ("was well designed") a possible lack of an over-arching strategy to align policies, actions, and words across USG and supporting entities, possibly including other State efforts.
"Monitoring and evaluation" are a basic requirement. It's malpractice not to have them, but their absence is a default setting for too many offices and efforts. As a result, we have fire-and-forget "well-designed" programs that may fail to achieve the desired results—and potentially backfire—with little awareness until it's too late.
If you don't like "fire-and-forget", "thrown over the transom" is another phrase that reflects the lack of immediate, relevant, and as timely as possible inquiry into how an effort is going, what it's achieving and why and with whom.
But the "m&e" piece is only one of the missing and relegated puzzle pieces. I pointed out in my prior comment that tolerating failure was a core feature of your private sector examples. This must be highlighted because the lack of this tolerance has a negative effect on pace, flexibility, creativity, and impact of efforts. It can, as you will know from your experience, prevent ideas from being hashed out or tested.
It seems like we agree on the central point that M&E, or whatever one wants to call it, is essential. And that good strategy, tactics, and operationalization are also essential and interrelated. Perhaps I'm putting words in your mouth here, but the three concepts all depend on one another: It's a recipe for failure if one develops tactics without strategy. And it's a recipe for failure if one executes (operationalizing) tactics without M&E (feedback) -- "fire and forget," as you suggest.
"Can We Monitor and Evaluate US Foreign Policy Strategy?" The best foreign policy question of 2024!
Here’s one for 2025. Can we monitor and evaluate VOA messaging of foreign audiences?
Judging by the 18 years of decline in freedom around the world, as documented by Freedom House, one could conclude our messaging about the values of freedom and the rule of law is not as effective as the counter-messaging of the bad guys.
Focussing on VOA, here’s what’s wrong in a nutshell. IMHO, VOA news in a world awash in news is not winning the information war. What’s missing from our messaging arsenal is advocacy, the marshaling of fact and truth to persuade. News is not advocacy. An attorney doesn’t read the day’s news to jurors. An attorney wins a case by persuading..
We need both news and advocacy. But VOA has evermore morphed into a news agency. It’s only advocacy is boring, milquetoast editorials.
Our adversaries' messaging has a strong point of view. Objective news is not supposed to have a point of view. That's why we need advocacy.
The solution is not more Michael Packs or Cari Lakes thrashing around in VOA. The solution is the creation of a separate agency to produce advocacy in all its creative and imaginative forms, including music and humor. Let VOA produce news. Let the new agency produce advocacy that puts us on the offense in the information war. It's time to get "them" before they get us.
Back to your theme, monitoring and evaluation. You could do Public Diplomacy a huge service by determining to what extent and in what manner VOA uses M&E to determine the effectiveness of its content. Judging by VOA's dismal performance in the info war, one doubts whether the tool is used and, if so, to what extent the information is acted upon to improve program content.
Regards,
Gordon Humphrey
Former U.S. Senator
Producer of Russian-language YouTube channel Nashi Emigranti and its English-language counterpart, Let’s Tell America’s Story to the World
Thanks for these suggestions. I think you're right: given the trends in US foreign policy, it's hard to argue that our tools are "working" (though it's theoretically possible that are tools are working, but washed out by much larger systemic forces beyond the scope of relatively small government tools). I'll say the public diplomacy folks at the State are investing heavily in M&E, though I'm not sure whether those efforts have filtered into the VOA or other news agencies. Either way, we're still operating in a culture that has a very difficult time admitting failure. Without the ability to learn and change course, any investment into M&E is a waste. Program administrators need the space to experiment, fail, and try something new.
Dan, you may want to look at OIG reports from prior years. In 2014 I led an OIG study for the U.S. Department of State on how to measure and evaluate the effectiveness of diplomacy itself. That is, work done by economic and political officers.
I rigorously read OIG reports and am a huge fan of the model. Not 100% sure if I've seen this one though. Can you link it for me?
Dan, as always, you have excellent questions. However, I suggest you are confusing (perhaps conflating) "strategy" with "tactics" and "implementation." Unless I missed something, the point of your piece is to help evaluate and improve how the State Departments implement a piece of some so-called "strategy." Except, as your narration indicates, the "strategy" in question is a bunch of tactical efforts that may or may not be synchronized with policy initiatives and actions over time by other elements of the government and work with partners.
You bring up the private sector's tolerance for failure in certain areas is helpful, to a point, but you fail to carry that forward to where you need for this discussion. In your first anecdote, you claimed the "project was well designed," so what? The design does not equal effectiveness, just as smartly designed medicine or software may fail in contact with the real world. Was "flooding" the zone effective? And, by the way, the program you mentioned as "going smoothly" was not, and this wasn't realized until the survey indicated a design failure (or, best, case adversarial activity that undermined the program's potential, which is still a failure since that action wasn't detected if it happened), highlighting the failure, in this part of the narrative, was tactical rather than with strategy.
Strategy and tactics are slippery concepts. A strategy at one level of analysis is a tactic for another level of analysis. Good luck finding an authoritative definition for those terms! Semantics aside, the question at hand is how the State Department can understand the impact of its most important efforts and increase its effectiveness. My point is that this requires focusing on a higher level of intervention --the policy and strategy level. It's not clear from your comments whether you think this is foolish or wise. Would you like to clarify?
No, they aren't slippery concepts. The use of the words is slippery and sloppy. You're talking about a strategy to implement tactical plans, but that doesn't make it a strategy as in "US Foreign Policy Strategy." Semantics shouldn't be put aside because they help inform your query. The State Department's efforts can be tactical solutions lacking strategic support, and if so, should be judged accordingly. Sometimes you will need good tactics to compensate for a bad or absent strategy.
I called out an example in your essay that suggested a tactical failure, or possibly several points of failure. Your narrative jumped to blaming something else ("was well designed") a possible lack of an over-arching strategy to align policies, actions, and words across USG and supporting entities, possibly including other State efforts.
"Monitoring and evaluation" are a basic requirement. It's malpractice not to have them, but their absence is a default setting for too many offices and efforts. As a result, we have fire-and-forget "well-designed" programs that may fail to achieve the desired results—and potentially backfire—with little awareness until it's too late.
If you don't like "fire-and-forget", "thrown over the transom" is another phrase that reflects the lack of immediate, relevant, and as timely as possible inquiry into how an effort is going, what it's achieving and why and with whom.
But the "m&e" piece is only one of the missing and relegated puzzle pieces. I pointed out in my prior comment that tolerating failure was a core feature of your private sector examples. This must be highlighted because the lack of this tolerance has a negative effect on pace, flexibility, creativity, and impact of efforts. It can, as you will know from your experience, prevent ideas from being hashed out or tested.
It seems like we agree on the central point that M&E, or whatever one wants to call it, is essential. And that good strategy, tactics, and operationalization are also essential and interrelated. Perhaps I'm putting words in your mouth here, but the three concepts all depend on one another: It's a recipe for failure if one develops tactics without strategy. And it's a recipe for failure if one executes (operationalizing) tactics without M&E (feedback) -- "fire and forget," as you suggest.
Dan,
"Can We Monitor and Evaluate US Foreign Policy Strategy?" The best foreign policy question of 2024!
Here’s one for 2025. Can we monitor and evaluate VOA messaging of foreign audiences?
Judging by the 18 years of decline in freedom around the world, as documented by Freedom House, one could conclude our messaging about the values of freedom and the rule of law is not as effective as the counter-messaging of the bad guys.
Focussing on VOA, here’s what’s wrong in a nutshell. IMHO, VOA news in a world awash in news is not winning the information war. What’s missing from our messaging arsenal is advocacy, the marshaling of fact and truth to persuade. News is not advocacy. An attorney doesn’t read the day’s news to jurors. An attorney wins a case by persuading..
We need both news and advocacy. But VOA has evermore morphed into a news agency. It’s only advocacy is boring, milquetoast editorials.
Our adversaries' messaging has a strong point of view. Objective news is not supposed to have a point of view. That's why we need advocacy.
The solution is not more Michael Packs or Cari Lakes thrashing around in VOA. The solution is the creation of a separate agency to produce advocacy in all its creative and imaginative forms, including music and humor. Let VOA produce news. Let the new agency produce advocacy that puts us on the offense in the information war. It's time to get "them" before they get us.
Back to your theme, monitoring and evaluation. You could do Public Diplomacy a huge service by determining to what extent and in what manner VOA uses M&E to determine the effectiveness of its content. Judging by VOA's dismal performance in the info war, one doubts whether the tool is used and, if so, to what extent the information is acted upon to improve program content.
Regards,
Gordon Humphrey
Former U.S. Senator
Producer of Russian-language YouTube channel Nashi Emigranti and its English-language counterpart, Let’s Tell America’s Story to the World
Thanks for these suggestions. I think you're right: given the trends in US foreign policy, it's hard to argue that our tools are "working" (though it's theoretically possible that are tools are working, but washed out by much larger systemic forces beyond the scope of relatively small government tools). I'll say the public diplomacy folks at the State are investing heavily in M&E, though I'm not sure whether those efforts have filtered into the VOA or other news agencies. Either way, we're still operating in a culture that has a very difficult time admitting failure. Without the ability to learn and change course, any investment into M&E is a waste. Program administrators need the space to experiment, fail, and try something new.