Tag: USAID

Rule of Law Programs and the Monitoring & Evaluation Conundrum – a Snapshot

November 4, 2024

One of the things that has always bothered me about Monitoring and Evaluation (M&E) of international Rule of Law programs has been the innumerable risks involving incorrect data collection and input, and the ever present danger of conflicts of interest, not to mention lack of experience and expertise. Add to all this the challenges of violence, a hostile host government, war, and other calamities, and the writing on the wall is quite clear.

Oversight of a project does not mean that M&E is actually being carried through. A while ago, I gathered some of my thoughts. I wonder how current they are. My review of reports and audits makes me think that not much has changed.

Historically, some U.S. Government agencies were late in understanding the importance of M&E to determine the impact that foreign assistance programs were having.  In the last few years, I always kept hearing that we needed to answer then Secretary of State Hillary Clinton’s “so what?” question regarding how effective our international aid projects were.

Many multi-million dollar programs had no internal nor external M&E experts to provide guidance.  In Afghanistan, for example, the U.S. Embassy’s 2013 rule of law strategy failed to incorporate any performance measures.  (For an interesting report that reveals what the problems relating to M&E were at the time, I suggest you read the Special Inspector General for Afghanistan Reconstruction (SIGAR) audit).

Through evaluation tools, M&E programs aim to demonstrate program impact.  This, in turn, provides feedback to guide program implementation staff to enhance future programming by identifying planned and unplanned results to allow donors, implementers and host country beneficiaries to understand what works and does not work, how to maximize efficiencies, and address any issues that might arise before they become a problem or a cataclysmic risk.

In government contracts, the Statement of Work (SOW) may provide the indicators to be used.  Sometimes, the implementer may develop a series of iterative evaluations as well, which might include a training evaluation and an audit, a trainee-satisfaction survey, a mentoring plan, and -depending on the program- a public outreach component.

Performance indicators may combine the Foreign Assistance Framework Indicators (F-Indicators), as well as customized indicators, with the goal to develop and utilize indicators that measure outputs and impact in the short, medium and long-term of the project.

Of course, the most perfect and all-encompassing M&E plan will not work unless both donors, implementers and beneficiaries take into account the critical risks inherent in, or coming from, the place of performance, and agree on some critical assumptions that, at the very least, encompass three contexts: political, security, and operational.

What I have learnt is that decision-makers and bureaucrats from both the government side and the corporate side make choices and issue “diktats” without having had the benefit of operating in the environment where the program is being carried out.  I never gave it much thought until I witnessed it first-hand.  Therefore, it is imperative that the “experts” who are hired to handle M&E issues understand that they may be dealing with people who have little or no knowledge of the hurdles the technical staff face day in and day out.

Sometimes, the mere fact that electricity is not available or the internet connection does not work, may mean that M&E data cannot be incorporated into a database.

While I applaud the importance of M&E in program management, I see some problem areas:

Who monitors and evaluates the authenticity and the accuracy of the M&E plan and its implementation in-house?  In other words, if I am the donor, would I fully trust the contractor or grantee to monitor and evaluate itself?

Through my own observations, I came to the conclusion that the Chief of Team of a project would have to have not only “Rule of Law” and international development experience, but a keen understanding of the host country – for example Afghanistan -, business development, human resources, management, accounting, psychology, history, security, culture, philosophy, and -most importantly- an understanding of how the State Department works. Who can meet all those requirements?

If the donor hires a third-party to do an indepent M&E of a program, how comfortable can the donor and implementer be that the third-party will do an unbiased and truly objective M&E assessment?  What are the chances that the M&E firm will have a former implementer employee evaluating the very same program that person put in place?

Rule of Law programs are not immune from a myriad of conflicts of interest.  Who pays attention to these things?  

A “Baad” Story from Afghanistan

October 8, 2024

“We bought you with money and will kill you with a stone “Da zar kharidim da sang mekoshim”“

Violence against women and girls is a universal problem. The fear and sadness in a victim’s face is something I will never forget. I witnessed those faces in many countries, while working on “Rule of Law” projects, where we were trying to make the public aware that there were “legal” avenues to combat such an abuse. The recurring theme from the victims I met had an underlying commonality: the cold hatred in the eyes of the perpetrator, before and after the violent acts, was worse than the actual physical pain.

When I worked in a program involving the justice sector of Afghanistan, I learnt about “baad”. The New York Times had a story in 2012 that explained the baad custom that is prevalent in certain areas of Afghanistan.

It is a practice that most of us find repulsive:  the giving of girls as reparation for the crimes or bad deeds of their male relatives.  It is a traditional form of dispute resolution that had been made illegal in Afghanistan at the national level. 

The Afghan Government in 2009 had enacted by decree, The Law on the Elimination of Violence against Women (EVAW), that specifically referred to the practice of baad, making it a criminal act to marry or “give away” girls and women to someone as blood price. The law prohibited the trading of women and girls to resolve disputes, including those related to murder, sexual violence, or other harmful acts. The UN’s Assistance Mission to Afghanistan (UNAMA) had to say about baad in 2010 added this explanation to the problem:

“UNAMA HR found that giving away girls to settle disputes, through baad, takes place in communities throughout the country. In spite of the prevalence of this practice, many Afghans expressed strong opposition to it. As an informal method of dispute resolution, UNAMA HR found that in the central region more baad is practiced in conflict zones where the Government exercises less authority and lacks legitimacy (for example, conflict-affected areas such as Tagab and Alasay district in Kapisa province, Uzbin in Sarobi district of Kabul province) and in remote areas where the formal rule of law institutions are weakest.“

One reality, though, was that the formal justice sector outside of major urban areas had limited resources and functionality.  At the local level, jirgas or shuras headed by community elders or religious leaders settled community disputes.  Another reality was the fact that many communities were totally unaware of what the national law stated.  

A booklet produced by the International Development Law Organization (IDLO), that was used by the Afghan Attorney General’s office to explain the EVAW, provided a glimpse of the enormity of the educational campaign needed to reach the many rural and remote provinces, communities and Government officials who did not know about the laws affecting the rights of women. There were other publications, as well as a comic book, Masooma’s Sunrise (see below). IDLO is a “global intergovernmental organization exclusively devoted to promoting the rule of law to advance peace and sustainable development”.  

The U.S. military intervention in Afghanistan did not contemplate advancing women’s status and rights. However, the U.S. reconstruction effort’s goals included improving the lives of Afghan women and girls.

The Special Inspector General for Afghanistan Reconstruction (SIGAR) found that from 2002 to 2020, the Department of State, the U.S. Agency for International Development (USAID), and the Department of Defense (DOD) disbursed at least $787.4 million for programs that specifically and primarily supported Afghan women and girls in the areas of health, education, political participation, access to justice, and economic participation. SIGAR also stated that “[t]his understates the total U.S. investment in women and girls, however, since hundreds of additional U.S. programs and projects included an unquantifiable gender component. State and USAID have not consistently tracked or quantified the amount of money disbursed for projects which directly or indirectly support Afghan women, girls, or gender equality goals. Therefore, the full extent of U.S. programming to support Afghan women and girls is not quantifiable.”

Nowadays, I find so little information about the plight of the women and girls in Afghanistan. So much time has gone by, and the little progress that had been made went up in smoke, so to speak, when the U.S. withdrew from Afghanistan.

I look back at my involvement in the Rule of Law work we did in Afghanistan and can’t help thinking that we were neophytes in a social and legal experiment that we did not understand and were not fully committed to carry through.

One of my prized possessions is a lapis lazuli stone and a CD that the Afghan ladies working in the gender-based violence program I was involved with gave me. The CD contained pictures and recordings of the numerous billboards and TV programs that had been created to bring awareness to the population at large, and to let the victims of violence know that there were shelters available for them to seek protection and peace. A small amount of those millions of funds went into that campaign.

Nowadays, I can’t help but wonder, was all this for nought?