Rule of Law Programs and the Monitoring & Evaluation Conundrum – a Snapshot

One of the things that has always bothered me about Monitoring and Evaluation (M&E) of international Rule of Law programs has been the innumerable risks involving incorrect data collection and input, and the ever present danger of conflicts of interest, not to mention lack of experience and expertise. Add to all this the challenges of violence, a hostile host government, war, and other calamities, and the writing on the wall is quite clear.

Oversight of a project does not mean that M&E is actually being carried through. A while ago, I gathered some of my thoughts. I wonder how current they are. My review of reports and audits makes me think that not much has changed.

Historically, some U.S. Government agencies were late in understanding the importance of M&E to determine the impact that foreign assistance programs were having.  In the last few years, I always kept hearing that we needed to answer then Secretary of State Hillary Clinton’s “so what?” question regarding how effective our international aid projects were.

Many multi-million dollar programs had no internal nor external M&E experts to provide guidance.  In Afghanistan, for example, the U.S. Embassy’s 2013 rule of law strategy failed to incorporate any performance measures.  (For an interesting report that reveals what the problems relating to M&E were at the time, I suggest you read the Special Inspector General for Afghanistan Reconstruction (SIGAR) audit).

Through evaluation tools, M&E programs aim to demonstrate program impact.  This, in turn, provides feedback to guide program implementation staff to enhance future programming by identifying planned and unplanned results to allow donors, implementers and host country beneficiaries to understand what works and does not work, how to maximize efficiencies, and address any issues that might arise before they become a problem or a cataclysmic risk.

In government contracts, the Statement of Work (SOW) may provide the indicators to be used.  Sometimes, the implementer may develop a series of iterative evaluations as well, which might include a training evaluation and an audit, a trainee-satisfaction survey, a mentoring plan, and -depending on the program- a public outreach component.

Performance indicators may combine the Foreign Assistance Framework Indicators (F-Indicators), as well as customized indicators, with the goal to develop and utilize indicators that measure outputs and impact in the short, medium and long-term of the project.

Of course, the most perfect and all-encompassing M&E plan will not work unless both donors, implementers and beneficiaries take into account the critical risks inherent in, or coming from, the place of performance, and agree on some critical assumptions that, at the very least, encompass three contexts: political, security, and operational.

What I have learnt is that decision-makers and bureaucrats from both the government side and the corporate side make choices and issue “diktats” without having had the benefit of operating in the environment where the program is being carried out.  I never gave it much thought until I witnessed it first-hand.  Therefore, it is imperative that the “experts” who are hired to handle M&E issues understand that they may be dealing with people who have little or no knowledge of the hurdles the technical staff face day in and day out.

Sometimes, the mere fact that electricity is not available or the internet connection does not work, may mean that M&E data cannot be incorporated into a database.

While I applaud the importance of M&E in program management, I see some problem areas:

Who monitors and evaluates the authenticity and the accuracy of the M&E plan and its implementation in-house?  In other words, if I am the donor, would I fully trust the contractor or grantee to monitor and evaluate itself?

Through my own observations, I came to the conclusion that the Chief of Team of a project would have to have not only “Rule of Law” and international development experience, but a keen understanding of the host country – for example Afghanistan -, business development, human resources, management, accounting, psychology, history, security, culture, philosophy, and -most importantly- an understanding of how the State Department works. Who can meet all those requirements?

If the donor hires a third-party to do an indepent M&E of a program, how comfortable can the donor and implementer be that the third-party will do an unbiased and truly objective M&E assessment?  What are the chances that the M&E firm will have a former implementer employee evaluating the very same program that person put in place?

Rule of Law programs are not immune from a myriad of conflicts of interest.  Who pays attention to these things?  

Afghanistan – a Labor of Love

When I first started working with Afghanistan, one of my job requirements was to prepare a presentation for future advisors in the justice sector. These advisors were primarily American and international lawyers, judges, corrections officers, and a sprinkling of other experts.

My research uncovered amazing stories and records of a time in the mid-20th century that seemed surreal. The shock of what had been versus what was. To wit, Once upon a Time in Afghanistan.

This research led me to a jewel of a movie, that I considered a love feast for the eyes, because in a short time, it captured the beauty of the country and its people. I had witnessed it myself, despite my being confined to a limited area in Kabul. This was around October 2012.

I never forgot that film, and that’s why I share it today.

I always think of Afghanistan, and everyone I met there, and I still feel sadness at how all our efforts seemed to go up in flames. Sometimes I wonder whether it was all for nought. Maybe, maybe, I am too pessimistic and there’s a glimmer of hope. Miracles do happen.  

Afghanistan – touch down in flight is a beautiful 5 minute film by Salome and Lukas Augustin. It is dedicated to the Afghan people and Gayle Williams, a British aid worker who worked with the disabled, who was murdered by the Taliban because they claimed she was spreading Christianity.

Watch it below! You won’t regret it.

Of Tigers and Lions and the Rule of Law ~ Kabul-kind-of days, circa 2-24-2012

“Corans brûlés”… I read the title doing a Google search, and, for an instance, found it amusing. For a fleeting moment, the term conjured up pleasant, sweet images. Yet, thanks to the “Corans brûlés” we had been on “lockdown” for a couple of days.

Life in Afghanistan, which, according to all expats I had spoken with, could be tediously monotonous, had the capacity of changing course in a blink.  I was happily ensconced in our working compound, researching away, when word came out that the Security office of the US Embassy was enforcing a lockdown because it had been told the Korans had been burnt in one of our bases there.

It was odd, traipsing to our SUVs and starting the trek back to our living quarters.  All of a sudden, I kept looking at the people standing on the sides of the roads with trepidation.  The men one saw on those same roads, carrying weapons, who were they?  Presumably, they were policemen.  But who were they meant to protect?

We hit some traffic, but all went well.

The next day, the American and international advisors were all ready to leave to our work compound by 7:15AM, but by 9:30AM it became obvious we were not going anywhere (we followed what the US Embassy dictated).  So we all stayed in our gilded cages, working from our rooms. 

Yet, all the classes for Afghan judges, prosecutors, defense lawyers, and criminal investigators were still being held inside our compound, and all our Afghan staff (instructors, justice advisors, cooks, char force) was still there. The irony of our work environment and the monumental dangers these Afghans faced did not escape me and I marveled at their dedication and strength of spirit.

By the end of the day we knew that the compound called Green Village, where a large group of contractors and other foreigners lived (and where we had been the week before), had been accosted – so much so, that apparently the residents had to stay in the concrete bunkers for a while.

But our own Camp X had been calm, probably because it was right next door to the airport and the airbase out of which US military and State Department flights took off.  The areas in my Camp containing what I thought were excess concrete dividers turned out to be where the bunkers had been set up to protect us.   There had been no need to use these bunkers until that day.

The previous day, for some strange reason, they had lifted lockdown for a while, so that we ended up going to the office (even though all travel to all places – like ministries – had been cancelled).   In the entire time I had been in Kabul, this was the first and only time that I was uncomfortable with the idea, because our Afghan driver was uncomfortable himself.  Yet, the trip was uneventful, and we took a completely different route, far away from the madding crowd. 

Because we left much later than usual (around 10AM), I noticed the squalid little stores all a-buzz with action:  the butcher shops with their mutton carcasses hanging outside; the cobblers sitting on their dilapidated wooden boxes; the men clearing snow off little shelves where wooden planks and poles were kept for construction purposes; the tin pot stores with their glittery gold and silver wares shining pretty in an otherwise bleak setting; and everywhere the women in their burqas, walking with little high heels or heavy boots over filthy slush, or squeezed inside a mini-bus or taxi.  I kept thinking I had never seen such pathetic penury with an eclectic whirring of sorts.  However, in all the places I had ever been, I had never looked at a city with such sadness. 

Despite it all, the morning commute turned into a veritable history lesson, and the 3 of us passengers, and the nervous Afghan driver, had a hilarious conversation that began when I tried to make light of a nerve-wracking trip. I asked my Pakistani Muslim colleague whether kids in Afghanistan had names signifying “barbarian”, like mine.  

Thus the driver and my colleague began the story of Babur, a former king of Afghanistan, descendant of Genghis Khan and Tamerlane, who settled in Delhi and began the Moghul Dynasty in India, and whose great-grandson built the Taj Mahal. 

They told me that Babur (or Babr in Persian) is very close to Barbara, and that Babur’s name really means “lion”. I pointed out that Babur and I had something in common -the lion- since my surname Dillon originally stems from “De Leon”.  There were some guffaws and by the time we entered Compound Y, there was no longer nervous tension in the car, and I was not a “barbarian”, but rather “Barbara the double lion!”. I didn’t quite get why “double”, but maybe it had something to do with my gray hair? It turned out that Babur can also mean “tiger” in Arabic, but the beauty of the laughter that helped alleviate tension was not lost on me. With a sense of relief, I genuinely felt close to my driver and fellow passengers. And so what; tigers and lions (and bears, oh my) at the end of the day conveyed the same image of strength and might.

By late afternoon, we were told the sobering news that 2 US soldiers and countless others had been killed, and we were once again put in our armored SUVs, but this time we had a third SUV with a couple of “shooters” inside, to escort us back.  Because it was Thursday afternoon, and the day before the Muslim “Sabbath”, there were few people in the streets, so our ride, thankfully, was uneventful as well, though we could see 3-4 helicopters flying in formation above the US compounds over my own Camp X.

Friday was the only day off at work, and we were also placed on lockdown, because it is the day when the Afghan males go to the mosque, and it was anticipated that the mullahs would be inciting the masses.  We were all surely safely ensconced in our little nests inside the gilded cage of our compound.  Other than the occasional helicopter flying over us, this was a totally calm and ordinary day in Kabul. 

I had no idea what would happen the next day.  The assumption was that we would all be getting ready to go to work.  But we would not find out until the early morning. It turned out the lockdown continued. Alas, our work in preparing class curricula, presentations, and reports to headquarters and the US Government, was never interrupted.

And so ended another day.  

Kabul commute 2 002.JPG
Kabul commute 2 002.JPG

See the water pump to the left and Nan bread to the right.

A bunker by the dining facilities.

Poles used in construction.

The Fall of Afghanistan

While time eases anguish for some, I still smart at the thought of all the work, human lives and dreams and treasure lost not so long ago. In August 18, 2021 I reflected having a very hard time engaging in every-day lovely thoughts and things.

Earlier that month had sent me into a spiraling depression. Why? Well… Six years of working with Afghanistan and many more years being engaged with the country.

I don’t think anyone other than those who were so thoroughly engaged with Afghanistan could understand, but maybe I was wrong.

I cannot begin to imagine what family/friends/colleagues of those who fought, worked and died there were feeling then -and even right now-, both in the US and abroad.

Although, for those who question today, “what was the point?”, I can only answer, the point was all the Afghans. We worked hard to make a difference for the Afghans.

Yes, indeed, contractors made a lot of money. There is a monetary value attached to high risk. And the US Government was aloof a lot of times, hiding behind the mighty fortresses of secured buildings in Afghanistan while the hired contractor employees, earning good salaries, risked much.

However, most people I knew who were hands-on (Afghans, Americans, USG employees, international employees), worked hard for a new future for all Afghans, and risked their lives. I didn’t know a lot of them, but I knew a few, who hailed from all over the world. From Colombia to Nepal.

At the time, while I did not visit Facebook that often, I had felt the need to share the overwhelming sadness I felt about Afghanistan.

Let us not forget the ugly corruption surrounding everything we worked on. What else is new? Corruption affects everything and everyone. Here, there and everywhere. The only difference is the Rule of Law as it is meant to be. Justice meted fairly for all. It is corroding around us nowadays. Hopefully, we can help save it the way we should.

Inevitably, the Fall of Afghanistan all ended up being an internecine battle here in the US, which I found not only revolting at that time, but it triggered an anger I have seldom felt.

I leave you with a quote from an email I sent from Kabul to family and friends on February of 2012:

“The snow makes the place more picturesque, but it is grim. I can handle most anything, except seeing the burqa-clad beggars sitting on the side of the roads, in the slush, getting soaked.

Yesterday, I attended for a brief period one of the classes set up for 38 judges, prosecutors, lawyers and investigators. It was fascinating. The Afghans, though loquacious, don’t engage in screaming matches like the Iraqis did. I find that my silver head amuses them a bit.

I am humbled by all the American and international advisors here who work under dire conditions. No one from the outside really knows all the work that these guys are doing. Will all these efforts yield fruit? Or will the country collapse into civil war after we pull out?

It is interesting to get feedback from these advisors who have been in remote locations. They all love working with the Afghans, although they realize that the common refrain here is: “brother against brother, brothers against father, family against tribe, tribe against tribe, tribes against country, country against the world”.”

A Painter’s Ramblings on War

On May 8, 2009 I discovered Ramblings from a Painter, a painter’s ramblings of his Iraq military tour of duty. At the time I had noted in my defunct old blog that artist Skip Rhode had a wonderful gift, and made some ugly landscapes look delightful! For example, his painting of a Containerized Housing Unit or CHU (a shipping container used for living quarters by the US military in Iraq and Afghanistan) makes it seem inviting and cozy.

But what caught my eye initially was Mr. Rohde’s blog entry on Iraqi children’s drawings. He said,

What got my attention was just how normal these drawings are. They could have been done by any kid in the United States. Here are happy families with little houses in the countryside with flowers and trees and puffy clouds.  I’m not quite sure what that thing is in the sky in the bottom picture – a bird? a bug? – but for sure it isn’t threatening.  All the figures have big happy smiles on their faces.  These are happy drawings from happy kids.

Lament, the Pietà-like painting above, evokes a sorrow, an anguish that is hard to fathom. It is the inescapable grief of a Mother who has lost her son, seemingly forever. The Mother’s pained look displays some determination, in my humble opinion. This Lament makes me think that she is a woman of faith, so that beyond the sadness there is a glimmer of hope.

What a poignant painting that encapsulates the senseless horrors of feckless times.

A “Baad” Story from Afghanistan

We bought you with money and will kill you with a stone “Da zar kharidim da sang mekoshim”

Violence against women and girls is a universal problem. The fear and sadness in a victim’s face is something I will never forget. I witnessed those faces in many countries, while working on “Rule of Law” projects, where we were trying to make the public aware that there were “legal” avenues to combat such an abuse. The recurring theme from the victims I met had an underlying commonality: the cold hatred in the eyes of the perpetrator, before and after the violent acts, was worse than the actual physical pain.

When I worked in a program involving the justice sector of Afghanistan, I learnt about “baad”. The New York Times had a story in 2012 that explained the baad custom that is prevalent in certain areas of Afghanistan.

It is a practice that most of us find repulsive:  the giving of girls as reparation for the crimes or bad deeds of their male relatives.  It is a traditional form of dispute resolution that had been made illegal in Afghanistan at the national level. 

The Afghan Government in 2009 had enacted by decree, The Law on the Elimination of Violence against Women (EVAW), that specifically referred to the practice of baad, making it a criminal act to marry or “give away” girls and women to someone as blood price. The law prohibited the trading of women and girls to resolve disputes, including those related to murder, sexual violence, or other harmful acts. The UN’s Assistance Mission to Afghanistan (UNAMA) had to say about baad in 2010 added this explanation to the problem:

UNAMA HR found that giving away girls to settle disputes, through baad, takes place in communities throughout the country. In spite of the prevalence of this practice, many Afghans expressed strong opposition to it. As an informal method of dispute resolution, UNAMA HR found that in the central region more baad is practiced in conflict zones where the Government exercises less authority and lacks legitimacy (for example, conflict-affected areas such as Tagab and Alasay district in Kapisa province, Uzbin in Sarobi district of Kabul province) and in remote areas where the formal rule of law institutions are weakest.

One reality, though, was that the formal justice sector outside of major urban areas had limited resources and functionality.  At the local level, jirgas or shuras headed by community elders or religious leaders settled community disputes.  Another reality was the fact that many communities were totally unaware of what the national law stated.  

A booklet produced by the International Development Law Organization (IDLO), that was used by the Afghan Attorney General’s office to explain the EVAW, provided a glimpse of the enormity of the educational campaign needed to reach the many rural and remote provinces, communities and Government officials who did not know about the laws affecting the rights of women. There were other publications, as well as a comic book, Masooma’s Sunrise (see below). IDLO is a “global intergovernmental organization exclusively devoted to promoting the rule of law to advance peace and sustainable development”.  

The U.S. military intervention in Afghanistan did not contemplate advancing women’s status and rights. However, the U.S. reconstruction effort’s goals included improving the lives of Afghan women and girls.

The Special Inspector General for Afghanistan Reconstruction (SIGAR) found that from 2002 to 2020, the Department of State, the U.S. Agency for International Development (USAID), and the Department of Defense (DOD) disbursed at least $787.4 million for programs that specifically and primarily supported Afghan women and girls in the areas of health, education, political participation, access to justice, and economic participation. SIGAR also stated that “[t]his understates the total U.S. investment in women and girls, however, since hundreds of additional U.S. programs and projects included an unquantifiable gender component. State and USAID have not consistently tracked or quantified the amount of money disbursed for projects which directly or indirectly support Afghan women, girls, or gender equality goals. Therefore, the full extent of U.S. programming to support Afghan women and girls is not quantifiable.”

Nowadays, I find so little information about the plight of the women and girls in Afghanistan. So much time has gone by, and the little progress that had been made went up in smoke, so to speak, when the U.S. withdrew from Afghanistan.

I look back at my involvement in the Rule of Law work we did in Afghanistan and can’t help thinking that we were neophytes in a social and legal experiment that we did not understand and were not fully committed to carry through.

One of my prized possessions is a lapis lazuli stone and a CD that the Afghan ladies working in the gender-based violence program I was involved with gave me. The CD contained pictures and recordings of the numerous billboards and TV programs that had been created to bring awareness to the population at large, and to let the victims of violence know that there were shelters available for them to seek protection and peace. A small amount of those millions of funds went into that campaign.

Nowadays, I can’t help but wonder, was all this for nought?