Rule of Law Programs and the Monitoring & Evaluation Conundrum – a Snapshot

One of the things that has always bothered me about Monitoring and Evaluation (M&E) of international Rule of Law programs has been the innumerable risks involving incorrect data collection and input, and the ever present danger of conflicts of interest, not to mention lack of experience and expertise. Add to all this the challenges of violence, a hostile host government, war, and other calamities, and the writing on the wall is quite clear.

Oversight of a project does not mean that M&E is actually being carried through. A while ago, I gathered some of my thoughts. I wonder how current they are. My review of reports and audits makes me think that not much has changed.

Historically, some U.S. Government agencies were late in understanding the importance of M&E to determine the impact that foreign assistance programs were having.  In the last few years, I always kept hearing that we needed to answer then Secretary of State Hillary Clinton’s “so what?” question regarding how effective our international aid projects were.

Many multi-million dollar programs had no internal nor external M&E experts to provide guidance.  In Afghanistan, for example, the U.S. Embassy’s 2013 rule of law strategy failed to incorporate any performance measures.  (For an interesting report that reveals what the problems relating to M&E were at the time, I suggest you read the Special Inspector General for Afghanistan Reconstruction (SIGAR) audit).

Through evaluation tools, M&E programs aim to demonstrate program impact.  This, in turn, provides feedback to guide program implementation staff to enhance future programming by identifying planned and unplanned results to allow donors, implementers and host country beneficiaries to understand what works and does not work, how to maximize efficiencies, and address any issues that might arise before they become a problem or a cataclysmic risk.

In government contracts, the Statement of Work (SOW) may provide the indicators to be used.  Sometimes, the implementer may develop a series of iterative evaluations as well, which might include a training evaluation and an audit, a trainee-satisfaction survey, a mentoring plan, and -depending on the program- a public outreach component.

Performance indicators may combine the Foreign Assistance Framework Indicators (F-Indicators), as well as customized indicators, with the goal to develop and utilize indicators that measure outputs and impact in the short, medium and long-term of the project.

Of course, the most perfect and all-encompassing M&E plan will not work unless both donors, implementers and beneficiaries take into account the critical risks inherent in, or coming from, the place of performance, and agree on some critical assumptions that, at the very least, encompass three contexts: political, security, and operational.

What I have learnt is that decision-makers and bureaucrats from both the government side and the corporate side make choices and issue “diktats” without having had the benefit of operating in the environment where the program is being carried out.  I never gave it much thought until I witnessed it first-hand.  Therefore, it is imperative that the “experts” who are hired to handle M&E issues understand that they may be dealing with people who have little or no knowledge of the hurdles the technical staff face day in and day out.

Sometimes, the mere fact that electricity is not available or the internet connection does not work, may mean that M&E data cannot be incorporated into a database.

While I applaud the importance of M&E in program management, I see some problem areas:

Who monitors and evaluates the authenticity and the accuracy of the M&E plan and its implementation in-house?  In other words, if I am the donor, would I fully trust the contractor or grantee to monitor and evaluate itself?

Through my own observations, I came to the conclusion that the Chief of Team of a project would have to have not only “Rule of Law” and international development experience, but a keen understanding of the host country – for example Afghanistan -, business development, human resources, management, accounting, psychology, history, security, culture, philosophy, and -most importantly- an understanding of how the State Department works. Who can meet all those requirements?

If the donor hires a third-party to do an indepent M&E of a program, how comfortable can the donor and implementer be that the third-party will do an unbiased and truly objective M&E assessment?  What are the chances that the M&E firm will have a former implementer employee evaluating the very same program that person put in place?

Rule of Law programs are not immune from a myriad of conflicts of interest.  Who pays attention to these things?  

Afghanistan – a Labor of Love

When I first started working with Afghanistan, one of my job requirements was to prepare a presentation for future advisors in the justice sector. These advisors were primarily American and international lawyers, judges, corrections officers, and a sprinkling of other experts.

My research uncovered amazing stories and records of a time in the mid-20th century that seemed surreal. The shock of what had been versus what was. To wit, Once upon a Time in Afghanistan.

This research led me to a jewel of a movie, that I considered a love feast for the eyes, because in a short time, it captured the beauty of the country and its people. I had witnessed it myself, despite my being confined to a limited area in Kabul. This was around October 2012.

I never forgot that film, and that’s why I share it today.

I always think of Afghanistan, and everyone I met there, and I still feel sadness at how all our efforts seemed to go up in flames. Sometimes I wonder whether it was all for nought. Maybe, maybe, I am too pessimistic and there’s a glimmer of hope. Miracles do happen.  

Afghanistan – touch down in flight is a beautiful 5 minute film by Salome and Lukas Augustin. It is dedicated to the Afghan people and Gayle Williams, a British aid worker who worked with the disabled, who was murdered by the Taliban because they claimed she was spreading Christianity.

Watch it below! You won’t regret it.

Of Tigers and Lions and the Rule of Law ~ Kabul-kind-of days, circa 2-24-2012

“Corans brûlés”… I read the title doing a Google search, and, for an instance, found it amusing. For a fleeting moment, the term conjured up pleasant, sweet images. Yet, thanks to the “Corans brûlés” we had been on “lockdown” for a couple of days.

Life in Afghanistan, which, according to all expats I had spoken with, could be tediously monotonous, had the capacity of changing course in a blink.  I was happily ensconced in our working compound, researching away, when word came out that the Security office of the US Embassy was enforcing a lockdown because it had been told the Korans had been burnt in one of our bases there.

It was odd, traipsing to our SUVs and starting the trek back to our living quarters.  All of a sudden, I kept looking at the people standing on the sides of the roads with trepidation.  The men one saw on those same roads, carrying weapons, who were they?  Presumably, they were policemen.  But who were they meant to protect?

We hit some traffic, but all went well.

The next day, the American and international advisors were all ready to leave to our work compound by 7:15AM, but by 9:30AM it became obvious we were not going anywhere (we followed what the US Embassy dictated).  So we all stayed in our gilded cages, working from our rooms. 

Yet, all the classes for Afghan judges, prosecutors, defense lawyers, and criminal investigators were still being held inside our compound, and all our Afghan staff (instructors, justice advisors, cooks, char force) was still there. The irony of our work environment and the monumental dangers these Afghans faced did not escape me and I marveled at their dedication and strength of spirit.

By the end of the day we knew that the compound called Green Village, where a large group of contractors and other foreigners lived (and where we had been the week before), had been accosted – so much so, that apparently the residents had to stay in the concrete bunkers for a while.

But our own Camp X had been calm, probably because it was right next door to the airport and the airbase out of which US military and State Department flights took off.  The areas in my Camp containing what I thought were excess concrete dividers turned out to be where the bunkers had been set up to protect us.   There had been no need to use these bunkers until that day.

The previous day, for some strange reason, they had lifted lockdown for a while, so that we ended up going to the office (even though all travel to all places – like ministries – had been cancelled).   In the entire time I had been in Kabul, this was the first and only time that I was uncomfortable with the idea, because our Afghan driver was uncomfortable himself.  Yet, the trip was uneventful, and we took a completely different route, far away from the madding crowd. 

Because we left much later than usual (around 10AM), I noticed the squalid little stores all a-buzz with action:  the butcher shops with their mutton carcasses hanging outside; the cobblers sitting on their dilapidated wooden boxes; the men clearing snow off little shelves where wooden planks and poles were kept for construction purposes; the tin pot stores with their glittery gold and silver wares shining pretty in an otherwise bleak setting; and everywhere the women in their burqas, walking with little high heels or heavy boots over filthy slush, or squeezed inside a mini-bus or taxi.  I kept thinking I had never seen such pathetic penury with an eclectic whirring of sorts.  However, in all the places I had ever been, I had never looked at a city with such sadness. 

Despite it all, the morning commute turned into a veritable history lesson, and the 3 of us passengers, and the nervous Afghan driver, had a hilarious conversation that began when I tried to make light of a nerve-wracking trip. I asked my Pakistani Muslim colleague whether kids in Afghanistan had names signifying “barbarian”, like mine.  

Thus the driver and my colleague began the story of Babur, a former king of Afghanistan, descendant of Genghis Khan and Tamerlane, who settled in Delhi and began the Moghul Dynasty in India, and whose great-grandson built the Taj Mahal. 

They told me that Babur (or Babr in Persian) is very close to Barbara, and that Babur’s name really means “lion”. I pointed out that Babur and I had something in common -the lion- since my surname Dillon originally stems from “De Leon”.  There were some guffaws and by the time we entered Compound Y, there was no longer nervous tension in the car, and I was not a “barbarian”, but rather “Barbara the double lion!”. I didn’t quite get why “double”, but maybe it had something to do with my gray hair? It turned out that Babur can also mean “tiger” in Arabic, but the beauty of the laughter that helped alleviate tension was not lost on me. With a sense of relief, I genuinely felt close to my driver and fellow passengers. And so what; tigers and lions (and bears, oh my) at the end of the day conveyed the same image of strength and might.

By late afternoon, we were told the sobering news that 2 US soldiers and countless others had been killed, and we were once again put in our armored SUVs, but this time we had a third SUV with a couple of “shooters” inside, to escort us back.  Because it was Thursday afternoon, and the day before the Muslim “Sabbath”, there were few people in the streets, so our ride, thankfully, was uneventful as well, though we could see 3-4 helicopters flying in formation above the US compounds over my own Camp X.

Friday was the only day off at work, and we were also placed on lockdown, because it is the day when the Afghan males go to the mosque, and it was anticipated that the mullahs would be inciting the masses.  We were all surely safely ensconced in our little nests inside the gilded cage of our compound.  Other than the occasional helicopter flying over us, this was a totally calm and ordinary day in Kabul. 

I had no idea what would happen the next day.  The assumption was that we would all be getting ready to go to work.  But we would not find out until the early morning. It turned out the lockdown continued. Alas, our work in preparing class curricula, presentations, and reports to headquarters and the US Government, was never interrupted.

And so ended another day.  

Kabul commute 2 002.JPG
Kabul commute 2 002.JPG

See the water pump to the left and Nan bread to the right.

A bunker by the dining facilities.

Poles used in construction.

Afghanistan: Reflections of a Not-so-Long Ago Era

Someone asked me about my legal work involving Afghanistan.  Her question triggered old memories of an enthralling time that appears to have been lost, at least for a while.  However, my propensity for keeping diaries and writing e-mails and letters, developed during elementary school in Buenos Aires, preserved memories of my impressions visiting and working in Kabul a decade ago. 

I was new to the country, but an old hand in what we used to call “development work in the Rule of Law”.  What follow are my personal impressions, as written then, updated with a couple of edits. All the work mentioned here can be found in the myriad of inspection reports, evaluation reports and other documents that the US Government makes available to the public.   

February 17, 2012:

It was a gloriously sunny day, clear blue skies, with not a hint of cloud or haze.  So much so that I could see, for the first time, the little houses built on the denuded slopes of the Hindu Kush.  

I had no idea that Hindu Kush actually means Hindu Slaughter.  I need to do more research about this.

So, two days ago I was up by 6:30am and by 7:45am we were all in 3 vehicles going from Camp X, where I live, to Compound Y, where we work (some of us live and work in the same sites, which can become quite onerous.  Can you imagine having breakfast, lunch and dinner with the same people you work with day in and day out?  For a year or two or more?).

Compound  Y is situated in the most exclusive neighborhood of Kabul.  It consists of villas walled in.  It is no different than all the other villas around (some held by Embassies, others by international organizations, and others by the ministers and sundry government officials).   The movie The Kite Runner apparently was filmed in this neighborhood called Wazir Akbar Khan.  Wazir Akbar Khan was the leader who fought during the famous First Anglo-Afghan War that ended in a monumental defeat of the British army in Gandamak. At one point the highest governmental award that the Afghan Government could bestow was an eponymous medal.

What is noticeable, though, is that -apparently- all the streets are unpaved, (I cannot tell right now because of the snow), have barriers across them that are always up, and can compete with the rest of the roads in terms of the deep craters that destroy cars.  Rumor has it that the residents don’t want these holes repaired because it protects them from nefarious sorts:  these craters don’t allow for a quick getaway.

I took the opportunity to go to the roof of my building to take photos of the city and the surrounding mountains, because I was told that it is very, very rare to get such a clear glimpse of the mountains that encircle the city.

I met a young lady, from abroad, with a Masters in Business Administration, who is a “procurement advisor” at a major Ministry and is helping them set up a system for tracking, budgeting for and storing, inventory.  She has been in Kabul for 3 years and loves the Afghans and her job.   

One of the aims of my program is to train Afghan judges, prosecutors, lawyers and criminal investigators to clinically apply the law.  Among the several courses we offer, one lasts 8 weeks, and another 4 weeks.  Since 2007 there have been more than 14 thousand individuals trained by us! 

Another aspect of the project is to have the Attorney General’s and the Ministry of Justice’s Offices identify promising Afghan lawyers who are then trained for 1 year before sending them to University of Washington’s Law School for a 1 year LLM program.  Why University of Washington?  Because they have, apparently, the best legal clinic that provides pro bono work for Native American tribes.  And this is the closest one can get to understanding the tension between the formal justice sector (courts and the power of the state) and the informal justice sector that handles tribal issues and customary law.

There are so many things that are happening here that one never hears about!  I shared a cup of tea with an engineer with the US Corps of Engineers, who has been managing construction projects all around Afghanistan.  Example:  building generators in remote villages (that have no roads) and that, through the use of little streams, generate the first electricity these people have ever had!  They have also built small schools in these small villages, so that, again, for the first time, children have a place to go and sit down to attend classes taught by Afghans who have been trained by the international community.

I share the pics of the mountains that form the Kabul bowl, and you can get a sense for the architecture of the place.  We cannot drink the water, so bottles are stored, en masse, everywhere.  

And we have a little kitty that sits by the front door of the building. 

We are, of course, in a state of alert, and, for good reason, the security task force here takes it very seriously. 

I leave you with a comment a wise advisor made to me:  

In Afghanistan, 2012 is the Persian year 1390, which, if one compares it to our Western world’s 1390, not much has changed:  there are still feudal lords or barons (the Afghan warlords) fighting each other with the vast majority of the people being illiterate serfs.

The Fall of Afghanistan

While time eases anguish for some, I still smart at the thought of all the work, human lives and dreams and treasure lost not so long ago. In August 18, 2021 I reflected having a very hard time engaging in every-day lovely thoughts and things.

Earlier that month had sent me into a spiraling depression. Why? Well… Six years of working with Afghanistan and many more years being engaged with the country.

I don’t think anyone other than those who were so thoroughly engaged with Afghanistan could understand, but maybe I was wrong.

I cannot begin to imagine what family/friends/colleagues of those who fought, worked and died there were feeling then -and even right now-, both in the US and abroad.

Although, for those who question today, “what was the point?”, I can only answer, the point was all the Afghans. We worked hard to make a difference for the Afghans.

Yes, indeed, contractors made a lot of money. There is a monetary value attached to high risk. And the US Government was aloof a lot of times, hiding behind the mighty fortresses of secured buildings in Afghanistan while the hired contractor employees, earning good salaries, risked much.

However, most people I knew who were hands-on (Afghans, Americans, USG employees, international employees), worked hard for a new future for all Afghans, and risked their lives. I didn’t know a lot of them, but I knew a few, who hailed from all over the world. From Colombia to Nepal.

At the time, while I did not visit Facebook that often, I had felt the need to share the overwhelming sadness I felt about Afghanistan.

Let us not forget the ugly corruption surrounding everything we worked on. What else is new? Corruption affects everything and everyone. Here, there and everywhere. The only difference is the Rule of Law as it is meant to be. Justice meted fairly for all. It is corroding around us nowadays. Hopefully, we can help save it the way we should.

Inevitably, the Fall of Afghanistan all ended up being an internecine battle here in the US, which I found not only revolting at that time, but it triggered an anger I have seldom felt.

I leave you with a quote from an email I sent from Kabul to family and friends on February of 2012:

“The snow makes the place more picturesque, but it is grim. I can handle most anything, except seeing the burqa-clad beggars sitting on the side of the roads, in the slush, getting soaked.

Yesterday, I attended for a brief period one of the classes set up for 38 judges, prosecutors, lawyers and investigators. It was fascinating. The Afghans, though loquacious, don’t engage in screaming matches like the Iraqis did. I find that my silver head amuses them a bit.

I am humbled by all the American and international advisors here who work under dire conditions. No one from the outside really knows all the work that these guys are doing. Will all these efforts yield fruit? Or will the country collapse into civil war after we pull out?

It is interesting to get feedback from these advisors who have been in remote locations. They all love working with the Afghans, although they realize that the common refrain here is: “brother against brother, brothers against father, family against tribe, tribe against tribe, tribes against country, country against the world”.”

Rule of Law and Lessons Learnt

The Special Inspector General for Afghanistan Reconstruction (SIGAR) had released at the end of May 2018 its latest “lessons learnt” report covering the period 2002-2017.  It was chock full of information for anyone who is interested in “development” work. For those who follow “rule of law” issues, the report still strikes, at least to me, a very familiar chord.

In my many years of working in international settings doing “development” work, I have found that one of the biggest problems is overcoming individual egos and the posturing that comes with those egos.  “Development” work is not just altruistic:  there is a lot of money to be made and prestige to be gained.  There is a door that is always “revolving” between the implementer, the donor, the supervising entity, the inspecting authority, the academicians, and other intellectuals.  It is human nature.

Once in a while, it is good to read that some of the lonely and knowledgeable individuals who point out some of the flaws in design are vindicated.   Note that I don’t use the term “expert”, because, in my work, most experts are really “Rupertos the “expertos””.  (My own label for the a few years was “subject matter expert”!).

Ideally, the genesis of an international Rule of Law “development” project should entail the meetings of the minds between the donor and the beneficiary.  Some of us like to say that there has to be “buy-in” from the host government as well as the individual host organizations that might be involved in the project.  This would show the cooperative efforts of all concerned towards what could be the key goals of the mission:  strengthening of the rule of law and the fostering of  accountability and transparency. 

For example, a specific justice sector program’s mission might be to build the capacity of the host country’s criminal justice sector institutions through improving the ability of their professional staff to deliver fair and effective justice services to citizens.  Sometimes, the initial focus is to help a country build its police and prosecutorial capacity through formal training programs of academic instruction.

What I have discovered is that what the donor organization (which may include the program implementer as well) wants to see happen may not necessarily be best suited to the way the project should be carried out. 

One does not become an international development practitioner overnight.  While an expatriate technical advisor may have stellar credentials from his or her prosecutorial days in their particular state or country, they may have never lived in a different environment other than their own.  This can lead to disastrous results because there is a lack of understanding of, and maybe a lack of empathy for, the recipient of the technical advice.

A long time ago, I witnessed a foreign “expert” deliver a lecture on American jurisprudence and individual rights to an academic group in what was then the U.S.S.R. , a communist/socialist country.  The audience was barely curious and did not seem to engage.  What the “expert” did not realize was the group’s lack of understanding of what he believed were common concepts, until someone asked “what is the right to privacy?”  Once it became obvious that there had been such a gulf between the lecturer and the trainees, the “expert” was able to correct the situation and begin to provide examples that the local nationals could finally relate to!

I was at the very beginning of my professional “Rule of Law” work when this happened, and it was a fine lesson for me too:  borrowing from the Spaniards, there are many “Ruperto el Experto” types, but few that meet the “experto crede Ruperto”  standard.

“I’d Give the Devil the Benefit of Law for My Own Safety Sake.”

No photo description available.

Except for a brief moment when I was a 1st grader and wanted to be either a teacher, a grocery store sales person, a fantastically wonderful Scottish sword dancer, a singer, or -more importantly- an amazing classical guitarist, I realized after I turned 7 years old that I wanted to be a lawyer. Family history and lore are partly to blame. However, I recall I always argued both sides of the coin… I loved sophistry then, the what ifs, and the what thens…without understanding the ugliness behind that word, for I had yet to meet the nasty sophists of legend.

Once upon a time and long ago, there was a wonderful movie that made an impact on me because of the philosophical argument that you give the devil the same rights that you give righteous people. An argument that helped me understand why it is necessary to provide legal counsel to the vilest and most despicable of defendants.

I am always asked this question: why would any lawyer defend the despicable? And, in all honesty, I always struggled with the answer: but no more!

I eventually began to understand the famous dialogue between Thomas More and his son-in-law Roper and the pragmatic side of the law. It may not seem fair nor kind, but we need to have structures that protect the devil so as to defend the rest of us, meek and strong alike.

Every Easter, which for some of us is a time of deep reflection, I have always remembered a special moment.

One day, in 2013, my sister Cynthia and I ended up staring at a golden cross. A crucifix that transcended history. Here was a cross, with a relic in it, that St. Thomas More had in his chapel, on his desk, and which he may have taken to the Tower of London: the patron saint of adopted children, lawyers, statesmen, politicians, and widowers would soon be beheaded. That was almost 5 centuries ago, and I was amazed to be looking at the cross that meant so much to the Englishman.

Today, I marvel at the fact that he is still studied and revered by many, even if they don’t share the same faith, because he was known to be a man of principles, honesty, and who considered it his duty to protect the integrity of his reputation.

Amazingly, even the Church of England nowadays remembers this man, commemorating July 6, as the “Thomas More, Scholar, and John Fisher, Bishop of Rochester, Reformation Martyrs of 1535” day.

What a story!

You can read the fabulous story written by Peter Ackroyd, The Life of Thomas More, which is a jewel of a book, reviewed in the New York Times in 1998.

(Originally published in 2013. Updated).