Realistic Expectations About the Intelligence Community

January 7, 2010
Jim Arkedis



Jim Arkedis is the director of PPI's National Security Project.

by Jim Arkedis

In today’s Washington Postan anonymous intelligence official talked about the intelligence community’s role in the attempted Christmas bombing:

Anyone who believes that a relatively small organization like NCTC [National Counterterrorism Center] is going to connect every electron in each of those 30 databases is either disingenuous or naive, and certainly knows very little about how intelligence analysis actually works.

Bingo! We as a public have to reorient our expectations about the intelligence community’s ability to ensure 100 percent security on a 24/7/365 basis. That’s not a knock on intelligence pros. As a former intel analyst, I’ve seen time and time again just how unrealistic the expectations are.

While individual quotes that dumb down the intelligence community’s capabilities are illustrative, they fail to drive home how difficult intel work really is. I think it’s more useful to examine what it’s actually like to “connect the dots” in the case of a potential terrorist operative. From my own experience, here’s how it works:

I’d receive a lead from the CIA Station in Rabat, Morocco, about a potential bad guy. For the purposes of this explanation, let’s say he’s a Moroccan named Abdul Aziz Mohammed Abu Sayaf, but I don’t know anything else about him, such as his date or place of birth. (I chose that name not because I want to stereotype all “terrorists” as Arab or because he’s an actual bad guy, but because – as I explain in detail below – it will help illustrate a point about transliteration’s role in analysis of suspected terrorists specifically from Muslim countries.)

My goal is to find out everything we know about this individual and determine whether he’s a legitimate threat. This is no small point — in order to raise the alarm, I need definitive intelligence corroboration that the individual in question has a reported history that solidifies him as a potential danger. In other words, we don’t just arrest people because of a single report from a source of unknown quality. For the record, 99 percent of the time, walk-in sources to U.S. Embassies are of poor-to-unknown quality. That includes friends and family members who walk into the embassy and claim their relatives are potential dangers. Why? Family relations are tangled webs, and who really knows if your uncle just might want you arrested in revenge for that unsettled family land dispute.

Therefore, I’ll take his name and plug it into NCTC’s terrorism search, a database that stores more information about terrorism suspects than you could ever imagine. Most of the information is contained in reports from the CIA, NSA, DoD, State Department, and foreign intelligence services that have shared with us. The reports range in length from just a paragraph or two about a specific individual, to tens upon tens of pages long of names, aliases, and birth dates of “suspected” individuals about whom these suspicions are undefined (thank the Italians for this).

“Abdul Aziz Mohammed Abu Sayaf” goes in the old database, and presto-changeo, 27 reports come back. I tear through them for information that matches what I know about my guy. Say I can throw out 22 of the reports because they’re all about an “Abul Aziz Mohammed Abu Sayaf” who lives in Indonesia and was arrested in 2004 and is now in jail.

That leaves five reports. Four are about an Egyptian. Out. And the last one is about some guy of the same name in an unknown country who doesn’t appear to have really done anything wrong. I’m interested in the last one, but need much information on him before taking action.

Here’s where it gets fun. Since there may be more information out there, I start looking for variations of Abul Aziz Mohammed Abu Sayaf’s name, as names like Aziz, Mohammed, and Sayaf can be spelled several different ways when transliterated into English from Arabic. But rather than guess at which combination of the spellings works in our guy’s case, I would enter into the database, “Abdul Aziz* M*h*m*d Abu Say*af*,” which accounts for the different vowels and multiple consonants that may be used in variant spellings.

The result? 2,453 new reports to comb through!

I would logically cut that number down by entering what little other information I know about this guy. Next search: “Abdul Aziz* M*h*m*d Abu Say*af* AND Morocco.” Down to 372. Next search: “Abdul Aziz* M*h*m*d Abu Say*af* AND Morocco adj! 20,” which means all of the above words must appear within 20 words of one another. Down to 87.

I diligently read or skim through all the 87 reports looking for any nugget of information that could corroborate the suspicions about our man. Perhaps I find an additional report or two about an individual who might be the person in question, but I can only say that with 50 percent confidence.

The end result is that I write another report saying only what I can definitively conclude:

Abdul Aziz Mohammed Abu Sayaf is suspected of wanting to enter the United States to conduct a terrorist attack. Sources of unknown quality indicate Abu Sayaf is interested in traveling this month, though it remains unknown whether Abdul Aziz Mohammed Abu Sayaf is a credible threat to the United States.

I file my report, and the receiving officer – given limited resources to follow leads – deems my report interesting, but not urgent.

Two days later, an individual named Abdull-Aziz Muhammad Abou Sayyaff buys a ticket on a flight to Newark and tries to detonate an explosive belt on board. With hindsight, it’s easy to point out the flaws in my analytic process: Should the name spelling be uniform? Why did you limit your search so much? This is national security – you mean to tell me you can’t be bothered to read 327 reports? Shouldn’t we chase down every lead?  And etc… sigh.

These are easy and obvious criticisms. And certainly, some improvements can and will continue to be made. However, given the vast amount of American and internationally derived information, the pressing need to run down several searches like this on any work day, and the permanent resource constraints, these are also criticisms by those who don’t understand the tremendous complexity of intelligence work and the diminishing marginal returns of hiring thousands more additional analysts.

In short, finding bad guys is often like looking for grains of sugar on a beach. Unfortunately, we have to accept that we might not find them all.

  • Share/Bookmark

10 Responses to “Realistic Expectations About the Intelligence Community”

  1. It seems that the suspect should be put on a no fly list immediately and then the intel. officer can write another report. The suspect is not a U.S. citizena and therefore has no absolute right to fly here. That seems to protect us alot more than just another report and it is simple!

  2. [...] did an excellent post last week based on his work as an intelligence analyst walking you through how difficult it is is actually “connect the dots” and find a bad guy. The mathematical fact underlying the [...]

  3. S says:

    I worked on naming conventions and search technology from the visa side of things at State for a while. THANK YOU for explaining this cogently in a way that my non-Fed family and friends can understand!

  4. johvance says:

    @Herman: That’s an _awesome_ idea. If it is ever implemented, please tell me where you work. If you’re in a business that’s competing with mine (which is in Europe) you’re going to realize that none of your business partners will be unable to enter the United States for some time because of unfounded suspicions voiced by a guy in an embassy in Moldavia. Good luck getting them off a secret no-fly list with no judicial oversight.

  5. [...] intelligence failures re: the christmas bomber, and examining them more carefully. in particular: Jim Arkedis, a former intelligence analyst: “For the record, 99 percent of the time, walk-in sources to [...]

  6. ORLY says:

    An informative post for sure, but it’s pretty depressing that our intelligence DB technology is this bad at dealing with related and substitute search terms. Manual wildcarding FTL.

  7. Thanks for posting, I very much enjoyed your newest post. I think you should post more often, you obviously have talent for blogging!

  8. [...] Jim Arkedis, a former intelligence analyst: “For the record, 99 percent of the time, walk-in sources to U.S. Embassies are of poor-to-unknown quality. That includes friends and family members who walk into the embassy and claim their relatives are potential dangers. Why? Family relations are tangled webs, and who really knows if your uncle just might want you arrested in revenge for that unsettled family land dispute.” [...]

  9. Interesting. I develop dictionaries for languages without a standard spelling, and accommodate this by having the software itself make spellings regular in such a way that the word sought after will be found, even tough the exact spelling is not known. This isn’t very difficult and could be implemented in a couple of days (but would result in a lot of extra matches, as mentioned.)

    Similarly, if you’ve ever looked up a name in an Indian telephone book, you will see all possible variants of names cross-referenced to a single spelling, where all users of the name (the same in the original script) are grouped together.

    I remember the case of a Jordanian citizen, who found himself with a different transcription of his name on his passport every time he renewed it, forcing him to go through some trouble with US immigration to get the details on his green card updated. (Until he finally ended up applying for citizenship.)

Leave a Reply