What do intelligence agencies mean when they express a ‘high confidence level’ in an intelligence finding?

In an interview with former CIA Acting Director Michael Morell, Suzanne Kelly asks: With the understanding that sources and methods need to be kept secret in order for an intelligence organization to be able to effectively do its job, can you give a sense of how rigorous a source is vetted by an intelligence agency? It’s not like they are taking the first thing they hear and calling it intelligence, right? Can you give us an idea of how rigorously information is checked before it is presented to the President?

Morell: The analytic process itself is fact-based. It’s rigorous from the perspective of the analyst who is doing the work, and it is, as you know, reviewed by a large number of people, including other analysts in the agency in which you’re working, other analysts in other intelligence community agencies, as well as your superiors. In the case of a significant judgment like the one we’re talking about, it goes to the very top of the intelligence community. So I’m sure that (Director of National Intelligence) Jim Clapper, (CIA Director) John Brennan, and the other leaders of the Intelligence Community have paid very close attention and have looked very closely at the judgments and how they were arrived at and asked a lot of questions, sent people back to the drawing board to look at this or look at that, so that’s point number one.

Point number two is, since the Iraq war, the Intelligence Community has put a huge amount of focus on stating their level of confidence in a judgment that they make. It turns out that the real mistake in the Iraq war was not the judgment that they came to, but the fact that if they had really thought about it, the analysts would have only said that they only had low confidence in that judgment that Saddam had weapons of mass destruction. That would have been a completely different message, right?

That was a mistake, so the lesson learned from Iraq was to really focus on your level of confidence in the judgment you’re making. ‘Not only do I think its going to rain tomorrow, but I have high confidence in that,’ or ‘It’s going to rain tomorrow but you guys have to know that I only have low confidence in that.’ That has become a big focus. What really caught my attention in the leaks that came out about the CIA’s judgment about what Putin was trying to achieve in his interference in the election is that the analysts applied ‘high confidence’ to that judgment. What that says to me, because we don’t attach high confidence levels to just any judgment, very few judgments actually have a high confidence level, so to get that, you have to have more than one source of data. I think we’re looking at multiple sources of data here, and you have to have something that is stronger than just a circumstantial case. I think you have to have some direct evidence, so I think we have some direct evidence.

The stuff that’s being talked about publicly, is all stuff that doesn’t really damage sources and methods, and that’s stuff that seems to be circumstantial, right? How do you know what Russian intentions are simply from the fact that they hacked the DNC, right? It’s the stuff that takes you directly to the top and directly to Putin’s intentions that probably have very sensitive sources and methods involved, and that’s why you’re not hearing anything about them.

So when the CIA says it has high confidence that they not only interfered in the election, but they did with the intent of helping Trump and hurting Clinton, I’d put very high stock in that for the two reasons we just talked about. [Continue reading…]

Print Friendly, PDF & Email