Analytic Tradecraft Standards in an Age of AI

A review of Gerald M. McMahon's article for the Belfer Center

Part of what I do, which is only somewhat visible here, is that I read a diverse range of topics at varying levels of attention. I write about a variety of things that are rightly described as tradecraft - the mixture of science, creativity, and current practices in a given area. There are roughly three overlapping topics:

• Field actor & case officer skills in the main feed.

• Adversary Resistant Computing & Networking in Tool Time.

• Analytical tradecraft services & tools in Disinfodrome and Tool Time.

There isn’t a specific area that’s just analytical tradecraft itself. The reason being that at the level were most irregular groups are, there’s typically at least one record keeper type, and their institutional knowledge shapes the groups understanding of their environment. The more formal approach is too much … but there IS a Tradecraft shelf on my Bookwyrm, and I try to periodically expand in this area.

This week I did that by reading Analytic Tradecraft Standards in an Age of AI. Let’s have a look at what’s in there.

Attention Conservation Notice:

This is a post only an intel analysis nerd could love …

Intelligence Failures:

Part of the reason I read this is that it’s an area of interest, but another part is the October 7th Hamas attack on Israel. This event is being described as an “intelligence failure”. That phrase holds a very specific meaning for intelligence professionals, it’s equivalent to the Secret Service’s performance review in December of 1963 - a profound failure to perform primary tasking resulting in a catastrophic loss.

The Sherman Kent School for Intelligence Analysis produces Occasional Papers, monographs on analytical tradecraft from the CIA internal training effort named after the father of intelligence analysis, Sherman Kent. These used to be on a single page, but now they’re part of the general archive, so you’ll have to dig to get to them. A significant portion of these papers focus on “analytical pathology” - teasing apart what happened when the U.S. intelligence enterprise missed something big.

So here on Substack there’s been a lot of talk about Hamas, Gaza, Likkud, Israel, and the obvious INTELLIGENCE FAILURE on the part of the Israel intel services. Some people have argued that there’s a dependence between Bibi and Hamas, that he let the attack happen. I think that’s plausible, but the scope of the attack was unexpected, and that is the fundamental issue.

The United States had its INTELLIGENCE FAILURE twenty three years ago, which I wrote about in Spilled Milk.

The intense review of that failure led to an addition to the Intelligence Community Directives - the mildly named ICD203 Analytic Standards, which includes a subsection entitled Analytical Tradecraft Standards.

ICD 203 Analytical Standards & Analytical Tradecraft Standards:

This is the relevant text from the document, so you guys don’t have to wade through the PDF. This is long and verbose, the AI stuff will be in the next section.

• a. Objective: Analysts must perform their functions with objectivity and with awareness of their own assumptions and reasoning. They must employ reasoning techniques and practical mechanisms that reveal and mitigate bias. Analysts should be alert to influence by existing analytic positions or judgments and must consider alternative perspectives and contrary information. Analysis should not be unduly constrained by previous judgments when new developments indicate a modification is necessary.

• b. Independent of political consideration: Analytic assessments must not be distorted by, nor shaped for, advocacy of a particular audience, agenda, or policy viewpoint. Analytic judgments must not be influenced by the force of preference for a particular policy.

• c. Timely: Analysis must be disseminated in time for it to be actionable by customers. Analytic elements have the responsibility to be continually aware of events of intelligence interest, of customer activities and schedules, and of intelligence requirements and priorities, in order to provide useful analysis at the right time.

• d. Based on all available sources of intelligence information: Analysis should be informed by all relevant information available. Analytic elements should identify and address critical information gaps and work with collection activities and data providers to develop access and collection strategies.

• e. Implements and exhibits Analytic Tradecraft Standards, specifically:

• (1) Properly describes quality and credibility of underlying sources, data, and methodologies: Analytic products should identify underlying sources and methodologies upon which judgments are based, and use source descriptors in accordance with ICD 206, Sourcing Requirements for Disseminated Analytic Products, to describe factors affecting source quality and credibility. Such factors can include accuracy and completeness, possible denial and deception, age and continued currency of information, and technical elements of collection as well as source access, validation, motivation, possible bias, or expertise. Source summary IC' 203 statements, described in ICD 206, are strongly encouraged and should be used to provide a holistic assessment of the strengths or weaknesses in the source base and explain which sources are most important to key analytic judgments.

• (2) Properly expresses and explains uncertainties associated with major analytic judgments: Analytic products should indicate and explain the basis for the uncertainties associated with major analytic judgments, specifically the likelihood of occurrence of an event or development, and the analyst’s confidence in the basis for this judgment. Degrees of likelihood encompass a full spectrum from remote to nearly certain. Analysts’ confidence in an assessment or judgment may be based on the logic and evidentiary base that underpin it, including the quantity and quality of source material, and their understanding of the topic. Analytic products should note causes of uncertainty (e.g., type, currency, and amount of information, knowledge gaps, and the nature of the issue) and explain how uncertainties affect analysis (e.g., to what degree and how a judgment depends on assumptions). As appropriate, products should identify indicators that would alter the levels of uncertainty for major analytic judgments. Consistency in the terms used and the supporting information and logic advanced is critical to success in expressing uncertainty, regardless of whether likelihood or confidence expressions are used

• (a) For expressions of likelihood or probability, an analytic product must use one of the following sets of terms: Analysts are strongly encouraged not to mix terms from different rows. Products that do mix terms must include a disclaimer clearly noting the terms indicate the same assessment of probability. almost no chance very unlikely unlikely roughly even chance likely very likely almost certain(ly) remote highly improbable improbable (improbably) roughly even odds probable (probably) highly probable nearly certain 01-05% 05-20% 20-45% 45-55% 55-80% 80-95% 95-99%

• (b) To avoid confusion, products that express an analyst’s confidence in an assessment or judgment using a “confidence level” (e.g., “high confidence”) must not combine a confidence level and a degree of likelihood, which refers to an event or development, in the same sentence.

• (3) Properly distinguishes between underlying intelligence information and analysts’ assumptions and judgments: Analytic products should clearly distinguish statements that convey underlying intelligence information used in analysis from statements that convey assumptions or judgments. Assumptions are defined as suppositions used to frame or support an argument; assumptions affect analytic interpretation of underlying intelligence information. Judgments are defined as conclusions based on underlying intelligence information, analysis, and assumptions. Products should state assumptions explicitly when they serve as the linchpin of an argument or when they bridge key information gaps. Products should explain the implications for judgments if assumptions prove to be incorrect. Products also should, as appropriate, identify indicators that, if detected, would alter judgments.

• (4) Incorporates analysis of alternatives: Analysis of alternatives is the systematic evaluation of differing hypotheses to explain events or phenomena, explore near-term outcomes, and imagine possible futures to mitigate surprise and risk. Analytic products should identify and assess plausible alternative hypotheses. This is particularly important when major judgments must contend with significant uncertainties, or complexity (e.g., forecasting future trends), or when low probability events could produce high-impact results. In discussing alternatives, products should address factors such as associated assumptions, likelihood, or implications related to U.S. interests. Products also should identify indicators that, if detected, would affect the likelihood of identified alternatives.

• (5) Demonstrates customer relevance and addresses implications: Analytic products should provide information and insight on issues relevant to the customers of U.S. intelligence and address the implications of the information and analysis they provide. Products should add value by addressing prospects, context, threats, or factors affecting opportunities for action.

• (6) Uses clear and logical argumentation: Analytic products should present a clear main analytic message up front. Products containing multiple judgments should have a main analytic message that is drawn collectively from those judgments. All analytic judgments should be effectively supported by relevant intelligence information and coherent reasoning. Language and syntax should convey meaning unambiguously. Products should be internally consistent and acknowledge significant supporting and contrary information affecting judgments.

• (7) Explains change to or consistency of analytic judgments: Analytic products should state how their major judgments on a topic are consistent with or represent a change from those in previously published analysis, or represent initial coverage of a topic. Products need not be lengthy or detailed in explaining change or consistency. They should avoid using boilerplate language, however, and should make clear how new information or different reasoning led to the judgments expressed in them. Recurrent products such as daily crisis reports should note any changes in judgments; absent changes, recurrent products need not confirm consistency with previous editions. Significant differences in analytic judgment, such as between two IC analytic elements, should be fully considered and brought to the attention of customers.

• (8) Makes accurate judgments and assessments: Analytic products should apply expertise and logic to make the most accurate judgments and assessments possible, based on the information available and known information gaps. In doing so, analytic products should present all judgments that would be useful to customers, and should not avoid difficult judgments in order to minimize the risk of being wrong. Inherent to the concept of accuracy is that the analytic message a customer receives should be the one the analyst intended to send. Therefore, analytic products should express judgments as clearly and precisely as possible, reducing ambiguity by addressing the likelihood, timing, and nature of the outcome or development. Clarity of meaning permits assessment for accuracy when all necessary information is available.

• (9) Incorporates effective visual information where appropriate: Analytic products should incorporate visual information to clarify an analytic message and to complement or enhance the presentation of data and analysis. In particular, visual presentations should be used when information or concepts (e.g., spatial or temporal relationships) can be conveyed better in graphic form (e.g., tables, flow charts, images) than in written text. Visual information may range from plain presentation of intelligence information to interactive displays for complex information and analytic concepts. All of the content in an analytic product may be presented visually. Visual information should always be clear and pertinent to the product’s subject. Analytic content in visual information should also adhere to other analytic tradecraft standards.

Artificial Intelligence Implications:

The fundamentals of McMahon’s paper are what everyone is wrestling with - LLM’s only SEEM to know what they’re talking about. AI tools can enhance the output of human analysts, but if the demand for analytical products exceeds human capability and AI becomes the author, are we trading reality for a pleasing simulacrum?

There are two Analytical Tradecraft Standards that are singled out for attention.

ATS 3 – Properly distinguishes between underlying intelligence information and analysts’ assumptions and judgments.

ATS 4 – Incorporates analysis of alternatives.

McMahon addresses what I will describe as the “incidental deception” that could arise from an analyst leaning too much on artificial intelligence in authoring conclusions. This is very different from dealing with “intentional deception” that enters the raw intel during the collection phase - instead of a shim layer that cuts out junk before the analyst even sees it, there’s a potential for some AI pathology to drop right into the middle of production. If the tool was formerly accurate and somehow develops problematic behavior, but the analyst has come to trust it … what then?

Localization:

AI is a big picture issue for our intelligence enterprise in general, but it’s also a thing right here in Happy Valley. I wrote Artificial Intelligence For Disinfodrome six months ago … and the demands of each day have piled up to yield no substantive forward movement. Part of that was covered two months later in Artificial Intelligence vs. Search Engines. Given that Disinfodrome exists to handle large tranches of documents and other voluminous information, this is an obvious place to employ Retrieval Augmented Generation. Given that I’m a network analysis nerd, I want something like Dify.ai, but that integrates a GraphRAG solution.

What concerns McMahon at the national level just isn’t an issue around here - I’m not going to have AI writing for me, I’m going to have it as a research librarian. What happens today with Open Semantic Search and Datasette will give way to some sort of ability to chat with the document stores, with the final result being a more focused set of things to review before writing.

There isn’t a Dify.ai type construction kit that brings GraphRAG to the game yet, at least not that I have seen. I could turn my integrator skills on this problem and get some sort of solution, but the creation and maintenance would be a burden, and it would quickly be eclipsed by the efforts of people much smarter/better funded than me. I am not a sit and wait kinda guy, but that’s been the right thing to do in this area.

Conclusion:

AI has been a plot device for all sorts of ad hoc futurism and it IS going to get into everything. Thus far around here it’s replaced the frisky college junior intern that codes in Python (whom I could not afford), it reads Stack Exchange for me, getting closer to what I actually want, and it’s been a nice memory jog device.

*struggling to put perceptions into some concise words*

Nationally AI is going to elevate those who have access and a way to use it in a productive fashion. It’s going to shake off masses of lower tier information economy employees. Those people will need to find other things to do and as a society we’re under demographic pressure, but I wonder how many white collar workers are going to want to retrain into something that involves working out in the heat. Overall things like that have been good in the past for our collective economy, but terrible for the individual workers.

And the thing I keep coming back to …

How does AI expand the scope of individuals displaying hacker style “wild talent”, and how does it further enable those who already have some prodigious, uncanny skill?

I can not point to any specific episode yet, but have a look at Meliorator Social Botnets. We’ve been talking about these features for years, and it’s interesting to finally see proof a nation state actor already did this. I know there are hard cyber things happening in this area, frameworks that will drive intrusions.

Hackers were already a wildcard the machine had trouble dealing with - now the IC is getting pressure from within and from the outside. This is … like the difference between armor and aircraft use in WWII versus today. Things are going to be automated and they’re going to happen faster than humans can react.

It’s a brave new world … one for which I do not feel prepared … and if it’s hard for me … I guess everybody better brace for impact.