Originally posted 03-Oct-24
Gary Klein is President of ShadowBox LLC. He pioneered the Naturalistic Decision Making (NDM) movement and helped initiate the new discipline of macrocognition.
Gary developed multiple cognitive methods and models. These include the Data/Frame Theory of sense-making, the Management by Discovery model of planning in complex settings, the Triple Path Model of insight, the Pre-Mortem method of risk assessment, Cognitive Task Analysis for uncovering the tacit knowledge that goes into decision making, and the ShadowBox Training approach for cognitive skills. He also developed the Recognition-Primed Decision (RPD) model to describe how people actually make decisions in natural settings, the Knowledge Audit for doing cognitive task analysis, the Critical Decision Method (CDM), and the Artificial Intelligence Quotient (AIQ).
For more about Gary, see Profiles in Knowledge. He created the following content which I have curated to represent his contributions to the field.
Books by Gary Klein
Titles by Gary Klein include Seeing What Others Don’t: The Remarkable Ways We Gain Insights, Sources of Power: How People Make Decisions, and Streetlights and Shadows: Searching for the Keys to Adaptive Decision Making.
A Data-Frame Theory of Sensemaking
Below is a diagram of Gary Klein’s Data-Frame Theory of Sensemaking.
Spotting Exaggerated Claims for AI Superiority Over Experts
- Many artificial intelligence developers are prone to dismiss human expertise or misunderstand it.
- Artificial intelligence/machine learning proponents tend to overstate the strengths of machines over humans.
- AI/ML projects can be useful for augmenting human expertise.
- Background: Machine Learning for treating cardiac problems in the emergency department (ED)
Tips for Assessing AI/ML Demonstrations
First, pay attention to a learning confound. The ML model was designed to learn from the data. But the physicians never got any chance to learn. This doesn’t seem like a fair comparison. ED physicians actually don’t get very much feedback on the patients they see that shift. Test results may not arrive until the next shift. Many patients aren’t diagnosed during their stay in the ED. They are released or admitted to the hospital and the ED physicians don’t learn what happened unless they take the effort to investigate afterwards — pretty unlikely. Physicians certainly don’t get a 30-day follow-up.
Second, don’t take “biases” at face value. In this study, the ED physicians were using the checklists and training and common knowledge they had, but they weren’t using the 16,405 variables that the ML algorithm had — variables the physicians were never shown. Of course the ED physicians were using available and representative information. These heuristics aren’t biases; they are generally useful. Without them the physicians would be helpless.
Third, look out for smuggled expertise. The ML algorithm used information from the electronic health record, and I suspect that a lot of this information reflected judgment calls based on the expertise of the staff. The ML algorithm was building on the expertise of ED physicians and staff, not substituting for it.
Fourth, don’t be cowed by big-data types of analysis. The ML algorithm grew to 16,405 variables. However, in predicting risk, the empirical optimum was 224 variables — still, a lot. Yet there was a plateau at only about 20 variables — not that many. Twenty variables seems well within the range for training ED physicians to be more accurate.
6 Questions to Size up Your Team’s Dynamics
- Cognitive diversity matters to a team’s performance.
- Teams can try to appraise their own cognitive diversity with a rapid self-test.
- The primary value of this self-test is to gauge where people are aligned and why they may not be.
Five dimensions that seem to matter in a workplace:
- Dimension 1: Reactions to novel ideas
- Dimension 2: General mindset or stance (contrarian, trend analyst, disruptor)
- Dimension 3: Knowledge and experience
- Dimension 4: Preference for working collaboratively or independently
- Dimension 5: Tolerance for uncertainty
Self-Test for Cognitive Diversity
What is your emotional reaction when someone in a meeting makes the following statement:
- “Now, this may sound strange to you, but …” (Are you curious or skeptical? This item relates to Dimension 1, reactions to novel ideas.)
- “We know from the past that in these situations …” (Are you curious or skeptical? This item relates to Dimension 2, the different mindsets and stances.)
- “I know this looks complicated, but here is what my research has shown …” (Are you relieved or dubious? This item also taps into Dimension 2, the different mindsets and stances.)
- “Let’s just get everyone around a table and hash this out.” (Are you enthusiastic or impatient? This item relates to Dimension 4, preference for working collaboratively vs independently.)
- “We’ve been studying this forever, let’s just take a position here and move on.” (Are you relieved or uncomfortable? This item relates to Dimension 5, tolerance for uncertainty.)
- When the team is just about to converge on a course of action, someone says, “But there’s another way to think about this …” (Are you curious or irritated? This item also relates to Dimension 5, the need for closure.)
How Decision-Makers Can Handle Uncertainty
- Uncertainty is the norm for decision-makers.
- Uncertainty can stem from missing information, ambiguity, unreliable data, or contextual complexity.
- For making decisions, analytical methods that work so well with clearly defined data elements become less useful in the face of uncertainty.
What can decision-makers do?
- Accept uncertainty as the norm.
- Don’t keep an open mind about what is going on.
- Don’t fall into the information-gathering trap.
- Don’t fall into the fixation trap.
- Look for anomalies.
- Engage in story-building.
- Develop flexible plans that do not depend on one specific set of circumstances but will work in various situations.
- Take action to reduce uncertainty.
- There are a few ways to strengthen your expertise.
- Wake up from rationalist fever dreams.
