(PDF) Improving Intelligence Analysis by Looking to the Medical Profession


shared this story

Improving Intelligence Analysis by
Looking to the Medical Profession
Intelligence agencies might benefit from assessing existing medical practices
for possible use in improving the accuracy of intelligence analysis and its
incorporation into policymaking. The processes used by the medical
profession to ensure diagnostic accuracy may provide specific models for
Intelligence Community use that could improve the accuracy of analytic
procedures. The medical profession’s way of accumulation, organization,
and use of information for purposes of decisionmaking could also provide
a model for the national security field to adopt in its quest for more
effective means of information transfer. Some limitations to the analogy
areinevitableduetointrinsicdifferences between the fields, but the study
of medicine could provide intelligence practitioners with a valuable source
of insight into various reforms with the potential to improve the craft of
The analogy between medical diagnosis and intelligence analysis has been a
thin thread running through the intelligence literature. In 1983, historian
Walter Laqueur, in ‘‘The Question of Judgment: Intelligence and
examined the analogy at a general level. He argued that
Stephen Marrin is a doctoral candidate in the Woodrow Wilson Department of
Politics at the University of Virginia, specializing in the study of intelligence. He
previously served as an analyst with the Central Intelligence Agency and
subsequently with the Congressional Government Accountability Office (GAO).
Jonathan D. Clemente, M.D., is a physician in private practice in Charlotte, North
Carolina. He is currently writing a scholarly history of the United States medical
intelligence program and medical support for clandestine operations from World
International Journal of Intelligence and CounterIntelligence, 18: 707–729, 2005
Copyright #Taylor & Francis Inc.
ISSN: 0885-0607 print=1521-0561 online
DOI: 10.1080/08850600590945434
medicine is more an art than a science because the process of diagnosis entails
the use of judgment as a means to address ambiguous signs and symptoms.
Laqueur also highlighted similarities between medicine and intelligence. For
example, in citing advances in medical technology he said it was ‘‘precisely
because of such progress [that] the similarity in concept between medicine
and intelligence has become more obvious.’’
He noted that ‘‘the
similarities extend to both collection and analysis, or in the case of
medicine, diagnosis.’’
In addition, Laqueur emphasized similarities in
analytic processes, pointing out that ‘‘the student of intelligence will profit
more from contemplating the principles of medical diagnosis than
immersing himself in any other field. The doctor and the analyst have to
collect and evaluate the evidence about phenomena frequently not
amenable to direct observation. This is done on the basis of indications,
signs, and symptoms. The same approach applies to intelligence.’’
Many aspects of intelligence practice can be found in medicine, including a
parallel to the steps in the intelligence cycle. Just as in intelligence, medical
practice includes tasking, collection, analysis, and dissemination. Consider
the case where a patient presents a ‘‘chief complaint’’ and asks the
physician to come up with a diagnosis and appropriate course of
treatment. The physician assembles bits of raw information about the
‘‘history of present illness,’’ analyzes the data to come up with both a
reasonable differential diagnosis and a presumptive diagnosis, and provides
a course of treatment and prognosis to the patient. The cycle repeats itself
as better information becomes available, new questions arise, and the
diagnosis and definitive treatment are refined.
Unfortunately, Laqueur’s observations have not been explored at length in
over two decades. No other articles have been published on the analogy
between intelligence and medicine, and no books have addressed it at
length. This failure by both practitioners and students of intelligence to
explore the ramifications of an analogous profession is indicative of the
conceptual insularity of the intelligence discipline writ large. Security
concerns constrain the intelligence community’s ability to reach out to
external sources for ideas and insight, and, as a result, the internal
discussions that occur in intelligence circles regarding ways to improve
existing practices—the same kinds of discussions that occur in every field—
are stultified because of the limited number of ideas that can proceed
through the narrow chokepoints to the outside world.
The similarities between intelligence analysis and medical diagnosis are
obvious at first glance, with intelligence producing analysis and estimates
regarding events in foreign countries and medicine producing diagnoses
and prognoses regarding the health of individuals.
In both intelligence and
medicine, the practitioner uses similar approaches and technology to gather
data, integrates this data into an assessment of what is going on today
patterned on existing understandings of causal relationships, and then
interprets the importance of the situation and forecasts what might happen
in the future in terms useful for decisionmaking. In addition, both
intelligence analysis and medical diagnosis are vulnerable to similar causes
of inaccuracy in their respective assessments.
Parallels in Collection
Both medical and intelligence practitioners apply the same general
approaches and similar technologies to acquire information. Medical
diagnosis and patient health assessment follow a fairly standard algorithm
taught to every second-year medical student and in use since the days of
the great diagnostician Sir William Osler. Each step within this algorithm
has a specific parallel to the processes used to collect intelligence.
The diagnostic process begins with the elicitation of the ‘‘history of present
illness,’’ where the patient relates the characteristics of the specific complaint
and other subjective qualitative and quantitative features to a physician. The
physician then ascertains any relevant past medical or surgical history,
medication use, and known allergies. In the intelligence profession, this
might be roughly equivalent to the acquisition of ‘‘basic intelligence’’—i.e.,
knowledge regarding foreign countries or groups for operational planning
at any level
—in order to determine the potential significance of any recent
changes. While the patient interview is a good information source for
diagnosing a patient, as in the human intelligence process, self-reporting by
patients can be notoriously unreliable, for any of a number of reasons. As
a result, medical schools train physicians to acquire information from the
patient via what intelligence practitioners might consider an approximation
of human intelligence (HUMINT) elicitation techniques including use of
body language to ‘‘enhance rapport and reinforce continuity of
conversation,’’ appropriate uses of closed and open questioning,
minimization of jargon, and the use of positive reinforcement and silence
as ways to control the interview.
The intelligence community’s equivalent
to the ‘‘patient interview’’ might be a State Department or military attache
report of a conversation with a foreign official, or perhaps, a defector or
refugee debriefing.
The second step in the medical diagnostic process is the ‘‘review of
systems.’’ At this stage, the physician literally performs an objective
head-to-toe assessment of specific organ systems, such as the cardiovascular
and gastrointestinal systems, in order to determine whether any specific signs
or symptoms of disease are present. The penultimate step is the ‘‘physical
examination’’ of the patient, beginning with a measurement of the
acknowledged vital signs: temperature, blood pressure, pulse, heart and
respiratory rate. This hands-on assessment of the patient—checking for
swollen lymph nodes, listening to the heart, feeling the belly, checking the
reflexes—is the true art of medicine. In the intelligence field, these hands-
on checks do not have a direct equivalent for analysts, other than perhaps
overseas familiarization tours made to gain first-hand knowledge of the
country they are responsible for. A second-hand version of the physical
exam might also be intelligence cables from State Department officers or
military attache
´s, reporting on what they saw during their travels in
foreign countries.
Finally, if additional information is required, physicians then order
laboratory tests. Some tests, such as X-rays or magnetic resonance imaging
(MRI), are equivalent to imagery intelligence (IMINT),
while other tests
such as those that measure blood products or other bodily functions could
be considered the rough equivalent of measurement and signatures
intelligence (MASINT).
In addition, just as the collection systems are similar in both medicine and
intelligence, so is the discussion over the relative utility of the information
provided by each system. An active debate exists within the intelligence
field over the relative value of various collection systems in divining the
capabilities or intentions of international actors. A similar debate occurs in
the medical field. According to a popular aphorism taught to generations
of medical students, ‘‘90 percent of all diagnoses are made by the clinical
history alone, 9 percent by the physical exam, and 1 percent by laboratory
tests and imaging studies such as CT and MRI scans.’’ While the medical
profession’s use of laboratory tests and medical diagnostic imaging
modalities, such as computed tomography (CT) scans and magnetic
resonance imaging (MRI), may be increasing, they are not infallible and
often do not reveal the definitive diagnosis. Ultimately, just as IMINT
cannot provide the same insight into intentions as HUMINT, no CT scan
or MRI can replace the physician–patient relationship, the hands-on
approach, or the experience of having examined patients before. In both
intelligence and medicine, all forms of collection must work in concert for
the all-source intelligence analyst or the physician to successfully complete
their tasks.
Yet, the collection of information in either the medical or intelligence field
does not ipso facto lead the practitioner to a conclusion, and an over-
emphasis on collection in either field may lead to excessive data collection.
According to Richards Heuer, the ‘‘rationale for large technical collection
systems’’ may be rooted in the misapplication of the so-called ‘‘mosaic
theory of intelligence.’’
This theory states that a ‘‘clear picture of reality’’
results from the assemblage of numerous bits of information into a
‘‘mosaic or jigsaw puzzle’’ and implies that accurate assessments can arise
only after accumulating a complete data set. However, as Heuer points
out, research into cognitive psychology suggests the opposite. Intelligence
analysts may first form a mental picture and then find individual pieces of
information—each of which may support independent hypotheses—to
support their initial estimate of the situation. The accuracy of these
estimates, therefore, may depend on the balance between data collection
and ‘‘the mental model used in forming the picture.’’ As a result, the analytic
and diagnostic processes used in both fields are very important because they
help the practitioners create the mental models that Heuer refers to.
Parallels Between Analysis and Diagnosis
Once the various streams of information are collected, the integration process
in medicine is very similar to that which occurs in intelligence because
practitioners in both fields use approximations of the scientific method—
observation, hypothesis, experimentation, and conclusion—as a means to
organize and interpret the collected information. Many empirical or data-
driven professionals, such as detectives in the law enforcement profession
and physicians in the medical profession, use the scientific method as a
way to derive causal relationships and test hypotheses. The ultimate goal is
to derive an accurate estimate of any given situation.
As has been addressed elsewhere,
the intelligence analysis process,
though an approximation of the scientific method, does not parallel it
exactly because no experiments are possible in the international arena. Yet,
most writers who focus on analytic tradecraft—whether they realize it or
not—portray the intelligence analysis process as a version of the scientific
method. In the end, intelligence analysis entails inductive and deductive
reasoning applied in turn to find patterns among data and derive
hypotheses that explain what the data mean. Most recommendations for
improving intelligence analysis are akin to the lessons taught in graduate-
level methodology courses: use good data, prevent bias, test hypotheses
through a competitive process, etc. Analysts tend to use intuitive ‘‘pattern
and trend analysis’’—consisting of the identification of repeated behavior
over time and increases or decreases in that behavior—to uncover changes
in some aspect of international behavior that could have national security
They then apply some aspect of disciplinary theory—
political science, economics, psychology, military science—informed by
their knowledge of the history and culture of the region to derive the
implications of the change. This analytic process is very similar to the one
physicians use to diagnose their patients.
For the most part, physicians must combine the signs and symptoms into a
hypothesis informed by theory—i.e., identified patterns associated with
diseases. The ability to arrive at a correct medical diagnosis goes far beyond
merely ordering the appropriate blood tests or X-rays. This clinical skill
requires years to master. At its core it requires a solid base of working
medical knowledge, involving the interpolation and synthesis of sometimes
incongruous facts into a logical diagnosis. Fundamentally, the most
effective physicians are good listeners, capable of at once noting the
pertinent elements of the patient’s complaint, adroit at recognizing nuances
in expression, body position, and vocal inflection, and able to use these to
discern the true nature of a patient’s complaint.
When the analytic processes in medical diagnosis and intelligence analysis
are assessed side-by-side, the parallels are striking. According to the Central
Intelligence Agency’s (CIA) Richards Heuer, medical diagnosis provides a
more accurate way of describing how intelligence analysis should work
than do other analogies,
The doctor observes indicators (symptoms) of what is happening, uses his
or her specialized knowledge of how the body works to develop
hypotheses that might explain these observations, conducts tests to
collect additional information to evaluate the hypotheses, then makes a
diagnosis. This medical analogy focuses attention on the ability to
identify and evaluate all plausible hypotheses. Collection is focused
narrowly on information that will help to discriminate the relative
probability of alternate hypothesis. To the extent that this medical
analogy is the more appropriate guide to understanding the analytical
process, there are implications for the allocation of limited intelligence
resources. While analysis and collection are both important, the
medical analogy attributes more value to analysis and less to collection
than the mosaic metaphor.
Even the process of distinguishing the relevant information from the
irrelevant—also known as differentiating the signals from the noise—is
similar in both professions. The process of arriving at a medical diagnosis
requires that the physician first establish a reasonable ‘‘differential
diagnosis,’’ which often includes two or more diseases that may have
similar signs and symptoms. The task of the physician is to systematically
compare and contrast the clinical findings to determine the most likely
etiology—or cause—of the patient’s malady. Similarly, Heuer argues that
without considering all alternative hypotheses, an intelligence analyst
cannot evaluate the ‘‘diagnosticity of evidence.’’ He considers this term to
mean ‘‘the extent to which any item of evidence helps the analysts
determine the relative likelihood of alternative hypothesis.’’ So, for
example, Heuer correctly points out that ‘‘a high-temperature reading may
have great value in telling a doctor that a patient is sick, but relatively
little value in determining which illness a person is suffering from.’’
Diagnostic evidence influences one’s ‘‘judgment on the relative likelihood
of the various hypotheses’’; whereas, evidence that ‘‘seems consistent with all
the hypotheses’’ at least in the case of medicine, does not narrow the
differential diagnosis, and ‘‘may have no diagnostic value.’’
Technology and Coordination
Technological tools developed to improve the rigor and accuracy of
intelligence analysis or medical diagnosis can help analysts and physicians
weed through data and discover patterns, but are less able to assist the
analysts in interpreting the intelligence and deriving meaning and
implications. Both medical diagnosis and intelligence analysis require
judgment in interpretation of the evidence that goes above and beyond
what can be quantified or automated. The scientific method helps
intelligence analysts and physicians form hypotheses regarding the cause of
the issue at hand, but in both cases ambiguous information and
circumstances require critical thinking and judgment in order to come to
conclusions regarding the accuracy of the hypothesis and its implications
for—respectively—a nation’s interests, or the patient’s well-being. An
implication stemming from this observation is that the accuracy of the
intelligence analysis or diagnosis may rest on the cognitive abilities of the
practitioners. ‘‘The key,’’ according to Richards Heuer, ‘‘is not a simple
ability to recall facts, but the ability to recall patterns that relate facts to
each other and to broader concepts—and to employ procedures that
facilitate this process.’’
Yet, just as in intelligence analysis, medical
diagnosis is occasionally arrived at serendipitously, as when a physician
reads about some obscure disease in a medical textbook or journal the
night before a case of this disease is coincidentally seen in his clinical practice.
Complicating matters, arriving at a judgment in both intelligence and
medical fields can require the interdisciplinary coordination of various
specialists. The development of expertise in the medical field was not only
the province of individual cognition, but required the creation of
specialties and sub-specialties focused on specific functional systems such
as neurology and orthopedics. But the broader implications of this
knowledge can be lost if the contribution of the specialty is not
reintegrated into a holistic assessment of the patient’s health. This entire
dynamic parallels the analytical specialization by the CIA’s Directorate of
Intelligence according to analysts’ political, military, economic, and
leadership disciplines. In intelligence, the integration of the various
specialist perspectives can at times be difficult, especially when events
overseas appear to have multiple explanations that cross the various
disciplines. The integration of perspectives can be easy if they all point
towards one explanation, but if different intelligence disciplines or medical
specialties have different explanations, doing so can be very difficult.
The parallels between the collection and analysis of information in the
medical and intelligence fields indicate that the underlying analytic
processes are similar, but these similarities also mean that the causes of
inaccuracy in their respective fields are also parallel.
Medical diagnosis and intelligence analysis have similar causes of inaccuracy
due to their similarities in collection and analysis. They share at least three
causes of inaccuracy; they undoubtedly have many additional sources of
error in common.
First, inaccuracy in both intelligence analysis and medical diagnosis can
arise from the unavoidable limitations in the collection and analysis of
information. Both medicine and intelligence collection are subject to some
amount of both random and systematic error resulting from built-in
limitations of the collection instruments themselves, and as a result the
information that feeds into the subsequent analysis is never an exact
representation of reality. For example, the ability of modern medical
imaging modalities such as the CT and the MRI to accurately depict
anatomic structures is limited by technical constraints of spatial-temporal
resolution and signal-to-noise ratio. An equivalent in the intelligence world
could be the subjective interpretations that case officers inevitably include
in their interpretations of an asset’s reliability and the information he or
she provides. In the aggregate, these errors can combine to cause
inaccuracy on the margins of both intelligence analysis and diagnosis.
Additional inaccuracy at the analytic level compounds whatever errors
may have been incorporated during the collection of information.
As has
been pointed out elsewhere,
the analytic process itself is subject to an
individual analyst’s cognitive limitations, and as a result ‘‘analysis is
subject to many pitfalls—biases, stereotypes, mirror-imaging, simplistic
thinking, confusion between cause and effect, bureaucratic politics, group-
think, and a host of other human failings,’’ according to administrators at
the Joint Military Intelligence College.
In the medical field, one of the
most often repeated pearls of wisdom for diagnosing patients is that
‘‘uncommon manifestations of common diseases are more common than
uncommon manifestations of uncommon diseases,’’ or ‘‘when you hear
hoofbeats, look for horses and not zebras.’’ The challenge faced by many
neophyte physicians is to adhere to this medical truism. The background
noise that arises from reading about and observing a multitude of new and
unusual diseases can obscure the signals of a more workaday illness. The
same can be said for intelligence analysts as well, and controlling for
possible causes of error in analysis has become the subject of many
intelligence articles.
In addition, errors may arise in both intelligence analysis and medical
diagnosis due to problems intrinsic to the implementation of the scientific
method. The deductive approach used by practitioners in both fields
requires some inductive ability to distinguish the relevant information
(signals) from the irrelevant (noise). Generally, conceptual frameworks
built out of hypotheses that tie together a number of cause=effect
relationships are used, but distinguishing the signals can still be a difficult
task. As Walter Laqueur observes, ‘‘like the intelligence analyst, the
clinician faces the problem of detecting signals. A weak signal may be
drowned in background noise. Perhaps the most frequent of such
situations facing him occurs when taking the case history of a loquacious
patient. In each case, a post mortem shows that all the necessary
information was available but it did not register, sometimes because of an
abundance of clues, sometimes because of a temporary eclipse in
observation or critical acumen.’’
In medicine, an example of this kind of
error would be the mistaken attribution of a health problem to an
innocuous external factor that was correlated with the problem but not the
cause of it. Specifically, the long-term false attribution of peptic ulcers to
‘‘spicy food, acid, stress, and lifestyle’’ rather than the presence of a
bacteria (Helicobacter pylori or H. pylori) that ‘‘causes more than 90
percent of duodenal ulcers and up to 80 percent of gastric ulcers’’ is an
example of an error due to the complexities of distinguishing signals from
noise in a medical context.
In the intelligence arena, many possible
explanations exist for specific outcomes, such as a foreign government’s
negotiating position at an international conference, but in many cases
intelligence analysts may have difficulty determining whether the position
taken is due to underlying political forces, economic conditions, or the
agenda of a single individual or groups of individuals. Errors in the
interpretation of events are likely when the conceptual frameworks for
explaining the outcome are insufficiently specified.
Finally, errors may occur in both intelligence analysis and medical
diagnosis due to the misapplication of the scientific method. For example,
in mid-2003 the Washington Post reported that ‘‘recommended ‘best
practices’ were followed about two-thirds of the time in diagnostic
testing,’’ presumably leading to suboptimal outcomes.
The parallels to
intelligence analysis are obvious. If the practitioner does not follow
analytic tradecraft, inaccuracies could be incorporated into the analytic
process unless specific means are implemented to ensure that the
conclusions follow directly from the evidence.
Because the mechanisms used to collect and analyze information in both
fields are so similar, the causes of inaccuracy are also similar. But, deriving
lessons from analogies requires an understanding of the limits of the
analogy that are defined by the differences between the fields. In addition
to the substantial similarities between the intelligence and medical fields,
substantial differences exist as well.
Prominent differences between intelligence analysis and medical diagnosis
limit the analogy and the lessons that can be derived from it. Differences
exist in the kinds of problems that practitioners in both fields address, the
kinds of knowledge used to address them, the reliability of the information
acquired, and the use of the information in decisionmaking. Nonetheless,
their existence does not remove all utility from the analogy. In each case,
the analogy continues to hold between intelligence analysis and a subset of
the medical profession.
Differing Types of Problems
Intelligence analysts and physicians obviously address different kinds of
problems. In general, intelligence analysts assess the international
environment for changes that could affect U.S. security interests. While the
identification of threats is a part of an intelligence analyst’s responsibility,
the analyst usually has to first assess whether or not there is a threat, while
a physician’s diagnostic mission tends to be more constrained. Patients
generally seek medical attention when they have identified an existing
health problem, and look to the physician to identify its cause and
establish a course of treatment for its resolution. As a result, the
intelligence analyst’s mission is roughly equivalent to the subset of the
medical diagnostic range known as preventive medicine, where patients are
assessed for underlying health problems for which no symptoms may be
observable or identifiable. Alternatively, subsets of each medical diagnostic
and intelligence analysis specialty may deal with a comparable range of
issues. For example, intelligence analysts who track identifiable problems
over time, such as nuclear proliferation or terrorism, may be more
analogous to the physician who assesses the condition of a patient with a
chronic health problem.
Epistemological Foundations
Intelligence analysis and medical diagnosis are grounded in different
epistemological foundations, with implications for how practitioners in the
respective fields make decisions.
Specifically, the greater accumulation of
knowledge and theory in the physical sciences than in the social sciences
provides medical practitioners with a relatively larger empirical base and
more precise causal relationships, enabling them to make diagnoses and
prognoses with a greater level of certainty than their intelligence
Medical knowledge of relationships between cause and effect exists at a
high level of specificity because the development of medical science—built
on the physical sciences—has allowed practitioners to aggregate knowledge
and build a progressively larger base of information regarding the effects
of diseases and pathologies on human health. The key to this growth has
been the ability of medical science to research the causes and effects of
various diseases in laboratories where researchers can limit the influence of
extraneous factors. In addition, medical researchers use incidence rates
of disease throughout the population as a way to approximate many
‘‘experiments’’ simultaneously. Once medical researchers have identified the
pathologic or cellular basis for disease and the full range of effects on a
typical patient’s health, new physicians are taught the patterns of signs and
symptoms in medical school, and are kept updated on current research
through their continuing professional education programs. As greater
knowledge of cause and effects is accumulated, more detailed and specific
diagnoses and prognoses become possible.
By way of contrast, most causal relationships derived from the social
scientific theories of interest to intelligence analysts are still indeterminate
due to the infrequent occurrence of important events on the international
stage, and the analyst’s inability to test hypotheses through laboratory
experiments. Intelligence analysts rely primarily on social scientific theories
that explain nation-state behavior at various levels of analysis, but none of
these theories is as precise as those in the physical sciences. For example,
intelligence analysts use international relations theory to ground their
analyses at the systemic level; political science and economic theory to
ground their analyses at the state level; and psychology to ground their
analyses at the individual level. Yet, for the most part, these theories do
not provide specific identifiable patterns akin to those physicians use to
diagnose pathology, because social scientists have been unable to define
the circumstances under which the various theories can individually explain
state behavior. Economics may be the social scientific theory that most
closely resembles the physical sciences,butevenithasdifficultywith
precise explanations because of its assumptions of perfect information and
rational behavior that rarely seem to occur in the real world. As a result,
Yale University historian John Lewis Gaddis asserts that most social
science theories ‘‘tend to be parsimonious, attributing human behavior to
one or two basic ‘causes’ without recognizing that people often do things
for complicated combinations of reasons’’ and as a result are ‘‘static,
neglecting the possibility that human behavior, individually or collectively,
might change over time.’’
Gaddis concludes that as a result of these
tendencies, ‘‘the social sciences are operating at roughly the level of
freshman physics experiments [and] that’s why the forecasts they make
only occasionally correspond with the reality we subsequently
If the theories that intelligence analysts use to forecast future
events produce accurate assessments only infrequently, it is no surprise
that intelligence analysis forecasts follow a similar path.
Over time, social scientists have been able to accumulate knowledge about
the causes of larger international events—such as war or international
cooperation—but for the most part these explanations are very general and
lack the precision necessary to explain or forecast the kinds of specific
events that intelligence analysts are interested in. In medical terms,
intelligence analysts have a similar understanding of the patterns that
underlie international relations that physicians had for disease some two
centuries ago. Some social scientists have attempted to model international
relations in a similar way to the physical sciences, but these models have
been—for the most part
—found wanting for intelligence purposes. As
Walter Laqueur explains, ‘‘For a long time, military and foreign political
intelligence have tried to become scientific, or at the very least more
scientific. But, inasmuch as assessment is concerned, the outcome of a
search for a scientific theory improving the predictive capacity of
intelligence has been quite disappointing.’’
As a result, for the most part,
medical diagnoses can be made with greater precision and accuracy than
can intelligence analysis.
Nonetheless, parallels do exist between medical diagnosis and intelligence
analysis in certain areas where medical knowledge has not yet acquired
sufficient ability to understand the cause of health problems or their
impact on a patient’s health. Many diseases and genetic syndromes have
no known cause or effective treatment and are deemed ‘‘idiopathic.’’
Medical literature frequently attributes the causative agent in these
‘‘idiopathic’’ cases to either an ‘‘autoimmune disorder’’ or a virus. In other
cases, the ability to diagnose various diseases may be fraught with
uncertainty and ambiguity. In describing the unpredictable biological
behavior of a certain cancer, a major pathology reference text quips ‘‘these
tumors don’t read textbooks.’’
Pathologists are supposed to provide the
clinician with the definitive ‘‘ground truth’’ of any given disease entity, but
for one particular class of tumors a surprising degree of internal
disagreement occurs over ‘‘final pathologic diagnosis,’’ not only at the
hospital level, but on a national and international level as well. Finally, the
effect of disease on individuals is highly variable. For many years, clinical
medicine was taught based on a ‘‘hypothetical 70 kilogram white male.’’
Yet physicians recognized through anecdotal experience what is now
accepted as fact: few individuals react exactly the same way to the same
disease, or the same treatment. To diagnose the patient effectively the
physician must be aware of these differences in presentation, but the medical
profession has only recently incorporated this paradigm shift into its
therapeutic regimens. As a result, a substantial practical component to
medicine requires a combination of experience and judgment that is not
codified in any text, but is simply passed down to young physicians in the
oral tradition of the clinical wards.
In those cases where levels of uncertainty faced by practitioners in both
fields are the same, their methods for handling uncertainty are also similar.
Intelligence agencies teach analysts to use alternative forms of analysis to
handle unconventional analytic challenges. Similarly, when physicians are
not able to make a positive diagnosis immediately because of the inherent
ambiguity in medicine’s ‘‘gray areas’’—when insufficient empiric knowledge
exists or a common disease presents atypical or protean manifestations—
physicians sometimes resort to alternative diagnostic methods. For example,
physicians can treat the patient with the ‘‘tincture of time’’ or through
‘‘diagnosing by observing natural history’’ where careful, close observation
and the allowance of a short passage of time permit the true cause of the
disease to ‘‘declare’’ itself. Some medical disorders, such as ‘‘fibromyalgia,’’
are generally considered by the medical profession to be ‘‘diagnoses of
exclusion.’’ In other words, such a diagnosis should be made only after other
more common or potentially serious conditions are ruled out.
Thus, even though medicine may have a large knowledge base of
information regarding disease, enabling physicians to make accurate
diagnoses in a majority of cases, a large subset of issues persists, where the
incidence rates are low or issues are complex, and, as a result, medical
knowledge of pathological etiology and resulting signs and symptoms are
scant. In these cases, the levels of diagnostic uncertainty approximate those
faced by intelligence analysts because of the inexactness of the social
science theories they use to interpret the raw intelligence at their disposal.
Rates of Denial and Deception
Because intelligence analysis entails deciphering meaning through a more
extensive ambiguity, caused by greater denial and deception than exists in
the medical field, intelligence analysts generally labor under greater levels
of uncertainty than their medical counterparts. For example, in the
intelligence field, concern over whether foreign governments and entities
are providing disinformation through U.S. collection capabilities so as to
deceive analysts and policymakers leads to pervasive uncertainty over the
reliability of almost all information collected. These concerns complicate
the assessment and validation process since no piece of evidence can be
considered reliable without excessive scrutiny into both its substance and
the process by which it was collected.
The bulk of the medical profession does not labor under similar levels of
uncertainty resulting from denial and deception efforts on the part of
patients. As Walter Laqueur observed: ‘‘There is one important difference:
the patient usually cooperates with the medical expert; he has no incentive
to hide and to mislead.’’
As noted, in the medical field some uncertainty
is intrinsic in the assessment of information, and other concerns about
reliability can creep in, due to laboratory error or errors in patient self-
reporting, but, for the most part, the uncertainty is not due to a conscious
effort on the part of individuals to manipulate the process. For a sub-set
of cases in medicine, however, physicians may also labor under conditions
of uncertainty analogous to those in the intelligence world due to denial
and deception efforts.
In medicine, intentional deception by patients for purposes of misleading
the diagnosis are rare, but can be found in cases where the patient has an
underlying incentive to deceive. For example, physicians responsible for
making disability determinations, and for managing pain by dispensing
narcotics, can encounter patients who attempt to deceive them in order to
acquire money or narcotics. In the medical profession, this kind of
deception is known as ‘‘malingering,’’ and the underlying incentive to
deceive is known as ‘‘external or secondary gain.’’ In addition, physicians
encounter denial in circumstances where a patient is embarrassed or
unwilling to share the complete circumstances of an injury. Also, rarer still,
are cases of unintentional denial—or patient self-deception—arising from
psychological disorders, in which symptoms expressed by the patient are
not indicative of underlying health problems. These incidents could be
roughly analogous to cases where inaccurate information is possessed by
foreign governments and subsequently acquired by intelligence agencies.
Examples from medicine include Munchausen syndrome (i.e., a habitual
and intentional effort to produce convincing physical or psychological
symptoms in order to gain attention through the sick role), and
hypochondriasis (i.e., morbid anxiety about one’s health with symptoms
unattributable to organic disease).
Malingering, hysterical symptoms, and hypochondriasis can be especially
difficult to detect, in part, because of a physician’s natural reluctance to make
such a ‘‘diagnosis’’ before an actual organic illness is excluded. As a result, no
firm epidemiological data on the incidence of such ‘‘deceptive’’ conditions is
available Nevertheless, physicians are taught to recognize certain signs of
‘‘functional’’ illnesses where no anatomic or pathologic causes can be
found. For example, the diagnosis of ‘‘pseudoseizures’’ may be established
through clinical history alone, or by the absence of signs associated with
true seizure disorders. Malingering may be detected when there is an
incongruity between claimed injury and an inconsistent mechanism of
injury. Ultimately, some cases may require the performance of specialized
tests to exclude a structural problem. ‘‘Hysterical blindness’’ can be
established by performing a visual-evoked response, where a flash of light
in the eye ‘‘evokes’’ an electrical signal in the portion of the brain involved
in vision, indicating intact visual pathways. Similar tests are used by
intelligence practitioners to determine whether a government or individual
is being actively deceptive or attempting to prevent the U.S. government
from acquiring certain kinds of information.
The relatively higher levels of uncertainty in the intelligence world are due
to the greater incentive for foreign governments to deny the U.S. government
information on their activities or deceive them regarding the extent of those
activities. But the subset of cases in the medical world, where patients have
incentives to deceive, can provide analogies and perhaps even lessons that
intelligence analysts can adopt to improve their own processes.
But the analogy between intelligence analysis and medical diagnosis fails the
closer it gets to the decisionmaking process. As Walter Laqueur points out:
‘‘the comparison between medicine and intelligence cannot be carried
beyond a certain point; the doctor engages not only in diagnosis but also
in curing the patient.’’
Because most physicians are also responsible for
treating patients, they are in essence roughly equivalent to national security
decisionmakers. Yet, an in-depth examination of the distinction between
diagnosis and treatment in medicine and intelligence and decisionmaking
in foreign policy helps define the extent to which the analogy can be used
as a means of exploring alternative ways of doing business.
Assessing the importance of information within a decisionmaking process
first requires understanding how information is used by decisionmakers.
Harvard University historian Ernest May uses a simple framework to
summarize that process:
[At] any time or place, executive judgment involves answering three sets of
questions: ‘‘What is going on?’’; ‘‘So what?’’ (or ‘‘What difference does it
make?’’); and ‘‘What is to be done?’’ The better the process of executive
judgment, the more it involves asking the questions again and again, not
in set order, and testing the results until one finds a satisfactory answer
to the third question—what to do (which may be, of course, to do
In national security policymaking, an individual decisionmaker requires
information regarding international events and issues that have the
potential to affect United States national interests (what’s going on?); the
analysis and evaluation of this information (so what?); and the ability to
create and implement effective policies (what is to be done?). In medicine,
the decisionmaking process works similarly. A treating physician must first
assess the patient and diagnose the cause of any problems, then evaluate
the significance of these problems by creating a prognosis, and finally
decide on a course of action to treat the patient. But national security
decisionmaking occurs on both individual and organizational levels,
thereby greatly complicating the analogy between medical diagnosis and
intelligence analysis.
National security policymakers generally follow the decisionmaking
process laid out by Professor May, but a policymaker does not derive
information as directly from first-person experience as does a physician
from an interview and subsequent examination of the patient. Rather, in
the national security world, information is collected, filtered, analyzed,
and disseminated in an organizational context, so that any assessment of
the role that intelligence plays in national security decisionmaking must
also be grounded in an institutional context. National security
policymakers have staffs that provide them with information-acquisition,
analysis, and decisionmaking assistance. Additional similar assistance is
provided by intelligence agencies. In fact, intelligence analysts at the
CIA are trained to answer two of Professor May’s three questions by
explicitly addressing the ‘‘what’’ and the ‘‘so what’’ in their finished
intelligence analysis. However, answering the question ‘‘what is to be
done?’’ in the national security realm is prohibited for intelligence
analysts while they monitor the international environment for foreign
policymakers, and alert them to any changes that might affect national
interests. Intelligence is thus subordinate to policymaking, and resembles
the product of the type of analysts, described by Geoffrey Vickers, as
the kind who monitor the decisionmaker’s environment for any changes
and acts as a ‘‘watchdog on a chain; he can bark and alert the
householder, but he cannot bite.’’
National security decisionmakers, however, do not make decisions only
after receiving finished intelligence analysis; in many cases they are their
own analysts, and have entirely separate sources of information. Many
policymakers have access to raw intelligence reporting as well as finished
intelligence analysis; they also have separate information streams outside
the intelligence community, such as contacts in academia, think tanks,
the domestic and international business world, and foreign government
officials. As a result, the medical analogy may be a better fit for
comparing the decisionmaking processes of physicians and national
security policymakers than intelligence analysts. The eminent
international relations scholar Alexander George came to a similar
conclusion when he looked at the uses of information in foreign policy
Correct diagnosis of a policy problem and of the context in which it
occurs should precede and—as in medical practice—is usually a
prerequisite for efforts to make the best choice from among treatment
options. The analogy with the medical profession is an apt one, since
the policymaker, like the physician, acts as a clinician in striving to
make a correct diagnosis of a problem before determining how best to
prescribe for it.
But even if this analogy between physicians and policymakers works
better, physicians rely on the advice of other diagnostic experts because of
economies of scale and limitations in both time and expertise. For
example, an oncologist may be the ‘‘analyst’’ and ‘‘policymaker’’ for a
given patient, but relies on other analysts, such as the radiologist, to
identify the initial manifestations of disease, the surgeon to provide a tissue
sample, and the pathologist to give the ‘‘final answer.’’ In this framework,
the medical equivalent of an all-source intelligence analysis would be a
delegated diagnostic sub-specialty with access to most of a physician’s data
sources, including written reports of patient interviews, but no role in the
treatment decision process. This describes the role of a ‘‘consulting
physician’’ who is presented with a clinical problem outside the primary
physician’s expertise. The consultant is usually asked to review data and
formulate a diagnosis or differential diagnosis, but not necessarily to
implement treatment. One type of consulting physician is a radiologist,
who—while closer to the intelligence equivalent to an imagery analyst—
helps diagnose but does not treat, and hence, does not implement ‘‘medical
This examination of the analogy between intelligence and medicine indicates
its possible use in acquiring greater insight into intelligence processes, as well
as serving as a source of models for improving analytic processes. The
obvious similarities between intelligence analysis and medical diagnosis
indicate possible avenues for intelligence practitioners to derive lessons that
could improve analytic accuracy. For example, the processes of medical
diagnosis are vulnerable to the same pathologies that cause intelligence
failure, and techniques developed to improve the accuracy of diagnoses or
prevent malpractice based on diagnostic error may also improve the
accuracy of intelligence analysis. In 2003, a New York Times article
highlighted a team of radiologists who established a feedback process that
improves the accuracy of their diagnosis. A similar mechanism could be
used to improve the accuracy of intelligence analysis.
Alternatively, the
medical subspecialties have long relied on the monthly ‘‘morbidity and
mortality conference’’ (the ‘‘M and M’’ conference) as a forum to discuss
complications in diagnosis and treatment, and methods of preventing adverse
events and outcomes. Both minor and major complications in patient care
are discussed. Though physician participants in these regular ‘‘M&M’’
conferences often provide brutally frank assessments of their colleagues’
patient care, they are meant to be a learning tool for doctors at all stages
of their career. Perhaps the intelligence community might adopt a similar
periodic peer review process, not only to discuss ‘‘intelligence failures’’ of
the sort that makes newspaper headlines, but as a spot check on other
forms of basic and current intelligence.
In addition, each difference between intelligence analysis and medical
diagnosis conversely points to a more specific way that aspects of
intelligence analysis and medical diagnosis are similar in a subset of cases.
Lessons for the practice of intelligence analysis can be derived from each.
The medical equivalent of an all-source intelligence analyst would be a
diagnostic assistant in a preventive medicine context—possessing access to
all information that the treating physician needs—required to use
indeterminate indicators to diagnose patients who may have a rare disease
but also an incentive to misrepresent the health problem. The difficulties
that medical professionals face during the early stages of identifying and
preventing a novel disease such as AIDS might approximate the level of
complexity encountered by intelligence analysts daily. Nonetheless, each
difference between the professions highlights a dynamic where the analogy
still holds, and further examination may provide greater benefit for each
For example, a lesson that intelligence could learn from medicine’s
experience with preventive medicine is that, in many cases, the attempt to
assess developing health problems diverts substantial resources away from
addressing existing health problems. The medical profession has learned
that ‘‘many diagnostic tests are given routinely to apparently healthy
people in the name of prevention,’’
and that this focus on testing, even
where there may not be any health problems, leads to the collection of
excessive amounts of information. As a result, the medical profession must
divert substantial diagnostic resources to analyzing the additional
information, even though most of it will indicate that no problem exists.
The lesson for intelligence agencies is that the possibility of collecting
information does not mean that it should be, because the additional
information may have a diversionary affect on analytic expertise.
Intelligence agencies could also learn from medicine’s foundation in the
physical sciences that specific procedures may have to be implemented in
order to aggregate knowledge and establish causal relationships specific
enough to be useful for purposes of intelligence analysis. Social scientists
in academia do not have access to the kinds of specific data that
intelligence analysts do. As a result, their models are usually general and at
a high level of abstraction. Due to security and classification concerns,
however, no established process exists for creating the kinds of indicator
patterns that intelligence analysts would find useful. Where would
medicine be if it had remained empirical, and knowledge not aggregated
into theory? The establishment of an internal intelligence community unit
of social scientists devoted to the production of mid-level theory and
hypotheses useful for intelligence analysts would provide intelligence
agencies with an improved base of theory for finding meaning in the raw
intelligence. In addition, new attempts are being made to improve the way
medicine learns about disease and its impacts. In 2003, the National
Institutes of Health started a multidisciplinary collaborative effort ‘‘to
improve the diagnosis of diseases,’’ including ‘‘identify[ing] scientists
who are exceptionally creative thinkers, and award[ing] them $500,000
grants’’ as a way to foster idea generation and cross-pollination.
efforts in the intelligence community could draw together disparate experts
with idiosyncratic knowledge residing in the corners of the intelligence
community, and provide them with the opportunity to assess intractable
intelligence issues from new multi-disciplinary perspectives. In the end, not
every collaborative project has to break new ground for such an approach
to be successful; as with scientific research and development, all that is
needed is a periodic breakthrough for the approach to be worthwhile.
Discerning the Deceivers
In the area of ‘‘denial and deception,’’ the intelligence community might also
learn from medicine’s experience in identifying how physicians distinguish
malingering from legitimate patient health concerns. The incidence of
malingering may be under-diagnosed when deception goes undetected.
Conversely, the incidence of malingering may be over-diagnosed in cases
where medical knowledge has not been able to fully capture the complexity
of the human physiological system. As noted earlier, gray areas exist in
medicine at the boundary between understanding and learning. Because
physicians may not fully understand the underlying causal mechanisms,
patients with rare diseases may be diagnosed as malingerers even though the
disease itself is real, but poorly understood by medicine. The challenge for
physicians, therefore, is to remain cognizant of the potential for deceptive
behavior on the part of patients, but not to the point that legitimate signs
and symptoms are dismissed out of hand. In the intelligence world, this
observation may have immediate relevance in the assessment of the status
of Iraq’s weapons of mass destruction (WMD). In that case, intelligence
analysts apparently assumed that Saddam Hussein’s failure to document
the destruction of all of his WMD indicated that he was deceiving Western
governments and diverting the weapons elsewhere, despite his protestations
to the contrary. In the end, a warning from the medical world applies just as
well to concerns of deception in the intelligence arena: ‘‘to recognize that
[because] the detection of malingering can be very difficult’’ any diagnosis
of it ‘‘must be sustained by evidence.’’
Lessons for intelligence could also come from acknowledgement of the role
that intelligence information plays in decisionmaking, and explicit efforts
to improve the kinds of information provided to policymakers. For
example, according to a David Brown in the Washington Post, ‘‘the body
of medical research on just about any important subject is vast—too big
for the average practitioner to grasp,’’
just as it is in national security
decisionmaking. To address this problem, a government agency—the
Agency for Healthcare Research and Quality (AHRQ)—has established
‘‘evidence-based practice centers’’ at thirteen universities, and is paying
researchers there to ‘‘examine all the studies on a given question, evaluate
their validity and ultimately extract conclusions—the ‘‘best evidence’’—
from the mass of information.’’ While this medical research addresses both
diagnosis and treatment, an intelligence adaptation might be to similarly
organize and assess both raw intelligence sets and finished intelligence—to
identify the good and the bad—for the benefit of providing decisionmakers
with a better sense of the intelligence information that already exists on a
particular topic. On a broader scale, the AHRQ’s mission is to assess how
medical processes work, and how the government might help improve
those processes.
A similar unit inside the intelligence community with
free rein to assess management practices could be invaluable.
Crossing Professional Lines
Finally, the lessons that intelligence can draw from an examination of the
similarities and differences with the medical profession indicate the
importance of looking to analogous professions for ideas that can be
adapted to an intelligence context. Doing so might help improve finished
intelligence production processes and the incorporation of intelligence into
decisionmaking. Analogies serve a number of purposes, such as aiding
communication about difficult topics by finding illustrative examples in
other fields, or by more directly affecting existing ways of doing business
through the incorporation of tools that exist to achieve similar purposes in
other fields. Many of the challenges intelligence analysts face are not as
unique as its practitioners believe, but the insularity of the field prevents
them from being able to identify the lessons from other professions that
could be useful as models to follow.
As a result, the first task is to identify analogous professions, and examine
them for the lessons they might provide. Any profession that encounters
similar problems—such as medicine, journalism, law, or law enforcement—
may provide fertile ground for deriving ideas to improve existing practices.
Perhaps if intelligence analysts adopted methods from analogous
professions—or adapted them to the unique requirements of intelligence
analysis—some of the obstacles they currently face in accurately portraying
their understandings of the international environment could be overcome.
Walter Laqueur, ‘‘The Question of Judgment: Intelligence and Medicine,’’
The Journal of Contemporary History, Vol. 18. 1983, pp. 533–548. See also:
Walter Laqueur, A World of Secrets: The Uses and Limits of Intelligence
(New York: Basic Books, 1985), pp. 302–305.
According to Dorland’s Medical Dictionary, a ‘‘sign’’ is ‘‘any objective evidence of
disease’’ that can be independently observed by the physician, whereas, a
‘‘symptom’’ is ‘‘any subjective evidence of disease’’ reported by the patient.
Dorland’s Pocket Medical Dictionary, 26th ed. (Philadelphia: W.B. Saunders,
Walter Laqueur, ‘‘The Question of Judgment,’’ p. 535.
Ibid., pp. 534–535.
According to Dorland’s Medical Dictionary, ‘‘diagnosis’’ is the determination of a
cause of disease, and ‘‘prognosis’’ is ‘‘a forecast of the probable course and
outcome of a disorder.’’
See United States Department of Defense, Joint Publication 1-02, Department of
Defense Dictionary of Military and Associated Terms (Washington, DC: Joint
Chiefs of Staff, 2003), p. 55.
Janice Williams, Henry Schneiderman, and Paula Algranati, Physical Diagnosis:
Bedside Evaluation of Diagnosis and Function (Baltimore: Williams and Wilkins,
1994), pp. 1–5.
For parallels in the technologies used in medicine and intelligence, see: Sam Grant
and Peter C. Oleson, ‘‘Dual Use of Intelligence Technologies: Breast Cancer
Detection Research,’’ Studies in Intelligence, Vol. 1, No. 1, 1997, at
Richards J., Heuer, J. Psychology of Intelligence Analysis (Washington, DC: CIA
Center for the Study of Intelligence, 1999), pp. 61–62.
Stephen Marrin, ‘‘Improving CIA Analysis by Overcoming Institutional
Obstacles,’’ in Russell G. Swenson, ed., Bringing Intelligence About:
Practitioners Reflect on Best Practices (Washington, DC: Joint Military
Intelligence College, 2003), pp. 40–59.
Mark V. Kauppi, ‘‘Counterterrorism Analysis 101,’’ Defense Intelligence Journal,
Vol. 11, No. 1, Winter 2002, p. 47.
Richards Heuer, Psychology of Intelligence Analysis, p. 62.
Ibid. While Heuer’s observations may be true in theory, the medical profession is
currently experiencing a debate over the possible over-collection of data that does
not conform to medical diagnostic theory. This problem with over-collection has
its parallels in the intelligence world as well. As a result, both fields struggle with
allocation and utilization of scarce resources.
Ibid., pp. 45, 101–102.
Ibid., p. 26. In this section Heuer cites Arthur S. Elstein, Lee S. Shulman, and
Sarah A. Sprafka, Medical Problem Solving: An Analysis of Clinical Reasoning
(Cambridge, MA: Harvard University Press, 1978), p. 276.
For a list of analytic errors that apply to both intelligence analysis and medicine,
see: Walter Laqueur, ‘‘The Question of Judgment,’’ p. 541.
Stephen Marrin, ‘‘Improving CIA Analysis by Overcoming Institutional
Obstacles,’’ pp. 40–59.
Ronald D. Garst and Max L. Gross, ‘‘On Becoming an Intelligence Analyst,’’
Defense Intelligence Journal, Vol. 6, No. 2, 1997, p. 48.
For more on the causes of analytic failure, see Richards Heuer, ‘‘Improving
Intelligence Analysis: Some Insights on Data, Concepts, and Management in
the Intelligence Community,’’ The Bureaucrat,Vol.8,No.1,Winter1979=80,
pp. 2–11. See also, Richard Betts, ‘‘Analysis, War and Decision: Why
Intelligence Failures Are Inevitable,’’ World Politics,Vol.31,No.1,October
Walter Laqueur, ‘‘The Question of Judgment,’’ p. 544.
For more on this dynamic, see: Center for Disease Control (CDC) Website; ‘‘Fact
Sheet: Helicobacter pylori and Peptic Ulcer Disease.’’ http:==www.cdc.gov=ulcer=
David Brown, ‘‘Medical Care Often Not Optimal, Study Finds,’’ The Washington
Post, 26 June 2003, p. A02.
For more on intelligence epistemology, see: Mark M. Lowenthal, ‘‘Intelligence
Epistemology: Dealing with the Unbelievable,’’ International Journal of
Intelligence and CounterIntelligence, Vol. 6, No. 3, Fall 1993, pp. 319–325.
John Lewis Gaddis, The Landscape of History: How Historians Map the Past
(New York: Oxford University Press, 2002), p. 57.
Ibid., p. 60.
An exception might be models developed internal to the intelligence community
that enable them to assess events of interest such as political stability. For
more, see: Stanley A. Feder, ‘‘FACTIONS and Policon: New Ways to Analyze
Politics,’’ in Inside CIA’s Private World: Declassified Articles from the Agency’s
Internal Journal, 1955–1992, H. Bradford Westerfield, ed. (New Haven: Yale
University Press, 1995), pp. 274–292. Also see: Stanley A. Feder, ‘‘Forecasting
for Policy Making in the Post Cold-War Period,’’ Annual Review of Political
Science, Vol. 5, June 2002, pp. 111–125.
Walter Laqueur, ‘‘The Question of Judgment,’’ p. 533.
Ramzi S. Cotran, Vinay Kumar, Stanley L. Robbins, Robbins Pathologic Basis of
Disease, 4th ed. (Philadelphia: W.B. Saunders Company, 1989).
Walter Laqueur, ‘‘The Question of Judgment,’’ p. 535.
See Dorland’s Pocket Medical Dictionary, 26th ed.
Walter Laqueur, ‘‘The Question of Judgment,’’ p. 545.
Ernest R. May, Strange Victory: Hitler’s Conquest of France (New York: Hill and
Wang, 2000), pp. 458–459.
Geoffrey Vickers, The Art of Judgment: A Study of Policy Making (Thousand
Oaks, CA: Sage Publications, 1995), pp. 225–226.
Alexander L. George, Bridging the Gap: Theory and Practice in Foreign Policy
(Washington, DC: United States Institute of Peace Press, 1993), p. xx.
Michael Moss, ‘‘Mammogram Team Learns from Its Errors,’’ The New York
Times, 28 June 2002, p. A1. Also cited in Steven Rieber, ‘‘Intelligence Analysis
and Judgmental Calibration,’’ International Journal of Intelligence and
CounterIntelligence, Vol. 17, No. 1, Spring 2004, pp. 97–112.
Shannon Brownlee, ‘‘The Perils of Prevention,’’ The New York Times, 16 March
2003, p. 52. For more on the diversion of resources to address aspects of
prevention, see Gina Kolata, ‘‘Annual Physical Checkup May Be an Empty
Ritual,’’ The New York Times, 12 August 2003, p. 71.
Rick Weiss, ‘‘Cross-Pollination in Pursuit of Cures: NIH Launches Drive to
Increase Collaboration Among Scientific Disciplines,’’ The Washington Post,
1 October 2003, p. A2.
‘‘Malingering: Can It Be Detected?,’’ Med League Support Service Inc.
http:== www.medleague.com=Articles=Medical%20Topics=Detecting Malingering.
David Brown, ‘‘Director Seeks ‘Just the Facts’ to Improve Medical Care,’’
The Washington Post, 5 February 2003, p. A2.
Agency for Healthcare Research and Quality Website: http:==www.ahrq.gov=