Hubbard's Approach to Cyber Risk Quantification: Calibration and Monte Carlo

If FAIR provides the structure for quantitative cyber risk analysis, Doug Hubbard provides the method. Hubbard's body of work — particularly How to Measure Anything in Cybersecurity Risk, co-authored with Richard Seiersen, now in its second edition — is the practitioner's foundation for actually doing the analysis: how to elicit numbers from people, how to combine them under uncertainty, and how to produce outputs that support real decisions.

This article is the working guide to Hubbard's methodology — calibrated estimation, Monte Carlo simulation, and the decision-theoretic frame that makes both useful. For the structural ontology Hubbard's methods populate, see FAIR Methodology Explained. For the broader case for quantification, see the risk quantification pillar.

This is not legal or actuarial advice.

The thesis in one sentence

Hubbard's central argument can be stated compactly: the methods used to make decisions under uncertainty in actuarial science, decision analysis, and applied statistics have been mature for decades, are not specific to cyber, and produce reliably better estimates than the qualitative methods cyber security has traditionally used — even when data is sparse.

The implications follow. If the methods are mature and accessible, the prevailing use of qualitative risk scoring in cyber is a choice — not a necessity imposed by the field's data scarcity. The choice is wrong, and Hubbard's work is partly an extended demonstration of why.

The four objections Hubbard demolishes

A working knowledge of Hubbard's approach starts with understanding what it argues against. Four objections to cyber risk quantification recur in practitioner conversation, and Hubbard addresses each at length.

Objection one: "There isn't enough data"

The most common objection, and the one Hubbard rebuts most thoroughly. The argument has several parts.

First, "not enough data" is itself a measurable statement. How much data is "enough"? Hubbard shows that the marginal information value of additional data points falls quickly — for most decisions, fewer data points than analysts assume are needed for the analysis to be decision-relevant. The argument is rooted in the value-of-information mathematics that decision analysis has used since the 1960s.

Second, where empirical data is sparse, calibrated expert estimation fills the gap with measurable reliability. Decades of research in decision sciences demonstrate that trained experts produce reliable interval estimates of unknown quantities. The training is short, the improvement is measurable, and the resulting estimates are sufficient input for analysis. Sparse data is a reason to use careful methods, not to abandon analysis.

Third, the alternative — qualitative scoring — does not solve the data problem; it merely hides it. Saying a risk is "high" rather than "between 5% and 40% likely with €200,000 to €5M impact" does not add information. It conceals the absence of information behind a label.

Objection two: "Cyber risk is too uncertain to quantify"

Variations: "We can't predict cyber events." "Cyber is non-stationary." "Past data doesn't predict future incidents."

The actuarial sciences have handled uncertainty about catastrophic, low-frequency, high-impact events — earthquakes, hurricanes, terrorism — for over a century. They do this not by claiming to predict individual events but by characterising the distribution of events. Quantitative cyber risk analysis does the same. It does not claim to predict whether the firm will be breached in 2027; it produces a distribution over plausible loss outcomes, and that distribution is decision-relevant even if no single incident can be predicted.

Hubbard makes the further methodological point that uncertainty is a feature, not a bug, of probabilistic analysis. The output of a Monte Carlo simulation is a distribution precisely because uncertainty is being represented honestly. Wide intervals, faithfully reported, are more useful than narrow intervals fabricated to look confident.

Objection three: "Numbers create false precision"

This is the objection of careful analysts, and it has merit when quantification is done badly. Reporting "annual loss is €4,237,892" gives an appearance of precision that the underlying analysis cannot support.

But this is an objection to bad reporting, not to quantification. The Hubbard tradition is explicit: report distributions and intervals, not point estimates. "Mean annual loss €4M with 90% interval €1M to €15M, and a 5% chance of exceeding €25M" is not false precision — it is honest about the underlying uncertainty. The qualitative alternative ("high impact, medium likelihood") hides more, not less.

The defence against false precision is good methodology, not avoiding the numbers.

Objection four: "It's too complicated"

The final objection is practical — the methods sound mathematically heavy. Hubbard's response is that the minimum useful version is accessible: calibration training is one day, Monte Carlo in a spreadsheet runs in seconds, and an organisation can produce a meaningful first analysis in a few weeks.

The methodologically advanced versions (Bayesian networks, copula dependence structures, non-stationary models) exist for organisations that need them — but the entry-level version is small enough that any organisation already doing qualitative risk assessment can adopt it.

The methodological core: calibration

The single most important contribution of the Hubbard tradition is the practice of calibrated estimation.

The problem calibrated estimation solves

When uncalibrated analysts are asked to produce 90% confidence intervals — ranges they are 90% certain contain the true value — the intervals are systematically too narrow. Decades of decision-science experiments document this. Untrained estimators' supposed 90% intervals contain the true value about 50-60% of the time. They are overconfident by margins large enough to make their estimates unreliable inputs for analysis.

The phenomenon is robust. It survives education, experience, expertise, and self-assurance. A senior analyst with deep domain knowledge is not, by virtue of seniority, calibrated. Calibration is a separate skill that must be trained.

How calibration training works

Calibration training, as Hubbard implements it, takes a single day. The structure:

1. Establish baseline. Trainees are given a quiz of general-knowledge questions — historical dates, geographic distances, scientific quantities — and asked to give 90% confidence intervals for each. The baseline shows how few of their "90%" intervals actually contain the truth.

2. Practice techniques. Several specific techniques widen intervals appropriately: the equivalent bet test (would you take a bet that the truth is inside your interval at 9:1 odds? if not, widen the interval), explicit consideration of how often you have been wrong before, deliberately searching for reasons the answer could be outside the initial interval.

3. Iterate with feedback. Trainees take repeated quizzes, see their hit rates, adjust their intervals, and converge — typically within a few hours — on intervals that contain the truth roughly 90% of the time.

The training is deeply unglamorous and works reliably. A calibrated analyst's 90% interval contains the truth about 90% of the time, robustly across new domains. The analyst is not pretending to know more than they know; they are honest about their uncertainty in a way that makes their estimates usable inputs.

Why calibration matters more than any individual estimate

The deep point: if analysts' intervals are calibrated, the combined analysis is reliable even when individual inputs have wide intervals. A Monte Carlo simulation built on calibrated 90% intervals produces output distributions whose tails reflect the actual uncertainty. The analysis is no longer brittle to the analysts' overconfidence.

A team of calibrated analysts producing wide-but-honest intervals produces better analysis than a team of uncalibrated analysts producing narrow-but-confident point estimates. The difference is mathematical, not stylistic.

We treat calibration in more depth in Calibrated Estimation: Training Analysts to Quantify Under Uncertainty when published.

The methodological core: Monte Carlo

The second pillar of Hubbard's method is Monte Carlo simulation — the technique that combines input distributions into output distributions.

What Monte Carlo does

The mathematical problem: given a model where several input quantities are uncertain — each represented as a probability distribution — what is the probability distribution of the output?

For complex models, this cannot be solved analytically. Monte Carlo solves it numerically: sample from each input distribution, compute the model's output for that sample, repeat tens of thousands of times, accumulate the outputs into an empirical distribution. The empirical distribution converges, with enough samples, to the true output distribution.

For a FAIR-style analysis, the inputs are the leaf nodes of the FAIR taxonomy (threat event frequency, vulnerability, primary loss forms, secondary loss). Each is represented as a distribution — lognormal is common for magnitudes, Poisson for frequencies, beta for probabilities. The Monte Carlo runs 10,000 to 100,000 "years" of the model and produces the annual loss distribution.

Running it in practice

A practical Monte Carlo for cyber risk does not require statistical software. A competent spreadsheet handles it natively: define each input as a distribution (Excel has built-in functions for normal, lognormal, beta, gamma, etc.), create rows for each simulated year, compute the output, summarise. Tens of thousands of rows run in seconds.

For larger analyses, dedicated tools help — @RISK, Crystal Ball, R, Python with NumPy. The mathematics is identical regardless of platform.

The methodological care comes in choosing the input distributions correctly. Threat event frequencies are typically Poisson (count of events per year); magnitudes are typically lognormal (skewed, positive, with long tails); probabilities are typically beta. Misspecifying the distribution shape produces misleading tails. We cover the practical mechanics in Monte Carlo Simulation for Cyber Risk when published.

Interpreting the output

The output of a Monte Carlo is a distribution of annual loss. Key summaries:

• Mean — the expected annual loss; the long-run average
• Median — the middle outcome; less affected by tail outliers
• Confidence interval — typically the 5th to 95th percentile range
• Tail values — the 95th, 99th percentile losses; what to plan for in adverse scenarios
• Loss Exceedance Curve — for each loss level, the probability of exceeding it in a year

Reporting the mean alone is malpractice. Cyber loss distributions are typically heavily right-skewed — the mean is pulled up by rare large events — and the median often differs materially from the mean. Both should be reported, along with the tail. See Loss Exceedance Curves and Annual Loss Expectancy when published.

The decision-theoretic frame

Beyond calibration and Monte Carlo, Hubbard's approach is grounded in decision analysis — the broader field concerned with making rational choices under uncertainty.

The decision-analytic frame asks: what decision is this analysis supporting? The methodology is tuned to that decision. A control investment decision needs a distribution of loss reduction. A capital allocation decision needs a tail value. An insurance purchase decision needs a probability of exceeding a coverage threshold. The analysis is built to answer the actual question.

This framing has a useful side effect: it forces analysts to ask whether the analysis is worth doing before they do it. Hubbard's Applied Information Economics methodology includes explicit value-of-information calculations — what is the most we should spend on reducing uncertainty here? If the uncertainty does not change the decision, the analysis is unnecessary. If it does, the analysis is justified up to the value it adds.

This discipline alone — what decision are we informing? — prevents many of the failures that produce risk analyses that are produced, filed, and never used.

How Hubbard relates to FAIR

The relationship is complementary, and most mature practice uses both.

FAIR provides structure. The taxonomy of risk into frequency factors and magnitude factors, with explicit decomposition, gives analyses a consistent shape. Without FAIR (or an equivalent ontology), Hubbard's methods would still work — but each analysis would invent its own decomposition.

Hubbard provides method. Calibration, Monte Carlo, and the decision-analytic frame populate the FAIR taxonomy with reliable inputs and combine them into reliable outputs. Without Hubbard (or equivalent methodology), FAIR is a structure with no rigorous way to fill in the numbers.

Together, the combination — FAIR ontology, calibrated estimates at the leaves, Monte Carlo through the tree — is the working practitioner's standard for quantitative cyber risk analysis in 2026.

Common pitfalls

Six errors that show up repeatedly in early-stage practice.

1. Skipping calibration training. The single highest-leverage discipline, and the most often skipped. Without calibration, the analysis is built on overconfident inputs and inherits the overconfidence in its outputs.
2. Reporting point estimates only. A single number from a Monte Carlo output is decision-misleading. Always report a distribution.
3. Misspecifying input distributions. Lognormal magnitudes, Poisson frequencies, beta probabilities are the defaults. Using normal distributions for magnitudes (symmetric, unbounded below) is a common error that distorts tails.
4. Treating analyses as one-off projects. A quantitative analysis becomes valuable through iteration — refresh as evidence accumulates, update inputs as controls change, revisit when major incidents trigger learning. A one-off analysis is filed and forgotten.
5. Avoiding the difficult inputs. Reputation loss, business interruption, the secondary loss forms — these are hard to estimate and analysts often skip them or assume zero. Calibrated estimation specifically addresses the case of difficult-to-measure quantities; use it.
6. Confusing the model with reality. The model is a representation, not the territory. Its outputs are informative given the model's structure. Sensitivity analysis — varying the model's assumptions and observing the output — is part of the discipline.

Frequently asked questions

Is calibration training really effective? Yes, and reproducibly so. The effect is documented in decades of decision-science research and replicates across populations. A calibrated analyst is not slightly better; they are reliably accurate at the calibration level.

Do I need statistical software for Monte Carlo? Not for the entry-level version. A spreadsheet runs Monte Carlo natively. For larger analyses, dedicated tools help.

How long does it take to learn the methodology? Calibration training: one day. Running a first Monte Carlo: a few days of work with a spreadsheet. Producing a defensible first FAIR-style analysis with Hubbard methodology: two to four weeks for a small team.

Is Hubbard's methodology specific to cyber? No. Hubbard's earlier work (How to Measure Anything) applies to general business decisions. How to Measure Anything in Cybersecurity Risk applies the same methods to cyber specifically. The methods are general.

Does this work for organisations with few past incidents? Yes. The methodology is built precisely for the case of sparse data. Calibrated expert estimation fills the gap, and the resulting analyses are honest about the wider uncertainty that sparse data produces.

What's the relationship between Hubbard's approach and Bayesian methods? The mathematical foundation is Bayesian — calibrated estimates are effectively prior distributions, and updating with evidence is Bayesian inference. The advanced practice uses explicit Bayesian networks; the entry-level practice does the Bayesian operations implicitly.

The bottom line

Three takeaways.

1. Calibrated estimation is the methodological foundation. A day of training produces estimators whose 90% intervals actually contain the truth 90% of the time. This is the difference between unreliable and reliable quantitative analysis.
2. Monte Carlo is accessible. A spreadsheet runs it. The mathematics is mature and unmysterious. The barrier is conceptual, not technical.
3. The decision-analytic frame disciplines the analysis. Ask what decision the analysis is supporting; build the analysis to answer that question; do not produce analyses that no one will use.

For the structural ontology this methodology populates, see FAIR Methodology Explained. For the simulation mechanics in detail, see Monte Carlo Simulation for Cyber Risk when published.

---

Sources & further reading

• Doug Hubbard and Richard Seiersen — How to Measure Anything in Cybersecurity Risk (2nd edition, 2023)
• Doug Hubbard — How to Measure Anything: Finding the Value of "Intangibles" in Business (3rd edition)
• Doug Hubbard — The Failure of Risk Management: Why It's Broken and How to Fix It (2nd edition)
• Hubbard Decision Research — Applied Information Economics methodology
• Decision-science calibration literature (Lichtenstein, Fischhoff, Tetlock)