ChaosWatch
New here? Start here →
Glossary

What the words mean

This site measures pressure in the world's news, then tests whether language models can forecast where it goes next. Both halves come with vocabulary. Every term below is linked from the page that uses it, so you can read on and click back when a word stops you.

The short version

A free public archive called GDELT machine-reads the world's news. From it this project computes five daily pressure readings per region — geopolitical, energy, climate, economic, digital. That half is plain arithmetic with no AI in it, which is why it can serve as the answer key.

Those five readings are then put to two tests. First: does some combination of them reliably come before things getting worse? Second: can frontier language models forecast what happens next when handed the identical evidence?

As of this data, both answers are no — and that is the result the site reports. No configuration of the five pressures reaches the consistency bar, and no model forecasts escalation better than chance. Nothing here is tuned to look more conclusive than it is; the numbers are shown in full so you can see how near or far each one fell.

Where the numbers come from

The raw material: public news data, and nothing self-reported.

GDELT#

A free, public archive that machine-reads the world's news and publishes a row for every event it detects.

GDELT (the Global Database of Events, Language and Tone) scans news media worldwide and publishes a file every 15 minutes listing the events it found: who did what to whom, where, and how the coverage was worded.

This project reads the daily bulk files. Nothing here is a survey, a poll, or an expert judgement — it is a count of what the world's press reported, with all the biases that implies. Places and days with more journalists produce more rows, and that is a property of the media, not of the world.

The spine#

The AI-free half of this project: the same input always produces the same numbers.

Everything on the Region Overview page is computed by arithmetic from GDELT files. No language model is involved, so re-running it on the same dates reproduces the same values exactly.

That is the point. The spine also computes what actually happened next, which becomes the answer key the language models are graded against on the Leaderboard. A grader that was itself a language model could not be trusted to mark its own homework.

Threat vector#

One of the five kinds of pressure measured each day: geopolitical, energy, climate, economic, digital.

Each vector is a separate daily measurement for each region. They are never added together into a single 'danger score', because they are not measured in the same unit.

Geopolitical is a severity sum built from conflict events. The other four are rates — how many articles per million mentioned that theme. Averaging a sum with four rates produces a number in no unit at all, so where this app needs one number per day it averages the calibrated memberships instead.

Raw value#

The measurement before calibration, in whatever unit that vector happens to use.

Raw values are honest but incomparable: a geopolitical raw of 4,000 and a digital raw of 4,000 have nothing to do with each other. Calibration exists to make them comparable.

QuadClass / Goldstein scale#

GDELT's own labels for how cooperative or hostile an event was.

The Goldstein scale scores each event type from about −10 (most hostile) to +10 (most cooperative). QuadClass sorts events into four buckets: verbal cooperation, material cooperation, verbal conflict, material conflict.

The geopolitical vector is built from the hostile end, weighted by how widely each event was reported.

Turning measurements into memberships

How five incomparable measurements become five comparable 0-to-1 scores.

Fuzzy membership#

A 0-to-1 score for how well a day fits the idea of 'a high-pressure day' for that vector.

0 means the day is clearly not a high-pressure day, 1 means it clearly is, and values in between mean partly. It is not a probability and not a percentage — it is a degree of fitting a description.

This is the idea Charles Ragin's fuzzy-set method is built on: real categories have edges, and forcing every day into 'yes' or 'no' throws away the information that some days sit near the boundary.

Crucially, membership is relative to THIS 365-day window. A membership of 0.51 means 'about like the median day of this window', not 'moderately dangerous'.

Anchor#

The three cut-points that convert a raw value into a fuzzy membership.

Calibration needs three reference points per vector: the value below which a day is definitely not high-pressure (full_out), the value that sits exactly on the fence (crossover), and the value above which it definitely is (full_in). Here they are the 10th, 50th and 90th percentiles of the calibration window.

Anchors are versioned and frozen. Once a version label such as v3 has been used, re-deriving different numbers under that same label is rejected outright, so a label can never come to mean two different calibrations. That is what makes an old chart still readable months later.

Choosing anchors AFTER seeing which answer you would like is the classic way to make a null result look like a finding. Freezing them is the defence.

full_out#

The low cut-point: at or below this raw value, the day is fully outside the category (membership floored at 0.001).

Here it is the 10th percentile of the calibration window — the quietest tenth of days.

Crossover#

The middle cut-point, where membership is 0.50 — the point of maximum ambiguity.

Here it is the median (50th percentile) of the calibration window. A day exactly on the crossover is as much in the category as out of it.

full_in#

The high cut-point: at or above this raw value, the day is fully inside the category (membership capped at 0.999).

Here it is the 90th percentile of the calibration window — the busiest tenth of days.

Membership is capped at 0.999 rather than 1.000 (and floored at 0.001 rather than 0) by Ragin's convention, so that no case is ever treated as a perfect, unarguable member.

Crisping / present / absent#

Rounding a fuzzy membership to a plain yes/no at the 0.50 line, only where a yes/no is unavoidable.

'Present' means membership above 0.50; 'absent' means 0.50 or below. It says which side of the crossover the day fell on and nothing more — a 'present' reading is not an alarm, and 0.51 and 0.99 both read as present.

The fuzzy value is always shown next to it, because that is the number that carries the information.

Looking for recipes (fsQCA)

Which combinations of pressure precede escalation — and here, the answer is none of them.

fsQCA#

Fuzzy-Set Qualitative Comparative Analysis: a method that asks which COMBINATIONS of conditions lead to an outcome, rather than which single factor matters most.

Developed by the sociologist Charles Ragin. A regression asks 'how much does energy pressure raise escalation, holding the rest constant?' fsQCA asks a different question: 'is there some recipe — say high energy AND high geopolitical AND low economic — that is reliably followed by escalation?'

That suits this problem, where causes plausibly work in combination and more than one distinct recipe could lead to the same outcome.

It is a method for finding patterns, not causes. A recipe that consistently precedes escalation still does not establish that it produced it.

Configuration#

One specific recipe: a yes/no setting for each of the five vectors.

With five vectors there are 2⁵ = 32 possible recipes, from all-absent to all-present. This app enumerates every one of the 32, including recipes no real day ever matched, so the table is a complete map rather than a list of whatever happened to occur.

Truth table#

The table of all 32 configurations and how each one scored.

Every case (region-day) contributes to every row, weighted by how well it fits that row's recipe. Cases are not sorted into the single row they most resemble — that would discard the partial fits fuzzy sets exist to preserve.

Consistency#

Of the days matching this recipe, how reliably did escalation follow? Runs 0 to 1; 0.80 is the conventional bar for calling a recipe sufficient.

High consistency means the recipe rarely fails: when you saw this pattern, escalation followed. This is the number that decides whether a configuration counts as a finding.

In this project's data the best consistency across all 32 corners is 0.527, well short of 0.80. So the honest reading is that no pattern of these five pressures reliably precedes escalation here.

Coverage#

Of all the escalation that happened, how much did this recipe account for? Runs 0 to 1.

Consistency and coverage answer different questions and trade off. A recipe can be perfectly consistent yet explain almost nothing (it fires once, and it is right), or cover a great deal while being wrong half the time.

Coverage is only worth reading once consistency clears its bar. A high-coverage, low-consistency recipe is a common pattern, not an explanation.

Unobserved (logical remainder)#

A configuration that no day in the data ever matched, so it has no score.

Such a corner is shown as unobserved rather than as scoring zero. Zero would mean 'we looked and escalation never followed'; unobserved means 'this combination never arose, so we have no evidence either way'. Collapsing the two is one of the easier ways to invent a finding.

Testing the language models

How the forecasts are collected and graded, and what 'better than chance' requires.

Model Arena#

The test: several language models get the identical evidence bundle and each forecasts what happens next.

Every model in a run sees exactly the same headlines and indicator values — same input, same question, same moment. Differences in their answers are therefore differences between the models, not differences in what they were shown.

Their forecasts are stored and later graded against what the spine says actually happened.

Evidence hash#

A fingerprint of the exact evidence bundle a run was given, so the comparison is provably fair.

All models in a run share one hash. If two models had somehow been shown different material, the hashes would differ and the comparison would be void. Displaying it is how you can check that rather than take it on trust.

Depth profile#

How much evidence the run gathered — shallow, standard or deep.

Depth is a deliberate experimental factor: does giving a model more material make it forecast better? Results at different depths are never pooled, because a model's score at deep depth is not a measurement of the same thing as its score at shallow depth.

Horizon (k)#

How many days ahead the forecast reaches. k = 7 means 'what will things look like a week from now'.

Like depth, horizon is a separate axis and rows at different k are never merged. Forecasting 3 days out and 14 days out are different tasks with different difficulty.

Escalation#

The thing being predicted: whether measured pressure rises over the next k days.

Computed by the spine from GDELT data that arrived after the forecast date, so it is an answer key no model could have seen. Calendar gaps are refused rather than interpolated — a missing day makes the outcome undefined instead of quietly wrong.

MAE (mean absolute error)#

On average, how far off a model's forecast was. Lower is better.

It is measured in the same units as the thing being forecast, so it reads directly: an MAE of 1.1 means the typical forecast missed by about 1.1 points.

No-skill baseline#

The error you would get from a trivial guess. Beating it is the minimum bar for a model being useful at all.

Here the baseline is 1.128. A model whose MAE is near or above that has told you nothing you could not have got for free, however fluent its reasoning sounded.

Spearman correlation (ρ)#

Whether a model ranked the tense days above the calm ones. Runs from −1 to +1; 0 is no relationship.

It compares orderings rather than exact values, so a model that is consistently too high but ranks days correctly still scores well. That is a generous test — it asks only 'did you know which days were worse?'

+1 is a perfect ordering, 0 is chance, and a negative value means the ordering ran backwards.

Critical value / significance#

The correlation a model would need before the result is distinguishable from luck, given how many days it was scored on.

With few data points, sizeable correlations happen by chance alone. The threshold depends on the sample: it is 0.207 at n = 90, but 0.404 at n = 24. A correlation below its own row's threshold is reported as indistinguishable from chance — it is not a small effect, it is not evidence of an effect.

In this project no model clears its threshold at n = 90. One model looked impressive at n = 24 (ρ = 0.499) and fell to 0.079 once the sample grew — a textbook illustration of why the small-sample number was never a finding.

What this site cannot tell you

Whether anything is about to happen. Nothing here is a forecast you should act on. The measurements describe what has been reported; the forecasting test came back negative.

How dangerous a region is. The scales are percentiles of this one 365-day window, so a high reading means unusual for this window, not severe. Two regions' numbers are comparable in that sense and in no other.

What caused anything. Every method here finds patterns. A pattern that precedes escalation has not been shown to produce it.

What the world did — only what was reported. GDELT measures news coverage. Places and periods with more journalists generate more rows, and an event nobody covered is, to this data, an event that did not occur.