Stable until the fifth slice

One of my CVEs is published twice, with two different severity scores.

CVE-2026-77826 is an unauthenticated authentication bypass in a registration plugin. The advisory page at the database that assigned the identifier shows 8.8. The Wordfence database keeps its own record and shows 9.8. Those sit either side of the boundary between high and critical.

Compare the full vectors and they agree on every metric but one: whether exploitation requires the victim to do something. One says it does and reaches 8.8. The other says it does not and reaches 9.8. That single judgement is the whole gap.

The attribution is messier than that makes it sound, and the mess is the point. Open the canonical CVE record and the assigning body published no metrics at all. The 8.8 vector lives in an enrichment container added by a third party, which the national database then carries onward as a secondary score. So one of those numbers is a published assessment. The other is an annotation. You can't tell which from the page.

I wanted to know whether that was a curiosity or a pattern. I measured it, got a clean answer, committed the answer, and then took one more sample.

The first number looked like a finding

I pulled comparable WordPress plugin CVEs, meaning records where two independent parties had each published a full CVSS vector for the same defect. Then I counted how often the two scores differed at all. Four disjoint samples, every pairwise overlap zero.

One structural limit, because it changes what the rest of this means. Wordfence is one side of every pair. This measures that database against whoever else scored the same record, not an all-pairs comparison between arbitrary bodies. I'd have preferred the matrix. It turns out not to be constructible at all. Across roughly 833 records, checked two ways, not one canonical CVE record carried two scoring containers from different parties.

The problem was the spread, about fifteen points on a metric I'd expected to hold steady. So I tried to explain it. Three times.

Divergence rises with CVE age. The slice reaching furthest back by CVE id came in lowest. Refuted, with a caveat I only measured later: CVE id order turns out not to be publication order. Across 702 id-sorted pairs, 173 adjacent ones are out of date sequence, about a quarter. So every slice here is id-stratified and none of them properly tests an age claim at all.

It's which scoring bodies happen to be in each sample. Samples 3 and 4 have much the same mix. Their rates are fifteen points apart. Refuted.

It's a stable property of each pair of bodies. Hold one pair fixed across four consecutive bands and you get 82.2 %, 80.9 %, 82.9 %, then 67.1 %. Refuted.

Then the class breakdown forced a better question, almost by accident. Cross-site scripting records differ in the score about 99 % of the time, and substantively only about a quarter of it. You see 7.2 against 7.1, or 6.4 against 6.5. A tenth of a point, as scoring convention rather than opinion, and my metric was counting every one as a disagreement.

So I raised the threshold to half a point and recounted. The spread collapsed from fifteen points to under six. The four samples landed at 54.4 %, 51.5 %, 57.1 % and 52.3 %.

That felt like the answer. The variation I'd spent three rounds explaining was an artefact of my own measurement. I committed it under a heading with two green ticks on it.

Then I took a fifth slice

Fourteen minutes later. It came in at 42.9 %.

487 + 534 + 599 + 598 + 599 = 2,817 pairs.

Not a special sample. The next link in the same contiguous chain, same size, same procedure, and I drew it to test something else entirely. The range is 42.9 to 57.1 percent, a 14.2 point spread. "Fixing the metric dissolved the mystery" does not survive it.

⚠ Here is the part I'd rather not write. That slice sat on disk for three days and changed nothing. I had the sample and I didn't notice. What overturned the headline was a tool I built afterwards, for an unrelated reason. The stable table couldn't be audited by reading it, so I wrote something to re-derive every figure from its named artefact. It re-derived the fifth slice on the way past.

The extra sample made the claim falsifiable. A mechanical re-derivation falsified it. Those are not the same event and I had been telling myself they were.

The replacement finding is better than the one it killed

The aggregate rate has no stable value, because it isn't one quantity. It's the mix of vulnerability classes in whatever sample you drew.

The nine largest classes, 2,400 of the 2,817 pooled pairs:

The five omitted classes sit between 21 % and 66 %, so the endpoints above are the real extremes. Ninety-nine percent at one end and four percent at the other. The fifth slice sits low because 135 of its 599 pairs are file inclusion, a four-per-cent class. There was never an aggregate to explain.

⚠ One disclosure on that table, because it's the part a reader should be able to check and mostly can't. The class labels come from a join done offline against a local cache. They were written back into only two of the six samples. So 2,218 of the 2,817 pooled pairs carry no stored label, and most of the table reproduces from the cache rather than from the sample files. Re-running that join resolves a class on 2,817 of 2,817, none unresolved, which is why I'm comfortable publishing it. But the audit trail runs through the cache. Know that before leaning on it.

The part I'm most confident about, because I called it first

Fitting per-class rates on four of the five slices and predicting the fifth, five times over, the worst miss is 6.3 points against a 14.2 point range. Better than nothing, and not an explanation.

So I pre-registered the prediction and drew a fresh sample, disjoint from all five priors. Six hundred records, 565 comparable, 444 of them against the single counterpart the model was trained on.

Both figures are over those 444 pairs. The prediction is priced on the 432 of them in classes the model had enough training data for. On the wider all-counterpart population the same sample reads 50.3 %.

On priority: the registration file was written about fifteen minutes before the sample artefact existed, and both timestamps sit on the files. The commit carrying it landed an hour later, after the data. If you were verifying the claim the normal way, by commit order, you would find it the wrong way round.

The precision is partly luck, and that matters more than the hit. The per-class rates themselves moved a long way between training and test. Missing authorization dropped ten points, cross-site scripting gained eleven, and deserialization gained twenty-one. The aggregate landed at two-tenths of a point because those cancelled against their weights.

The honest claim is class mix carries the aggregate. The claim it would be nicer to make is per-class rates are stable, and that one is measurably false. A result this clean is exactly when you should go looking for the reason it's cleaner than it should be.

What they actually argue about

The most checkable thing here is not a rate. It's which metric moves. These are the two bodies in the largest counterpart pair, which is not the pair in the opening example.

On substantive SQL-injection disagreements they differ on Scope in every one of 257 cases, and on Availability in all but one. A single metric differing on essentially every record in a class is a rule applied differently, not case-by-case judgement. On deserialization they differ on attack complexity in 99 percent of cases. On information disclosure, confidentiality, in 85.

The direction is consistent too. On SQL injection one body scores a median 1.8 points higher, and higher in all but one of the 257 non-tied pairs. That replicated on the pre-registered sample at +2.0 and 33 of 33. I'm not going to tell you why the gap is that size. I had an account of it and the arithmetic behind the account was broken, so it's withdrawn.

And one prediction I got wrong, which is the most interesting result in the set. I'd characterised the cross-site-scripting noise as directionless, from 587 training pairs, and pre-registered that it would stay between 35 and 65 percent. On the new sample it came back at 72 % of 116 non-tied pairs. The magnitude is still tiny, a median moving from −0.1 to +0.1. ⚠ I'm deliberately not calling that a trend. Broken out by publication month, the same figure runs high in the older months and low in the recent ones, and September rebounds. It's a shift in one sample, not a direction over time.

What I can't tell you

Of the counterpart bodies in this data, one supplies 2,397 pairs and the next supplies 345. The third supplies 49. That third one is WPScan, where I file most of my own reports. I cannot currently characterise the CNA I use most.

There's also a definitional problem I haven't resolved. The second-largest counterpart enriches records rather than independently scoring them. Pooling an enrichment programme with a body that scores from scratch may be a category error, and every pooled figure here includes it. The pre-registered test and the per-class metric results are drawn from the largest counterpart alone, so those are clean. The five-slice table and the class table are not.

And none of this measures error. Neither body is ground truth, there is no key to mark them against, and a difference is a difference. Any reading of the above as one party being more accurate than the other is unsupported.

If you're checking a CVSS score for a WordPress plugin against a second source, the useful question isn't how often they disagree. It's what class of bug you're looking at. File inclusion and they'll almost certainly match. SQL injection and they almost certainly won't.

What I'd take from this has nothing to do with CVSS. Three times I found a pattern and immediately explained it, and three times the explanation was the only part that was wrong. Each of those died to another sample. The fourth time I fixed the measurement instead, which was right, and that one didn't die to a sample at all. It died to a tool I built for an unrelated reason, which re-derived the number and contradicted the heading it was sitting under. Being more careful was never what caught any of it. Taking another sample caught three. Building something that recomputed the claim from scratch caught the one I was most sure of.