September 10, 2026 · Disclosure Transparency

Project Glasswing, five months in · VulnCheck reconciliation, September 8, 2026

Anthropic's coordinated vulnerability disclosure dashboard, covered here in August as a genuine attempt at verifiable transparency, gave the industry something unusual: a live, cryptographically checkable account of an AI-discovery program's own performance. VulnCheck's Patrick Garrity took that invitation seriously. Following the ledger's first update since its May 22 launch, he cross-checked Anthropic's own published figures against each other and against independent maintainer accounts — and found they don't reconcile, in several specific, checkable ways.

The headline numbers, five months in

Of the 26,153 findings Anthropic claims Project Glasswing has surfaced, only 2,736 — 10.5% — have reached the public ledger at all. Of those, just 202 have actually been fixed. Put against the full claimed finding count, that's 0.8%. Anthropic's own framing acknowledges the bottleneck directly: independent human triage and review, not discovery, is the rate-limiting step, and the number disclosed is necessarily a subset of everything Mythos has actually found.

Three sets of numbers that should match, and don’t

Here's where it gets harder to explain away as simply "still early." Anthropic's own dashboard states 421 findings patched upstream, resulting in 462 CVE/GHSA advisories. But the ledger itself shows only 202 fixed findings. The problem compounds from there: Anthropic maintains three separate ledgers — the main ledger, a CVE-specific ledger, and a GHSA-specific ledger — and none of them agree with each other.

Garrity's read is direct: this suggests the ledger itself may be AI-assisted and under-reviewed, likely some combination of both.

Severity claims don’t hold up against maintainer review either

The dashboard's own severity-agreement analysis, covered here approvingly in August, reported Claude's severity assessments matching external reviewers 85.2% of the time. VulnCheck's independent look at the same underlying data tells a different story: across findings with both a Claude-assigned and a maintainer-assigned severity, Claude rated 91.5% as critical or high. Maintainers, actually reviewing the same findings, agreed on only 51.3%. That's a real, significant gap between an AI-generated severity score and the human judgment of the people who actually have to act on it — and Anthropic's ledger doesn't publish the underlying CVSS scoring to help explain where the gap comes from.

The true-positive rate looks different up close than it does in aggregate

Anthropic's claimed 91.4% true-positive rate was one of the more striking figures in the original dashboard coverage. VulnCheck checked it against a specific, named, independently verifiable account: curl maintainer Daniel Stenberg's own public writeup of his experience with Mythos-reported findings. Of five reports Stenberg received, one became a confirmed vulnerability. The other four broke down as three false positives and one real bug that didn't rise to a security vulnerability. That's a 20% hit rate on this one documented sample — a meaningfully different number than the aggregate 91.4% figure, even accounting for the fact that one maintainer's experience isn't a full statistical sample.

A real, concrete cost of the slow pipeline

VulnCheck also found 18 findings across the ledger that were already patched by someone else before Anthropic ever reported them to the maintainer — a direct, measurable cost of a disclosure pipeline that's currently converting less than 10% of claimed findings into anything a maintainer has even seen. With that much of the finding backlog still unprocessed, VulnCheck expects this specific problem, sometimes called a research collision, to compound as findings continue to age.

Worth being precise about what this does and doesn’t mean

None of this is an argument that AI-assisted vulnerability discovery isn't valuable, or that Glasswing has found nothing real — the confirmed fixes, however small a share of the total, are real security improvements in real software. What VulnCheck's analysis actually shows is a gap between the specific verification claims Anthropic has published and what an independent, methodical check of the same underlying data supports. That gap is exactly why this site covered Anthropic's dashboard as a meaningful step toward verifiability in the first place — the mechanism only earns that credit if the numbers hold up when someone actually checks them, and right now, several of the headline ones don't.

Sources

This brief synthesizes and cross-verifies publicly available primary and secondary sources, listed above. It is independent analysis, not first-party research.