Measured, Not Believed ← home
Public measurement ledger · 2026

Every claim I re-measure goes in this ledger — both ways it goes.

Independent re-computation of published quantitative claims, from raw or released data. Where I could, the test was fixed in writing before it ran; not every entry had that, and the entries make no stronger claim than their own test. When a claim holds, that is recorded with the same care as when it does not: an auditor who only ever says no isn't checking anything. Author and institution responses are described only with permission.

The work is done with an AI model doing much of the coding, computation and drafting. I choose what to test, check every number on this page against its evidence file, and answer for all of it; own mistakes are corrected in public.

31claims re-measured
6held as published
19overstated or not supported
6open / in correspondence

Astrophysics & physics

MNRAS 537, 2313 (2025) · Fermi-GBM

Kilohertz QPOs in three short GRBs, reported at p < 10⁻¹⁰.

Rebuilt the full search from public GBM photon data; the three peaks reproduce. Against a null that keeps each burst's own light-curve shape (instead of a constant rate), the same search matches or beats the observed statistic in 89%, 40% and 4.9% of simulations.

not significant under a burst-shaped null

Correspondence with the authors open; methods paper in preparation (A&A).

ApJL 983 (2025) · CHIME

A possible ~126-day period in FRB 20240209A.

The published bootstrap tests whether burst times are random — not whether they are periodic. With the burst storms kept intact, the period can be neither confirmed nor refuted from three cycles: a power test with a truly periodic source shows the folding statistic detects it in only 5–7% of trials.

not decidable from these data

In correspondence with the author; an out-of-sample test as monitoring grows is the test that matters.

MNRAS 540, 545 (2025) · BAO compilation

A local void preferred over Planck-ΛCDM at Δχ² = 28.5 across 42 "independent" BAO points.

The 42 points contain the same galaxy samples two to four times, and the Eisenstein–Hu sound horizon is converted for some points but not others. With one measurement per sample and a free H0·r_d, ΛCDM fits the paper's own data better than the void profile.

independence assumption fails

The authors' own follow-up paper (Universe 2026) independently reached the free-H0·r_d point. In correspondence with the authors.

2025 preprint · H(z) compilation

A 6.4σ descending trend in H0 with redshift.

The supernova points are calibrated on SH0ES and the BAO points on the Planck sound horizon, so the "trend" restates the H0 tension between the two anchors. With calibrations included as uncertainties: 2.3σ; with them free: 0σ.

trend = tension, restated

Manuscript submitted to A&A (Sep 2026); code and data: doi:10.5281/zenodo.23002508.

MNRAS Letters 543, L27 (2025) · Kuiper belt

A warp in the distant Kuiper belt attributed to an unseen planet.

The signal reproduces only with one shape parameter frozen; profiled over it the significance is 2.6σ / 2.4σ, and the binning choice carries a global p of 0.20. (An earlier note here, that newly catalogued objects go against the warp, was withdrawn: their orbits are too short to apply the paper's resonance filter.)

below discovery-grade once profiled

Shared with the lead author; a research note is planned.

2026 preprint · LHC combinations

A 152 GeV scalar at 5.4σ.

With the flat WW* background treated consistently and the search's own look-elsewhere effect (~13 effective trials), the combination drops to roughly 4.4σ, and one channel's status changes between the 2023 and 2025 datasets.

overstated; parked until journal publication
JADES · galaxy spin directions

A 3.39σ asymmetry in galaxy rotation directions.

Three duplicate galaxies and a σ-convention reduce it to 3.05σ; parity and spatial checks pass. Within its own framing the claim stands — no comment was warranted.

held (3.4σ → 3.05σ)
Science Advances · geomagnetism

Geomagnetic dipole moment correlates with atmospheric O₂ (r = 0.72).

Autocorrelation-corrected: N_eff ≈ 15, p = 0.002–0.005. The correlation survives the same test that kills most smoothed-series claims.

held
Fermi-LAT stacking

A 1.5 GeV line feature in stacked AGN spectra.

Re-measured with the paper's own sample size: 1.45σ.

consistent with noise
Own analysis · Fermi-LAT

The 43 GeV line candidate in galaxy clusters, re-examined.

Independent re-analysis of a published cluster line candidate; the significance depends on one cluster (Ophiuchus) and a background model choice (ρ², 3.7σ).

under review (minor revision), Physics of the Dark Universe

Earth & life sciences

JGR Oceans (2026) · coral palaeoclimate

Typhoon rainfall reconstruction correlates with AMO (r = 0.54) and solar activity (r = −0.48).

Both series are 11-year running means; the effective sample size is 6–8, and phase-randomised surrogates give p = 0.33 and 0.42. The unsmoothed annual correlations are ±0.3 and not significant. Other results in the paper held in the same tests: the change in the ENSO and PDO relationships around 1950 (surrogate p = 0.002 and 0.01) and excess multidecadal power over red noise (p = 0.02).

smoothing artefact

Shared with the authors; no comment will be submitted.

Sea level · drought indices

A published sea-level / PDSI drought correlation.

Autocorrelation-corrected: N_eff ≈ 80, p ≈ 0.003. It holds; no letter was needed.

held
Forbush decreases · cloud cover

Cloud response to Forbush decreases, supported by forecast errors.

The main correlation reproduces. The "forecast error" signal, however, equals the forecast's own +2σ cloud excess before the event — it is not additional evidence.

partly reproduces; letter with the authors
Antibody ML benchmark

A binding-affinity model's headline results.

The protein-affinity result is genuine (p = 0.005 with leakage rules enforced). The peptide ordering is not (p = 0.56).

split: half held, half didn't

Letter with the corresponding author.

Public data & official statistics

Kela · Finnish benefit statistics

A structural inconsistency in published benefit series.

Finding sent to Kela's statistics unit. Kela reviewed it, had no objections, suggested one methodological merge, and gave permission to cite their response. Submitted to a Finnish economics journal (Sep 2026).

acknowledged by the institution
DataCite · global DOI index

166,283 records carry impossible publication years.

99.3% of the index's impossible years are Buddhist-era dates: the offset is exactly +543 in 600 of 600 sampled records. Reported to the registering agency.

systematic metadata defect, root cause identified
Finnish municipal purchase-invoice data

"20.5% duplicates" in one city's open invoice data — apparently an error?

Not an error: the rate is an artefact of the export level. The data itself is sound — but totals across cities are not comparable. The ledger cuts both ways: this one defends the data.

data defended
CWA 18150 · C-UAS standard

A published European workshop agreement states detection performance…

…with sample size 0 and no confidence intervals anywhere in the document.

performance claims without measurements
EU AI Act · GPAI standardisation

Statistical-rigour comments into the drafting process.

Eight of eight submitted comments accepted so far (four more under moderation), including independence requirements and LLM-judge validation. Separately: the 10²⁵ FLOP threshold lacks a tolerance statement — noted to the AI Office (doi:10.5281/zenodo.22966931).

accepted into the draft
Finnish Ministry of Social Affairs and Health · tender EF-57779

Comments on a ministry's draft tender, then the published tender.

The ministry opened its draft tender for an evaluation contract to comment. Of the ten points raised, nine changed in that direction in the published version and one was clarified. The data section now requires bidders to state how sources are chosen, how they are made comparable, and how uncertain they are. One change traces to these comments directly: the added sentence that lets a new firm without a confirmed financial year bid is the wording proposed. Others commented too, so the other changes are consistent with the comments, not proven to follow from them.

adopted into the tender

AI evaluation & benchmarks

Eleven public case files

The original case-file series: judges, base rates, rank stability, hidden dimensions.

Eleven audits of AI benchmarks and leaderboards from released data — ten overstated, one certified as holding. Now with a deposited rank-stability study (doi:10.5281/zenodo.22541065).

see the case files →

Full write-ups on the front page.

Kaggle · 653 leaderboards · two test slices

Neighbouring leaderboard ranks are a coin flip. Measured, then replicated on 653 boards.

Kaggle scores every competition on two disjoint test slices, so rank noise is observed, not modelled. The August measurement over 19 competitions replicated on an independently built dataset of 1,157 finished boards, method file unchanged and pre-registered: pairs closer than one local movement keep their order in 55.1% of 2.2M pairs, pairs further apart in 90.8% of 7.3M; a shuffled control stays at 50%. 58 boards invert and are kept in every figure.

replicated, 19 → 653 boards

Report, pre-registration, code and per-board results: doi:10.5281/zenodo.23017379 · data: Kaggle Shakeup Atlas (G. Mamarin).

Writing systems

Rongorongo · corpus statistics

A machine-readable transcription of the rongorongo corpus, with its error structure measured.

Deposited (doi:10.5281/zenodo.22825533, code 10.5281/zenodo.22834627); under review at JDMDH. A companion measurement: the same proto-Elamite corpus yields 1,738 / 680 / 468 signs under three published counting conventions — sign-count disagreements are notation, not discovery.

under review

Method & tools

Published method

R = spread ÷ effect: a one-number robustness check for published claims.

Method paper deposited (doi:10.5281/zenodo.22162050). The periodicity work above runs on an open null-model toolkit (burst-shaped nulls, selection-aware calibration, trials-factor verification) that ships with each paper.

published

Every entry above was computed from public or released data, with the test pre-registered in a version-controlled file before it ran. Where an author or institution is quoted, it is with permission. If you rely on a number — a benchmark, a procurement criterion, a published effect — and want it re-measured the same way: ilpo@ipezygj.com.

How to verify, mechanically. The pre-registration claim is checkable, not a promise: deposits are moving to include a PREREG_PROOF.txt — the git first-add commit and UTC timestamp of the pre-registration and of every result file, generated by an open script, so a reader can confirm the test was locked before any result existed (first deposit with one: doi:10.5281/zenodo.23017379, the Kaggle threshold above). The counts on this page are re-derived from the entries by a checker that runs on every update.