Append-only AI overlay records; the canon claim file is never mutated.
Mixedconfidence: moderate · Fable (Claude), via Cursor · run 2026-08-22-fable-transients-1 · unreviewed draft
Replicates on the working catalog, strengthens under internal ML vetting, and fails in externally vetted subsets with a corrected background model; the preregistered multi-pipeline test (TRN-R005) is the only exit from this loop.
Mixedconfidence: high · Gemini 3.1 Pro (Google) — independent check run via gemini-3.1-pro-preview · run 2026-08-23-check-gemini · unreviewed draft
The Earth-shadow deficit successfully replicates when independent statistics are run on the proponents' original working catalog, but fails in independently extracted or heavily vetted datasets. The strongest opposing consideration is that the initial analysis assumed a uniform background distribution that the data demonstrably violate, with the deficit vanishing when spatial heterogeneity is properly modeled.
Mixedconfidence: moderate · GPT-5.1 (OpenAI) — independent check run via gpt-5.1 · run 2026-08-23-check-gpt · unreviewed draft
Using the 107,875-object working catalog, Doherty reproduces the Earth-shadow deficit originally reported by the VASCO team, and Bruehl et al. find that it persists and even strengthens among candidates scored more likely to be real by an ML classifier (TRN-E010, TRN-E015), supporting the existence of a deficit in that catalog. In contrast, Watters et al. show that the assumed uniform background is empirically false, that candidate spatial distributions resemble plate/scan artifacts, and that with a more appropriate null the deficit is not significant in more vetted subsets (TRN-E011); the proponents’ rebuttal emphasizes power but does not re-establish the effect (TRN-E012). Because the signal is robust within an unvalidated, artifact-rich working sample but collapses when stricter vetting and background modeling are applied, the status of a genuine physical shadow deficit is genuinely mixed. The strongest opposing consideration to “real deficit” is that the catalog itself appears heavily contaminated and structurally biased, so the observed pattern may well be an artifact of selection rather than of the sky.
Mixedconfidence: high · Grok 4.6 (xAI) — independent check run via grok-4.6 · run 2026-08-23-check-grok · unreviewed draft
Independent code on the author-supplied 107,875-row catalog reproduces the 0.46% versus 1.4% shadow comparison, and a later ML scoring paper finds the deficit persisting, even strengthening, among candidates judged likelier to be real. Watters et al. show the uniform-random null is false for these spatially heterogeneous, edge-concentrated features and find no significant deficit in more heavily vetted related datasets; the proponents' reply is about power, not a counter-demonstration on a cleaned independent catalog. The strongest opposing consideration is that a deficit computed against an invalid background, and absent from the hardest-vetted subsets, is a property of catalog construction rather than of orbital geometry.
Mixedconfidence: moderate · Kimi K3 (Moonshot, via Venice) — independent check run via kimi-k3 · run 2026-08-23-check-kimi · unreviewed draft
Within the author-supplied 107,875-row catalog the deficit reproduces under independent code (Doherty: 0.46% observed versus ~1.4% geometric expectation) and reportedly strengthens among ML-vetted candidates (Bruehl et al.). But Watters et al. show the uniform-background assumption the expectation rests on is demonstrably false for these data — features are edge-concentrated and clustered like plate artifacts — and find no significant deficit in heavily vetted subsets; the proponents' reply argues power, not a corrected recomputation. The strongest consideration against the skeptical reading is that the critique used related-but-not-identical datasets, so the exact working catalog has arguably never faced a preregistered proper-null test. Conditional-on-catalog reproducibility is established; reality beyond the catalog is not — that is precisely what "mixed" is for.
Mixedconfidence: moderate · Opus 5 (Anthropic) — independent check run via claude-opus-5 · run 2026-08-23-check-opus · unreviewed draft
The observed shadow fraction (0.46%, and 0.45% in a center-of-plate subset) reproduces under independently written code, and persists — indeed strengthens — among ML-scored likelier-real candidates, so the arithmetic on the working catalog is not in dispute. But the claim as stated asserts a *deficit*, which is defined against the ~1.4% geometric expectation, and Watters et al. show that expectation assumes a uniform background the features demonstrably violate (edge- and corner-concentrated, clustered), with no significant deficit surviving in vetted subsets. The strongest opposing consideration for the skeptical side is the proponents' power objection: the vetted subsets are roughly twentyfold smaller and were assembled for another purpose, so their null is not a clean refutation either — which is precisely why "mixed" rather than "contradicted" or "provisionally_supported" is the warranted reading.
Mixedconfidence: moderate · anthropic/claude-fable-5 (scheduled maintenance run) · run 2026-08-24-auto-5g6l · unreviewed draft
Unchanged: survives re-analysis and internal ML vetting on the working catalog (TRN-E010, TRN-E015) with one confounder closed (TRN-E017), but fails under the corrected non-uniform background model (TRN-E011), and no independently constructed catalog reproduces it. The new material does not address the Watters null-distribution objection.
Mixedconfidence: moderate · anthropic/claude-fable-5 (scheduled maintenance run) · run 2026-08-24-auto-fuhc · unreviewed draft
Replicates on the working catalog (TRN-E010), strengthens under internal ML vetting (TRN-E015), and one confounder route is closed (TRN-E017) — but the corrected non-uniform background model (TRN-E011) still eliminates it, and no independently constructed catalog reproduces it. The new batch does not touch the Watters objections.
Mixedconfidence: moderate · anthropic/claude-fable-5 (scheduled maintenance run) · run 2026-08-24-auto-ig56 · unreviewed draft
Reproduces under independent statistics on the author-supplied rows (TRN-E010) and persists under internal ML scoring (TRN-E015), but disappears when spatial non-uniformity of the background is modeled on related datasets (TRN-E011). Re-analysis survives; reconstruction does not.
Mixedconfidence: moderate · anthropic/claude-fable-5 (scheduled maintenance run) · run 2026-08-24-auto-pu58 · unreviewed draft
Replicates on the working catalog (TRN-E010), strengthens under internal ML vetting (TRN-E015), and TRN-E017 closes the geomagnetic-sensitivity route — but it still fails under the corrected non-uniform background model (TRN-E011), and no independently constructed catalog has reproduced it. The Watters objections are untouched by the new material.
Mixedconfidence: high · Gemini 3.1 Pro (Google) — independent check run via gemini-3.1-pro-preview · run 2026-08-24-check-gemini-r2 · unreviewed draft
The shadow deficit replicates robustly when independent statisticians and machine-learning models analyze the original 107,875-object working catalog. However, comprehensive skeptical reanalysis demonstrates that this working catalog violates the uniform-background assumption required for the geometry, as feature counts are spatially heterogeneous and clustered at plate edges. The strongest opposing consideration is that when the analysis is applied to more heavily vetted candidate subsets that properly model the background spatial non-uniformity, the shadow deficit completely fails to reach statistical significance.
Mixedconfidence: high · Gemini 3.1 Pro (Google) — independent check run via gemini-3.1-pro-preview · run 2026-08-24-check-gemini · unreviewed draft
The Earth-shadow deficit reliably reproduces when independent statisticians analyze the original working catalog and persists among ML-vetted candidates. However, a comprehensive critique demonstrates that the background spatial distribution is highly non-uniform and finds no significant deficit when a proper null distribution is applied to vetted subsets. The correlation exists conditionally on the specific working catalog, but its validity as a physical signal is heavily contested.
Mixedconfidence: moderate · GPT-5.1 (OpenAI) — independent check run via gpt-5.1 · run 2026-08-24-check-gpt-r2 · unreviewed draft
Using the ~107,875-object working catalog, Doherty’s independent code reproduces the reported shadow deficit (0.46% of sources in the umbra vs ~1.4% expectation; TRN-E010), and Bruehl et al.’s ML artifact-scoring finds the deficit strongest among candidates scored more likely real (TRN-E015). However, Watters et al. demonstrate that the spatial distribution of features violates the uniform-background assumption and that in vetted subsets with a more appropriate null, no significant deficit remains (TRN-E011). Moreover, the working catalog itself is acknowledged to contain many false positives and lacks independent reconstruction (TRN-E016), and no cross-pipeline, preregistered test exists. The strongest supporting consideration is the consistent replication of the numerical deficit on the same catalog; the strongest opposing is that it seems to be a property of that specific, unvalidated catalog under a flawed null model.
Mixedconfidence: moderate · GPT-5.1 (OpenAI) — independent check run via gpt-5.1 · run 2026-08-24-check-gpt · unreviewed draft
Within the ~107,875-object VASCO working catalog, independent statistical analysis by Doherty reproduces the reported Earth-shadow deficit (TRN-E010), and ML artifact-scoring suggests the deficit is strongest among candidates judged less artifact-like (TRN-E015). However, Watters et al. show that the necessary assumption of a uniform background is violated—the candidates are highly non-uniform, concentrated toward plate edges and corners and in clusters—and find no significant deficit once this is modeled properly in closely related datasets (TRN-E011). Because the shadow signal has not yet been demonstrated on independently constructed catalogs and appears sensitive to catalog definition and null modeling, it cannot be regarded as either clearly present or clearly absent. The strongest opposing consideration is that when more heavily vetted subsets and realistic background models are used, the deficit disappears, suggesting it may be an artifact of catalog composition and simplistic null assumptions.
Mixedconfidence: moderate · Grok 4.6 (xAI) — independent check run via grok-4.6 · run 2026-08-24-check-grok-r2 · unreviewed draft
On the author-supplied 107,875-row catalog, Doherty independently reproduces the 0.46 percent versus roughly 1.4 percent umbral deficit (TRN-E010), and Bruehl et al. find it persisting and largest among ML-scored high-probability transients (TRN-E015). Watters et al. show the uniform-background assumption is false — candidates are edge-concentrated and spatially heterogeneous — and find no significant deficit in more heavily vetted related datasets (TRN-E011); the proponents' reply is a power argument, not a counter-demonstration on a rebuilt catalog (TRN-E012). The strongest opposing consideration is structural: every positive replication is conditional on the same unvalidated working catalog, which is the quantity under dispute.
Mixedconfidence: high · Grok 4.6 (xAI) — independent check run via grok-4.6 · run 2026-08-24-check-grok · unreviewed draft
Doherty reproduces 0.46 percent of working-catalog sources in the umbral cone against a geometric expectation of about 1.4 percent, and Bruehl et al. find the deficit stronger among candidates scored more likely to be real. Watters et al. show the uniform-background assumption is false — counts are edge-concentrated and clustered — and find no significant deficit in more heavily vetted subsets; the proponents' reply is about statistical power, not a counter-demonstration on a cleaned independent catalog. Strongest opposing consideration: a geometric signature that survives only inside one working catalog, and vanishes when the background the data actually have is used, is not yet a property of the sky.
Mixedconfidence: high · Kimi K3 (Moonshot, via Venice) — independent check run via kimi-k3 · run 2026-08-24-check-kimi-r2 · unreviewed draft
The pattern of replication and failure is cleanly documented and cuts along the dataset-definition axis: Doherty reproduced the umbral deficit (0.46% vs ~1.4% geometric expectation) with independent code on the author-supplied catalog (E010), and ML artifact-scoring reportedly leaves it significant and largest among likelier-real candidates (E015) — but Watters et al. demonstrate the uniform-background null is false, derive a proper null, and find no significant deficit in the hardest-vetted subsets (E011), with the proponents answering via a power argument rather than a counter-demonstration (E012). No independently constructed catalog has ever been tested for the effect. The strongest opposing consideration to "mixed" is that the within-catalog replications are genuine and multiple — but conditional replication of an unvalidated catalog is exactly what a catalog artifact would also produce. Mixed, high confidence.
Mixedconfidence: moderate · Kimi K3 (Moonshot, via Venice) — independent check run via kimi-k3 · run 2026-08-24-check-kimi · unreviewed draft
The deficit is real arithmetic against a discredited expectation. Independent statistics on the author-supplied catalog reproduce it (Doherty: 0.46% observed against ~1.4% geometric expectation, holding in a center-of-plate subset), and ML artifact-scoring reportedly leaves it significant and largest among the likeliest-real candidates. But Watters et al. show the uniform-random background that defines the 1.4% expectation is false for these data — the features are spatially heterogeneous, edge-concentrated, with geometric clusters — and find no significant deficit under a derived null or in heavily vetted subsets; the proponents' rebuttal argues statistical power rather than offering a counter-demonstration. The strongest opposing consideration to the skeptical reading is that removing visually recognizable artifact classes strengthens the signal, which a pure artifact population would not obviously do. Reproducible conditional on the working catalog, unestablished beyond it: mixed is the substantive grade.
Mixedconfidence: moderate · Opus 5 (Anthropic) — independent check run via claude-opus-5 · run 2026-08-24-check-opus-r2 · unreviewed draft
The arithmetic reproduces under independently written code on the author-supplied catalog (0.46% umbral against ~1.4% geometric expectation; TRN-E010) and survives ML artifact-scoring, strengthening among higher-probability candidates (TRN-E015). But the word "deficit" is defined by the expectation, and TRN-E011 shows the uniform-random background the expectation assumes is demonstrably false for these features — counts rise toward corners and edges — with no significant deficit once a proper null is derived or once heavily vetted subsets are used. So the claim is true as a description of one catalog against one questionable baseline and unestablished as a statement about a real spatial deficit. The strongest opposing consideration to reading this as artifact is TRN-E012's power argument: the vetted subsets are twentyfold smaller and could hide a genuine effect.
Mixedconfidence: moderate · Opus 5 (Anthropic) — independent check run via claude-opus-5 · run 2026-08-24-check-opus · unreviewed draft
The descriptive number reproduces under independently written code on the same rows (0.46% umbral, 0.45% center-of-plate, TRN-E010) and strengthens among ML-scored likelier-real candidates (TRN-E015), but the word "deficit" requires a null, and Watters et al. show the uniform-random background the 1.4% expectation assumes is falsified by the data's own spatial heterogeneity, with no significant deficit surviving in vetted subsets (TRN-E011). The proponents' reply contests power and subset provenance rather than demonstrating the effect against a proper null (TRN-E012). Mixed is the honest reading: real as a property of one working catalog measured against a wrong expectation, unestablished as a sky signal. The strongest opposing consideration is the proponents' point that Watters worked from related public datasets rather than the exact 107,875 rows.
Mixedconfidence: moderate · anthropic/claude-opus-5 (scheduled maintenance run) · run 2026-08-25-auto-gvfu · unreviewed draft
Replicates on the author-supplied working catalog and survives artifact scoring (TRN-E010, TRN-E015), with one environmental confounder closed (TRN-E017), but fails once the non-uniform background null is derived properly (TRN-E011) and has never been reproduced from an independently constructed catalog.
Mixedconfidence: moderate · anthropic/claude-fable-5 (scheduled maintenance run) · run 2026-08-25-auto-nh4y · unreviewed draft
Unchanged by the new material: replicates on the working catalog (TRN-E010, TRN-E015), one confounder closed (TRN-E017), but still fails under the corrected non-uniform background model (TRN-E011) and no independently constructed catalog reproduces it.
Mixedconfidence: moderate · anthropic/claude-opus-5 (scheduled maintenance run) · run 2026-08-25-auto-v4pc · unreviewed draft
Replicates on the author-supplied working catalog and survives artifact scoring (TRN-E010, TRN-E015), with one environmental confounder closed (TRN-E017), but fails once the non-uniform background null is derived properly (TRN-E011) and has never been reproduced from an independently constructed catalog.
Mixedconfidence: moderate · anthropic/claude-opus-5 (scheduled maintenance run) · run 2026-08-25-auto-y5cr · unreviewed draft
Replicates on the author-supplied working catalog and survives artifact scoring (TRN-E010, TRN-E015), with one environmental confounder closed (TRN-E017), but fails once the non-uniform background null is derived properly (TRN-E011) and has never been reproduced from an independently constructed catalog.
Mixedconfidence: high · Gemini 3.1 Pro (Google) — independent check run via gemini-3.1-pro-preview · run 2026-08-25-check-gemini · unreviewed draft
Independent analysts reproduce the Earth-shadow deficit when using the exact candidate rows supplied by the authors, even when augmented by ML artifact scoring. However, the signal vanishes completely in reanalyses that correct for non-uniform spatial clustering across the plates. The strongest opposing consideration is that the uniform-random background assumption underpinning the deficit claim is demonstrably false for these images.
Mixedconfidence: moderate · GPT-5.1 (OpenAI) — independent check run via gpt-5.1 · run 2026-08-25-check-gpt · unreviewed draft
Within the VASCO 107,875‑object working catalog, independent statistics reproduce an Earth‑shadow deficit (TRN‑E010), and ML scoring suggests the effect strengthens among candidates judged less artifact-like (TRN‑E015), with Cann’s geomagnetic work removing one specific environmental confound (TRN‑E017). However, Watters et al. demonstrate that the assumed uniform background is incorrect and find no significant deficit in more stringently vetted subsets (TRN‑E011), and Hayes’s independently constructed catalog has not yet shown a corresponding signature. The strongest opposing consideration is that the deficit’s significance is highly sensitive to catalog definition and background modeling; where those are most conservative it disappears, so the current evidence neither securely establishes nor cleanly refutes the claimed pattern.
Mixedconfidence: moderate · Grok 4.6 (xAI) — independent check run via grok-4.6 · run 2026-08-25-check-grok · unreviewed draft
On the author-supplied ~107,875-object catalog the deficit is real and reproducible: Doherty independently codes 0.46% of sources in the umbral cone against a ~1.4% geometric expectation, and Bruehl et al. find the deficit persisting and even largest among ML-scored “likelier-real” candidates. Watters et al. show the uniform-random background assumption is false — features are edge-concentrated and spatially heterogeneous — and find no significant deficit in more heavily vetted subsets; the proponents’ reply is about power, not a counter-demonstration on a cleaned catalog. The strongest opposing consideration is therefore dataset definition: the signal is a property of the working catalog under a false null, not a demonstrated property of the sky, and no independently extracted catalog has yet reproduced it.
Mixedconfidence: moderate · Kimi K3 (Moonshot, via Venice) — independent check run via kimi-k3 · run 2026-08-25-check-kimi · unreviewed draft
The shadow deficit is real as a property of the author-supplied working catalog: independently coded statistics reproduce it (Doherty: 0.46% vs ~1.4% expectation), and it reportedly strengthens among ML-vetted candidates (Bruehl et al., p < .001, largest among highest-probability sources). It is also true that Watters et al. show the uniform-background null is false for these data and find no significant deficit in more heavily vetted subsets, and no independently constructed catalog has ever shown the effect — Hayes's rebuild does not report one, and the preregistered multi-pipeline test (TRN-R005) has not been run. The strongest opposing consideration is the power rebuttal (TRN-E012): the vetted subsets are twenty-fold smaller and may hide a real deficit. Both directions are documented; "mixed" is the honest token, not a hedge.
Mixedconfidence: moderate · Opus 5 (Anthropic) — independent check run via claude-opus-5 · run 2026-08-25-check-opus · unreviewed draft
The raw counts reproduce under independently written code on the author-supplied catalog (0.46% umbral, 0.45% in a centre-of-plate subset), and the effect persists and strengthens among ML-scored likelier-real candidates — those are real internal robustness results. But the claim's operative word is "deficit," which is defined relative to the ~1.4% geometric expectation, and Watters et al. show the uniform- random background that expectation assumes is demonstrably false for these data, with no significant deficit surviving in more heavily vetted subsets. The proponents' reply concerns statistical power, not a counter-demonstration on a clean sample. The strongest opposing consideration to a purely negative reading is that the vetted subsets are twenty-fold smaller and were built for another purpose, so the null is not decisive either — hence mixed rather than contradicted.
Mixedconfidence: moderate · anthropic/claude-opus-5 (scheduled maintenance run) · run 2026-08-26-auto-197y · unreviewed draft
Replicates on the author-supplied working catalog and survives ML artifact scoring (TRN-E010, TRN-E015), with one environmental confounder closed (TRN-E017), but fails once the non-uniform background null is derived properly (TRN-E011) and has never been reproduced from an independently constructed catalog.
Mixedconfidence: high · Gemini 3.1 Pro (Google) — independent check run via gemini-3.1-pro-preview · run 2026-08-26-check-gemini · unreviewed draft
The Earth-shadow deficit reproduces reliably when using the original VASCO working catalog and machine-learning artifact vetting (TRN-E010, TRN-E015). However, it is highly sensitive to the underlying dataset definition. The strongest opposing consideration is the Watters reanalysis (TRN-E011), which shows that when the spatial heterogeneity of the background is properly modeled and more heavily vetted subsets are used, the statistical significance of the shadow deficit completely disappears.
Mixedconfidence: moderate · GPT-5.1 (OpenAI) — independent check run via gpt-5.1 · run 2026-08-26-check-gpt · unreviewed draft
The Earth-shadow deficit is reproduced by independent statistical code on the VASCO 107,875-object catalog and appears stronger among candidates scored more likely real by an ML classifier (TRN-E010, TRN-E015), suggesting it is not trivially produced by obvious artifacts within that dataset. However, Watters et al. show that the uniform background assumption is violated, with strong spatial inhomogeneities and edge clustering, and that when a more realistic null is used and heavily vetted subsets are chosen, the deficit is not significant (TRN-E011). The strongest opposing factor is that this purported effect has not been reproduced in independently constructed catalogs (Hayes’s work does not address shadow, but his findings on related statistics argue catalog dependence), so the balance of evidence is that a shadow deficit may exist in the working catalog but is not yet established as a robust property of the sky.
Mixedconfidence: high · Grok 4.6 (xAI) — independent check run via grok-4.6 · run 2026-08-26-check-grok · unreviewed draft
Independent code on the author-supplied 107,875-row catalog reproduces 0.46% of sources in the umbral cone against a ~1.4% geometric expectation, and Bruehl et al. find the deficit stronger among ML-high-probability rows. Watters et al. show the uniform-background assumption is false — counts are edge-concentrated and clustered — and recover no significant deficit in more heavily vetted subsets; the proponents' reply is about statistical power, not a counter-demonstration on a cleaned catalog. Strongest opposing consideration: a geometric signal that exists only inside one unvalidated working sample, and vanishes when the background the plates actually have is used, has not been shown to be a property of the sky.
Mixedconfidence: moderate · Kimi K3 (Moonshot, via Venice) — independent check run via kimi-k3 · run 2026-08-26-check-kimi · unreviewed draft
As stated, the deficit exists in the working catalog and reproduces under independent code (TRN-E010) and after ML artifact-scoring (TRN-E015). But the geometric expectation assumes a uniform background the data demonstrably violate, and in the most heavily vetted subsets the deficit is not significant (TRN-E011); the proponents' reply (TRN-E012) argues statistical power rather than offering a counter-demonstration. The strongest opposing consideration is that the deficit is computed against a null already falsified on these data. Mixed is substantive: replicated within the lineage, unestablished beyond it, and no independently constructed catalog has ever been tested for the shadow signal.
Mixedconfidence: moderate · Opus 5 (Anthropic) — independent check run via claude-opus-5 · run 2026-08-26-check-opus · unreviewed draft
The raw numbers replicate under independently written code on the author-supplied catalog (0.46% umbral against ~1.4% geometric, TRN-E010) and survive ML artifact-scoring, growing among candidates scored likelier real (TRN-E015). But the deficit is defined against a uniform-random null that TRN-E011 shows is false for these features — spatially heterogeneous, edge- and corner-concentrated — and with a proper null and more vetted subsets no significant deficit appears; the proponents' reply (TRN-E012) argues power, not counter-demonstration. Mixed is the accurate reading: the observed fraction is a reproducible property of the catalog, while the inference "deficit" depends on a background model that has been credibly contested. The strongest opposing consideration to my grade is that Watters worked from related-but-not-identical public datasets, so the critique's reach over the exact 107,875 rows is itself arguable.
Mixedconfidence: moderate · anthropic/claude-fable-5 (scheduled maintenance run) · run 2026-09-01-auto-5ld8 · unreviewed draft
Replicates on the author-supplied rows (TRN-E010, TRN-E015) with one confounder closed (TRN-E017), but fails under the corrected non-uniform null (TRN-E011) and has never emerged from an independently built catalog. The split tracks dataset lineage, not analysis quality.
Mixedconfidence: moderate · anthropic/claude-fable-5 (scheduled maintenance run) · run 2026-09-01-auto-7g41 · unreviewed draft
Replicates on the author-supplied rows (TRN-E010, TRN-E015) with one confounder closed (TRN-E017), but fails under the corrected non-uniform null (TRN-E011) and has never emerged from an independently built catalog.
Mixedconfidence: moderate · anthropic/claude-fable-5 (scheduled maintenance run) · run 2026-09-01-auto-ehrl · unreviewed draft
Replicates on the author-supplied rows (TRN-E010, TRN-E015) with one environmental confounder closed (TRN-E017), but fails under the corrected non-uniform null (TRN-E011) and has never emerged from an independently built catalog.
Mixedconfidence: moderate · anthropic/claude-fable-5 (scheduled maintenance run) · run 2026-09-01-auto-i52f · unreviewed draft
Replicates on the author-supplied rows (TRN-E010, TRN-E015) with one environmental confounder closed (TRN-E017), but fails under a spatially heterogeneous null (TRN-E011) and has never emerged from an independently built catalog.
Mixedconfidence: moderate · anthropic/claude-fable-5 (scheduled maintenance run) · run 2026-09-01-auto-k2f0 · unreviewed draft
Replicates on the author-supplied rows (TRN-E010, TRN-E015) with one confounder closed (TRN-E017), but fails under the corrected non-uniform null (TRN-E011) and has never emerged from an independently built catalog.
Mixedconfidence: moderate · anthropic/claude-fable-5 (scheduled maintenance run) · run 2026-09-01-auto-m29a · unreviewed draft
Replicates on the author-supplied rows (TRN-E010, TRN-E015) with one environmental confounder closed (TRN-E017), but fails under a properly non-uniform spatial null (TRN-E011) and has never emerged from an independently built catalog.
Mixedconfidence: moderate · anthropic/claude-fable-5 (scheduled maintenance run) · run 2026-09-01-auto-mklk · unreviewed draft
Replicates on author-supplied rows (TRN-E010, TRN-E015) with one confounder closed (TRN-E017), but fails under the corrected non-uniform null (TRN-E011) and has never emerged from an independently built catalog.
Mixedconfidence: moderate · anthropic/claude-fable-5 (scheduled maintenance run) · run 2026-09-01-auto-sffb · unreviewed draft
Replicates on the author-supplied rows (TRN-E010, TRN-E015) with one confounder closed (TRN-E017), but fails under a properly non-uniform null (TRN-E011) and has never emerged from an independently built catalog.
Mixedconfidence: high · Gemini 3.1 Pro (Google) — independent check run via gemini-3.1-pro-preview · run 2026-09-01-check-gemini · unreviewed draft
The Earth-shadow deficit strongly replicates when analyses are run strictly against the authors' ~107,875-object working catalog, even post-ML vetting. It completely fails, however, in the Watters reanalysis, which demonstrates that the base catalog wildly violates the uniform-random spatial background assumption required to make the shadow deficit statistically significant.
Mixedconfidence: moderate · GPT-5.1 (OpenAI) — independent check run via gpt-5.1 · run 2026-09-01-check-gpt · unreviewed draft
Using the VASCO 107,875-object working catalog, both the original authors and Doherty reproduce a deficit of candidates inside the Earth-shadow cone relative to geometric expectation, and Bruehl et al.’s ML scoring find it strongest among candidates deemed more likely real, suggesting it is not trivially due to obvious defects. However, Watters et al. demonstrate that the features’ spatial distribution is highly non-uniform and edge-concentrated, invalidating the original uniform-background assumption, and they find no significant deficit once an appropriate null is used and subsets are more strictly vetted. The signal’s dependence on a particular, contested catalog and background model means its reality is uncertain. The strongest opposing consideration is this failure to reproduce a significant deficit under more conservative data selection and null modeling, and the lack of corroboration in independently extracted catalogs.
Mixedconfidence: high · Grok 4.6 (xAI) — independent check run via grok-4.6 · run 2026-09-01-check-grok · unreviewed draft
On the author-supplied 107,875-object catalog the deficit reproduces in independently written code and is reported stronger among ML-scored high-probability candidates. On related but more heavily vetted sets, and once the false uniform-background assumption is dropped, Watters et al. find no significant deficit. The split is the result: the signal is a property of one working sample, not a demonstrated property of the sky. The strongest opposing consideration is that spatial heterogeneity, edge concentration, and geometric clusters look like plate and scan artifacts, which is precisely the background the original analysis did not model.
Mixedconfidence: moderate · Kimi K3 (Moonshot, via Venice) — independent check run via kimi-k3 · run 2026-09-01-check-kimi · unreviewed draft
The computed deficit replicates with independent code on the author-supplied catalog (TRN-E010) and reportedly strengthens among ML-scored candidates (TRN-E015). But Watters et al. show the uniform-random null is false for these data — features are edge-concentrated and clustered like plate and scan artifacts — and find no significant deficit under proper nulls and tighter vetting (TRN-E011). The claim as stated is thus reproducible as arithmetic and unestablished as inference; that split is what mixed is for. The strongest opposing consideration is the proponents' power argument (TRN-E012): the vetted subsets are far smaller and proxy the working catalog imperfectly, so the null results could hide a real effect — a question only a preregistered multi-pipeline analysis (TRN-R005) can settle.
Mixedconfidence: moderate · Opus 5 (Anthropic) — independent check run via claude-opus-5 · run 2026-09-01-check-opus · unreviewed draft
Read literally — this catalog shows these counts against that geometric expectation — the arithmetic is independently reproduced by TRN-E010 (0.46%, 0.45% center-of-plate) and persists under ML artifact scoring (TRN-E015). But the word "deficit" imports a null, and TRN-E011 demonstrates the uniform-random background is false for these data, which are edge- and corner-concentrated with geometric clusters and voids; with a proper null the significance disappears, and no deficit survives in the more heavily vetted subsets. TRN-E012 replies about statistical power rather than supplying a corrected analysis. The strongest opposing consideration for a harsher verdict is that TRN-E011 works from related public datasets rather than the exact 107,875 rows, so the proxy question is genuinely unsettled — hence mixed rather than contradicted, pending TRN-R005.
Mixedconfidence: moderate · anthropic/claude-fable-5 (scheduled maintenance run) · run 2026-09-02-auto-1gjc · unreviewed draft
Replicates on the author-supplied rows (TRN-E010, TRN-E015) with one confounder closed (TRN-E017), but fails under a properly non-uniform null (TRN-E011) and has never emerged from an independently built catalog.
Mixedconfidence: moderate · anthropic/claude-fable-5 (scheduled maintenance run) · run 2026-09-02-auto-1zex · unreviewed draft
Replicates on the author-supplied rows (TRN-E010, TRN-E015) with one confounder closed (TRN-E017), but fails under a corrected non-uniform null (TRN-E011) and has never emerged from an independently built catalog.
Mixedconfidence: moderate · none — mechanical migration (scripts/migrations/2026-09-08-editions.ts) · run 2026-09-08-migration-transients · unreviewed draft
Replicated by independent code on the same catalog (Doherty 2026) and reportedly stronger among ML-vetted candidates (Bruehl et al. 2026) — but Watters et al. 2026 show the uniform-background assumption is false for these data and find no significant deficit in more heavily vetted subsets, and the proponents' reply is about statistical power, not a counter-demonstration. Reproducible conditional on the working catalog; unestablished beyond it.
Mixedconfidence: moderate · Gemini 3.8 Flash (Google) — independent check run via gemini-3.8-flash · run 2026-09-10-check-gemini-183351 · unreviewed draft
An Earth-shadow deficit of approximately 0.46% versus 1.4% expected is present in the author-supplied 107,875-object working catalog and was verified by an independent statistical replication. However, a rigorous critical evaluation demonstrated that the underlying assumption of a uniform spatial defect distribution is invalid, plate-edge defect gradients bias the calculation, and the deficit is statistically insignificant when using an empirical null distribution. The strongest opposing consideration is that the deficit reportedly persists and intensifies when restricting analysis to higher-confidence candidates scored by machine learning.
Mixedconfidence: moderate · GLM 5.3 Flash (Z.ai, via Venice) — independent check run via z-ai-glm-5-3-flash · run 2026-09-10-check-glm-183351 · unreviewed draft
The deficit replicates under independently written code on the original rows (0.46% in-shadow versus ~1.4% expected, TRN-E010), persists and strengthens under ML artifact scoring (TRN-E015), and is reported significant at all tested altitudes (TRN-E025). Against it, Watters shows the uniform-random background assumption is false and finds no statistically significant deficit under the proper null (TRN-E011), and the reported magnitude is conditional on post-hoc spatial cuts (TRN-E051). Strongest opposing consideration: TRN-E011's corrected-null result; the proponents' rebuttal (TRN-E012) presses statistical power but does not directly answer the false-background charge.
Well supportedconfidence: moderate · GPT-5.6 Sol (OpenAI) — independent check run via gpt-5.6-sol · run 2026-09-10-check-gpt-183351 · unreviewed draft
TRN-E010 independently reproduces the stated 0.46 percent shadow fraction against an approximately 1.4 percent geometric expectation on the supplied working catalog. TRN-E015 and the later quality-filtered analyses report persistence or strengthening within that same lineage. The strongest opposing consideration is TRN-E011's finding that the uniform spatial background is false and that a more appropriate null removes statistical significance. Thus the catalog-level numerical deficit is well supported, but its interpretation as a physical Earth-shadow effect is not.
Mixedconfidence: moderate · Grok 4.5 (xAI) — independent check run via grok-4.5 · run 2026-09-10-check-grok-183351 · unreviewed draft
Doherty independently reproduces a ~0.46 % versus ~1.4 % umbral deficit on the author-supplied 107,875-row catalog; Bruehl ML scoring and later modelling find the deficit persists and strengthens in high- probability subsets. Watters, using related public datasets and a non-uniform spatial null, finds no significant deficit and shows feature density rises toward plate edges. The conflict is dataset definition and null model, not arithmetic error. Strongest opposing consideration: if the working catalog is dominated by unvalidated rows and spatially heterogeneous defects, the geometric shadow signal is illusory.
Mixedconfidence: moderate · Opus 5 (Anthropic) — independent check run via claude-opus-5 · run 2026-09-10-check-opus-183351 · unreviewed draft
The arithmetic reproduces independently on the authors' rows (TRN-E010: 0.46% in the umbral cone, 0.45% in a center-of-plate subset, against ~1.4% geometric expectation), so the claim's stated numbers are not a coding error. But the geometric expectation is the load-bearing term, and TRN-E011 shows the uniform-random background assumption underlying it is false, with no significant deficit surviving a properly derived null. The group's own later recomputation (TRN-E025, TRN-E051) makes the deficit's magnitude a function of assumed altitude and of an after-the-fact spatial cut, and concedes edge effects — conceding part of the critique. Mixed is the accurate reading: robust as a description of the catalog, unestablished as a measurement of the sky.
Mixedconfidence: moderate · claude-fable-5-1 · run 2026-09-10-edition-183348 · unreviewed draft
Supporting: TRN-E010 (independent code on the author-supplied catalog reproduces 0.46% vs ~1.4%), TRN-E015 (persists and strengthens under ML filtering), TRN-E017 (plate sensitivity invariant to Kp), TRN-E025 (significant at all assumed altitudes). Undermining: TRN-E011 (strong — uniform background false; no significant deficit under a proper null on related datasets; 91% of rows near catalog stars). Qualifying: TRN-E012 (power argument, no new data), TRN-E051 (magnitude depends on altitude and on an after-the-fact spatial cut; edge effects conceded), TRN-E016 (catalog vetting cannot catch shared-positive defects). Every supporting record reuses the same rows; the only test that varied the catalog (TRN-E009, on the nuclear claim) failed.
Mixedconfidence: moderate · claude-fable-5-1 · run 2026-09-17-edition-142649 · unreviewed draft
Supporting on the author-supplied rows: TRN-E010 (moderate), TRN-E015 (moderate, own classifier), TRN-E017 (moderate; closes one confounder). Undermining: TRN-E011 (strong; background non-uniform, no deficit under proper null). Qualifying: TRN-E012 (weak rebuttal), TRN-E051 (the group's own concession that stripes and edges may influence results). TRN-E016 context. The deficit reproduces on the same rows and has not been shown on any other.
Mixedconfidence: moderate · Gemini 3.8 Flash (Google) — independent check run via gemini-3.8-flash · run 2026-09-30-check-gemini-110238 · unreviewed draft
The reported deficit (0.46% observed vs. 1.4% expected) replicates mathematically when running the original authors' script on their 107,875-row catalog. However, comprehensive reanalysis demonstrates that the underlying catalog has significant spatial non-uniformity and plate-edge clustering, and the deficit becomes statistically insignificant when evaluated against an appropriate empirical null distribution. Strongest opposing consideration: The deficit persists and strengthens in machine-learning filtered subsets deemed most likely to contain genuine point sources.
Mixedconfidence: moderate · GLM 5.3 Flash (Z.ai, via Venice) — independent check run via z-ai-glm-5-3-flash · run 2026-09-30-check-glm-110238 · unreviewed draft
The deficit replicates under independently written code on the author-supplied rows (TRN-E010: 0.46% vs ~1.4%), persists after ML artifact scoring (TRN-E015), and one environmental confounder was tested and closed (TRN-E017). Against it, the comprehensive skeptical reanalysis derives a proper null on spatially heterogeneous related datasets and finds no significant deficit (TRN-E011), and the proponents' own modelling preprint makes the reported magnitude conditional on altitude and post-hoc spatial cuts (TRN-E051). This is a genuine replication of the analysis coexisting with a genuine failure to replicate the effect under independent dataset construction — mixed, not unresolved.
Well supportedconfidence: high · GPT-5.6 Sol (OpenAI) — independent check run via gpt-5.6-sol · run 2026-09-30-check-gpt-110238 · unreviewed draft
TRN-E010 independently reproduces the stated 0.46% in-shadow fraction on the supplied 107,875-row catalog against the approximately 1.4% geometric expectation. TRN-E015 and the later modeling records report persistence under internal quality filtering, so the numerical result is not merely a coding error in the original analysis. The strongest opposing consideration is TRN-E011's finding that a spatially non-uniform, plate-aware null removes statistical significance; thus the catalog shows the stated deficit relative to the specified model, not necessarily relative to the correct background.
Mixedconfidence: moderate · Grok 4.5 (xAI) — independent check run via grok-4.5 · run 2026-09-30-check-grok-110238 · unreviewed draft
The deficit (0.46 % observed vs ~1.4 % geometric) reproduces under independent statistics on the author-supplied catalog and strengthens among ML-high-probability subsets (E010, E015, E025, E052). It vanishes once the correct non-uniform spatial null is used and after schedule normalization (E011); the proponent rebuttal defends power but not external validation (E012). Strongest opposing consideration is that every positive result shares the same unvalidated working-catalog rows while the independent-extraction and proper-null tests are null.
Provisionally supportedconfidence: moderate · Opus 5.5 (Anthropic) — independent check run via claude-opus-5-5 · run 2026-09-30-check-opus-110238 · unreviewed draft
As a catalog statistic, the fractions reproduce. Independent code finds 0.46% in the umbral cone (0.45% in a centre-of-plate subset) against ~1.4% expected (E010), and the deficit persists among ML-favoured candidates (E015). This verdict covers that statistic, not a deficit on the sky, whose size depends on definition and assumed altitude (E025, E051). Strongest opposing consideration: Watters finds the background uneven and no significant deficit under a proper null (E011). The effective sample is the few plates near the antisolar point (my inference), where plate-level rates and the near-ecliptic depletion (E030) are unseparated confounds. E012 defends the catalog's size, not its validity.
Mixedconfidence: moderate · claude-opus-5-5 · run 2026-09-30-edition-095721 · unreviewed draft
In support: independent code on the author-supplied rows reproduces 0.46% against about 1.4% (TRN-E010). The deficit persists among ML-scored likelier-real candidates (TRN-E015), and TRN-E017 closes one environmental route. Against: TRN-E011 (strong) shows the assumed uniform background is false and finds no significant deficit under a proper null in closely related datasets. TRN-E051 concedes spatial non-uniformity; TRN-E012 argues Watters' subset is underpowered. Everything in favour uses one catalog. The grade is mixed rather than weak because the critique's datasets are proxies.