Skip to the content
Aletheia

Deep Memory · claim DEEP-C038 · Observation rung · When resemblance becomes evidence

In the Deep Memory image pilots, matching the kinds of objects in a lineup changed the lineup result. True if the pilot data show a different ranking under kind-matched controls.

In the page's image tests, comparing like with like — pillars with pillars — changed which objects ranked as most similar.

AI-extractedorigin: report 2026-09-09-inbox-deep-memory-200334

credibility — is the claim itself true?

Weakly supported

DEEP-E022 gives the page's statement that matching object kinds changed the lineup result; a self-report with pilot data not in hand, and the page notes incomplete runs and cropping constrain it.

diagnosticity — how much does it decide the thesis?

moderate

If robust, part of the resemblance in any mixed gallery comes from comparing unlike kinds.

Evidence

Supports (1)

Supports· weakDEEP-E022

Deep Memory reports what the image pilots exposed — part 2

what the source states
In the image pilots, "Matching the kinds of objects changed the lineup result."
what we infer
Editor: self-report of the site's pilots; these are negative or cautionary findings about method rather than about the objects.
limitations
  • Self-report; pilot data not retrieved.
  • The page notes incomplete runs and cropping constrain the conclusions.
  • Second reader (claude-sonnet-5, 2026-09-09) disputes the stated direction: The quoted line exactly matches the pilot's finding about kind-matched lineups and fairly supports DEEP-C038, with the locator correctly pointing to the pillar & moai section. However, the record also lists DEEP-C037 (about strong inter-judge disagreement), and this particular quote says nothing about judge agreement statistics, so labeling the record as 'supports' toward both named claims is not right for C037; it should not be treated as bearing on that claim in the same way.

strongest unresolved objection

Pilot runs were incomplete and the models have not been shown to match human judgment.

what would change our mind

  • The pilot data, or a human replication with kind-matched controls.

Assessment history

Append-only AI overlay records; the canon claim file is never mutated.

Provisionally supportedconfidence: moderate · Gemini 3.8 Flash (Google) — independent check run via gemini-3.8-flash · run 2026-09-09-check-gemini-210940 · unreviewed draft

Evidence record DEEP-E022 directly attests to the finding of the image pilots, quoting that matching the kinds of objects in the lineup changed the lineup result. This demonstrates that morphological resemblance rankings are sensitive to controls for object category. The strongest opposing consideration is that the image pilots remain unaudited self-reports subject to small sample sizes and unresolved cropping artifacts.

Weakly supportedconfidence: low · GLM 5.3 Flash (Z.ai, via Venice) — independent check run via z-ai-glm-5-3-flash · run 2026-09-09-check-glm-210940 · unreviewed draft

DEEP-E022 quotes the pilot's own finding that matching the kinds of objects changed the lineup result, and the second reader confirmed this quote fairly supports the claim. The support remains a weak self-report with pilot data not retrieved, incomplete runs and cropping noted as constraints. The strongest opposing consideration is that a changed result under kind-matching could reflect an artefact of the matching procedure rather than exposure of bias in the original lineups.

Weakly supportedconfidence: moderate · GPT-5.6 Sol (OpenAI) — independent check run via gpt-5.6-sol · run 2026-09-09-check-gpt-210940 · unreviewed draft

DEEP-E022 directly states that matching object kinds changed the lineup result, so it weakly supports the claim as a report of the pilot's outcome. The evidence is a founder self-report without data, effect size, direction, or analysis. The strongest opposing consideration is that incomplete runs, cropping, or a small design change could produce an unstable result that would not replicate.

Weakly supportedconfidence: low · Grok 4.5 (xAI) — independent check run via grok-4.5 · run 2026-09-09-check-grok-210940 · unreviewed draft

E022 directly quotes the pilot result that matching object kinds changed the lineup ranking. The finding is a self-reported caution about method, not about the artefacts. Strongest opposing consideration is incomplete runs and cropping noted by the page itself, which limit generalisation.

Weakly supportedconfidence: low · Opus 5 (Anthropic) — independent check run via claude-opus-5 · run 2026-09-09-check-opus-210940 · unreviewed draft

DEEP-E022 quotes the page directly — "Matching the kinds of objects changed the lineup result" — and the second reader accepts the match for this claim while disputing its attachment to DEEP-C037. It remains a self-report of an unretrieved pilot whose limitations note incomplete runs and cropping effects. The finding is methodologically important because it implies the lineup measures object category as much as iconographic resemblance. The strongest opposing consideration is that "changed the result" is unquantified: a small rank shift and a total reversal are both consistent with the wording, and only the underlying data would distinguish them.

Weakly supportedconfidence: low · claude-fable-5-1 · run 2026-09-09-edition-210937 · unreviewed draft

One weak self-report record (E022) states that matching the kinds of objects changed the lineup result; pilot data are not retrieved and the page notes incomplete runs and cropping constrain the conclusions.

Provisionally supportedconfidence: moderate · Gemini 3.8 Flash (Google) — independent check run via gemini-3.8-flash · run 2026-09-10-check-gemini-121549 · unreviewed draft

Supported by DEEP-E022, which documents the pilot finding that matching object kinds in visual lineups altered the similarity rankings. Support is provisional because it relies on the author's self-report from an unverified source, with acknowledged limitations concerning incomplete runs and image cropping constraints. The strongest opposing consideration is that small pilot sample sizes or uncalibrated vision model weights might cause lineup instability unrelated to the matching of object kinds.

Weakly supportedconfidence: low · GLM 5.3 Flash (Z.ai, via Venice) — independent check run via z-ai-glm-5-3-flash · run 2026-09-10-check-glm-121549 · unreviewed draft

DEEP-E022 supports the claim as a self-reported pilot finding, and the second reader confirms the quote matches the kind-matching result. The strongest opposing consideration is the record's own limitations: pilot data unretrieved, incomplete runs and cropping flagged, and the report gives neither the direction nor the magnitude of the change, so the finding carries methodological weight but little quantitative one.

Weakly supportedconfidence: low · GPT-5.6 Sol (OpenAI) — independent check run via gpt-5.6-sol · run 2026-09-10-check-gpt-121549 · unreviewed draft

DEEP-E022 directly reports that matching object kinds changed the lineup result, so it weakly supports the claim as a description of the pilot. The underlying rankings, effect magnitude, model prompts, crops, and incomplete runs were not retrieved, preventing reproduction or assessment of practical importance. The strongest opposing consideration is that a direction-changing control result can reveal a serious confound even in a small pilot.

Weakly supportedconfidence: low · Grok 4.5 (xAI) — independent check run via grok-4.5 · run 2026-09-10-check-grok-121549 · unreviewed draft

E022 quotes the pilot result that kind-matched controls changed lineup rankings, but the underlying scores and incomplete-run caveats remain unretrieved. The methodological warning is noted at weak strength. The strongest opposing consideration is that the shift could be noise from small samples rather than a robust bias.

Weakly supportedconfidence: low · Opus 5 (Anthropic) — independent check run via claude-opus-5 · run 2026-09-10-check-opus-121549 · unreviewed draft

DEEP-E022 states directly that matching the kinds of objects changed the lineup result, which is on point, but it is a weak self-report with pilot data unretrieved and with the page itself noting incomplete runs and cropping. The finding is methodologically important because it implies earlier lineup outcomes partly tracked object category rather than motif resemblance. Opposing consideration: "changed the result" is unquantified — a direction and magnitude are needed before it can bear weight either for or against the resemblance case.

Weakly supportedconfidence: moderate · claude-fable-5-1 · run 2026-09-10-edition-121547 · unreviewed draft

DEEP-E022 records the page's report that matching the kinds of objects in a lineup changed the result; self-report, with incomplete runs and cropping constraints noted by the page.

Weakly supportedconfidence: low · claude-fable-5-1 · run 2026-09-19-edition-130205 · unreviewed draft

DEEP-E022 records the page's statement that matching the kinds of objects changed the lineup result; a weak self-report with pilot data not retrieved and incomplete runs acknowledged.

Weakly supportedconfidence: low · Gemini 3.8 Flash (Google) — independent check run via gemini-3.8-flash · run 2026-09-28-check-gemini-140630 · unreviewed draft

Record DEEP-E022 documents the author's explicit finding in the site's image pilots that "matching the kinds of objects changed the lineup result." However, this evidence is an unverified self-report from an informal web publication, lacking access to the underlying image corpora, feature extraction models, or comparative ranking matrices. While it constitutes weak support for the claim as a report of the pilot's stated outcome, it cannot be independently verified from the ledger. The strongest opposing consideration is that the image pilot data have not been published or independently replicated, leaving open the possibility that the shift in rankings was an artifact of small sample size or arbitrary image cropping.

Provisionally supportedconfidence: moderate · Gemini 3.8 Flash (Google) — independent check run via gemini-3.8-flash · run 2026-09-28-check-gemini-142201 · unreviewed draft

Record DEEP-E022 directly supports this claim, citing the founder's image pilot findings where controlling for object type and medium significantly altered the AI lineup rankings, proving that raw visual models were matching material class rather than underlying symbolic iconography. The evidence remains provisional because it represents an internal, unaudited pilot whose full dataset and statistical controls have not been formally published. The strongest opposing consideration is that the image pilot relied on limited image sets and uncalibrated computer vision architectures that may not reflect robust multi-modal feature representations.

Weakly supportedconfidence: low · Gemini 3.8 Flash (Google) — independent check run via gemini-3.8-flash · run 2026-09-28-check-gemini-143733 · unreviewed draft

Evidence record DEEP-E022 documents the author's self-report from SRC-JHONG-2026 stating that "Matching the kinds of objects changed the lineup result" in the image pilots. While this provides direct testimony from the researcher that the control ranking altered the outcome, the finding is weakly supported because the raw pilot data, image sets, and statistical ranking shifts have not been retrieved or audited. The strongest opposing consideration is that this self-critique represents an explicit admission of a confounding variable that directly satisfies the stated truth condition of the claim.

Weakly supportedconfidence: low · GLM 5.3 Flash (Z.ai, via Venice) — independent check run via z-ai-glm-5-3-flash · run 2026-09-28-check-glm-140630 · unreviewed draft

DEEP-E022 weakly supports the claim, and its second reader affirmed the quote fairly matches the kind-matching finding; but it is a self-report with pilot data not retrieved and acknowledged incomplete runs and cropping constraints. The strongest opposing consideration is that "changed the result" is unquantified — no record states how much it changed or in which direction, so the claim's method- fragility import is unmeasured.

Weakly supportedconfidence: low · GLM 5.3 Flash (Z.ai, via Venice) — independent check run via z-ai-glm-5-3-flash · run 2026-09-28-check-glm-142201 · unreviewed draft

DEEP-E022 records the site's statement that "Matching the kinds of objects changed the lineup result", which matches the claim, but it is a weak self-report with pilot data not retrieved and acknowledged incomplete runs and cropping constraints. The finding is methodologically important — it suggests resemblance measures are sensitive to control design — but nothing independent reproduces it.

Weakly supportedconfidence: low · GLM 5.3 Flash (Z.ai, via Venice) — independent check run via z-ai-glm-5-3-flash · run 2026-09-28-check-glm-143733 · unreviewed draft

DEEP-E022 weakly supports the claim as a direct self-report of the pilot finding that kind-matched lineups changed the result, with pilot data unretrieved and the page itself noting incomplete runs and cropping constraints. The strongest opposing consideration is that the second reader had to correct this same record's scope, indicating the self-report's mapping to claims is fragile, and the magnitude of the change is unstated.

Provisionally supportedconfidence: low · GPT-5.6 Sol (OpenAI) — independent check run via gpt-5.6-sol · run 2026-09-28-check-gpt-140630 · unreviewed draft

DEEP-E022 directly reports that matching the kinds of objects changed the lineup result. It is only a founder self-report, with the pilot data, rankings, and control construction unavailable; incomplete runs and cropping are also acknowledged. The strongest opposing consideration is that no effect size or replication shows whether the change was substantial or robust.

Weakly supportedconfidence: low · GPT-5.6 Sol (OpenAI) — independent check run via gpt-5.6-sol · run 2026-09-28-check-gpt-142201 · unreviewed draft

DEEP-E022 directly reports that matching object kinds changed the lineup result. It is nevertheless a founder self-report without the pilot data, rankings, effect size, or completed-run accounting, and the record notes cropping and incomplete runs. The strongest opposing consideration is the exact correspondence between the quoted finding and the claim, but that establishes only what was reported rather than reproducing the result.

Weakly supportedconfidence: moderate · GPT-5.6 Sol (OpenAI) — independent check run via gpt-5.6-sol · run 2026-09-28-check-gpt-143733 · unreviewed draft

DEEP-E022 directly reports that matching object kinds changed the lineup result. It is nevertheless a self-report from a pilot whose data were not retrieved and whose runs and cropping were incomplete. The strongest opposing consideration is that neither the magnitude nor direction of the change is supplied, preventing verification of the result or its importance.

Weakly supportedconfidence: low · Grok 4.5 (xAI) — independent check run via grok-4.5 · run 2026-09-28-check-grok-140630 · unreviewed draft

DEEP-E022 restates the page's claim that kind-matched controls changed lineup rankings; the underlying pilot data remain unretrieved and the page itself notes incomplete runs. Strongest opposing consideration is that the observed change could be noise from the same unstable AI judges documented under DEEP-C037.

Weakly supportedconfidence: low · Grok 4.5 (xAI) — independent check run via grok-4.5 · run 2026-09-28-check-grok-142201 · unreviewed draft

DEEP-E022 records the founder's statement that matching the kinds of objects in a lineup changed the lineup result. Pilot data, cropping notes and incomplete runs are not retrieved. Strongest opposing consideration is that the finding is a negative methodological result about the pilots themselves, not yet a calibrated measure of object resemblance.

Weakly supportedconfidence: low · Grok 4.5 (xAI) — independent check run via grok-4.5 · run 2026-09-28-check-grok-143733 · unreviewed draft

The site's self-report states that matching the kinds of objects changed the lineup result; pilot data are not retrieved and incomplete runs are acknowledged. The directional claim is therefore only weakly anchored. Strongest opposing consideration is that any lineup metric sensitive to category matching will shift under that control without illuminating historical resemblance.

Weakly supportedconfidence: moderate · Gemini 3.8 Flash (Google) — independent check run via gemini-3.8-flash · run 2026-09-30-check-gemini-103112 · unreviewed draft

Record DEEP-E022 provides direct support by quoting the founder's pilot report, which states that matching the kinds of objects compared in an image lineup fundamentally changed the lineup results. This is an important methodological observation demonstrating that apparent cross-cultural visual matches were artifacts of comparing heterogeneous object classes against one another. However, the support is rated weak because it is an unaudited self-report from an unverified web page without access to the image corpora, the classification taxonomy, or quantitative ranking matrices. The strongest opposing consideration is that without seeing the image sets, one cannot rule out whether the kind-matching control simply introduced arbitrary visual noise that masked genuine high-level stylistic resemblances.

Weakly supportedconfidence: low · GLM 5.3 Flash (Z.ai, via Venice) — independent check run via z-ai-glm-5-3-flash · run 2026-09-30-check-glm-103112 · unreviewed draft

DEEP-E022 quotes the pilot's own finding that kind-matching changed the lineup result, which matches the claim, but it is a weak self-report with pilot data unretrieved and acknowledged constraints from incomplete runs and cropping. The strongest opposing consideration is that the second reader's note shows this record cannot also carry DEEP-C037, so the two method findings must stand on separate evidence — and C037's has not arrived.

Weakly supportedconfidence: moderate · GPT-5.6 Sol (OpenAI) — independent check run via gpt-5.6-sol · run 2026-09-30-check-gpt-103112 · unreviewed draft

DEEP-E022 directly reports that matching object kinds changed the image- lineup result. It is a founder self-report from a pilot whose data were not retrieved and whose runs and cropping were incomplete. The strongest opposing consideration is that neither the direction nor the size of the change can be independently checked.

Weakly supportedconfidence: low · Grok 4.5 (xAI) — independent check run via grok-4.5 · run 2026-09-30-check-grok-103112 · unreviewed draft

E022 quotes the founder's statement that matching the kinds of objects changed the lineup result in the image pilots. The claim is therefore supported as a self-report of a methodological finding. Strongest opposing consideration is that pilot data, cropping decisions and incomplete runs are unrecovered, leaving the magnitude and robustness of the kind-matching effect unknown.

Weakly supportedconfidence: moderate · Opus 5.5 (Anthropic) — independent check run via claude-opus-5-5 · run 2026-09-30-check-opus-053723 · unreviewed draft

One record (DEEP-E022) quotes the page: "Matching the kinds of objects changed the lineup result." The pilot data are not in hand, the page notes incomplete runs and cropping, and the record does not say which way the result moved, so the claim is supported only as self-report. The strongest opposing consideration is that 'changed' is unquantified: a shift within run-to-run noise would satisfy the wording without showing a real control effect. If accurate, the finding cuts against using unmatched lineups as evidence of specific resemblance, since the judges were then responding partly to object category.

Weakly supportedconfidence: low · Opus 5.5 (Anthropic) — independent check run via claude-opus-5-5 · run 2026-09-30-check-opus-103112 · unreviewed draft

E022 states the claim directly, but it is the founder's unaudited self-report. The pilot data are not in hand, and the page itself says incomplete runs and cropping constrain the conclusions. The claim is also underspecified: "changed the lineup result" gives neither the direction nor the size of the change. The truth condition, a different ranking under kind-matched controls, therefore cannot be checked even in outline. The strongest opposing consideration is that this is a cautionary finding against the author's interest. That makes misreporting less likely and could justify a somewhat stronger grade.

Weakly supportedconfidence: low · claude-opus-5-5 · run 2026-09-30-edition-091201 · unreviewed draft

DEEP-E022 gives the page's statement that matching object kinds changed the lineup result; a self-report with pilot data not in hand, and the page notes incomplete runs and cropping constrain it.