# When a model changed, its open recipe helped trace why.

Goodfire used Ai2's open OLMo post-training stack to trace a known regression and inspect behavioral shifts.

Canonical: https://brightaifuture.com/discoveries/goodfire-olmo-post-training
Format: discovery
Source publication: 2026-09-09
Bright publication: 2026-09-19
Substantive update: None recorded
Evidence and review: Demonstrated; confidence: medium; approved; ai-assisted. AI-assisted editorial comparison with the cited primary sources, explicit evidence limits, and held alternatives. Publication authorized by the site owner on 2026-09-19; no human source review or independent replication is claimed.

## The human problem

When post-training changes model behavior, developers may see the regression without being able to inspect how it formed.

## The prior constraint

Closed data, code, and checkpoints make causal debugging of model behavior difficult.

## AI’s actual role

Interpretability tools examined internal and behavioral changes across an openly documented post-training process.

## The documented result

The 9 September case study reports tracing a known regression using OLMo's available stack. It does not establish detection of unknown problems in general.

## Why it may matter

Openness can support investigation after a benchmark moves, not only reuse of final weights.

## Limitations

This is a case study from participating organizations.

It begins with a known regression.

The method may not transfer to closed models or every failure mode.

## Unresolved questions

Can it discover unanticipated regressions?

Which artifacts are essential for a reproducible explanation?

How should competing causal interpretations be tested?

## Provenance and history

{
  "dates": {
    "eventDate": "2026-09-09",
    "publicationDate": "2026-09-09",
    "captureDate": "2026-09-19",
    "lastReviewedDate": "2026-09-19"
  },
  "provenance": {
    "origin": "editorial",
    "externalId": "https://allenai.org/blog/goodfire-olmo"
  },
  "revisions": [
    {
      "id": "revision:sept26-goodfire-01",
      "recordedAt": "2026-09-19",
      "summary": "Initial open-debugging follow-up draft.",
      "sourceIds": [
        "source-goodfire-olmo"
      ]
    }
  ],
  "corrections": []
}

## Original sources

- [How Goodfire used Ai2’s open post-training stack to trace unwanted model behavior](https://allenai.org/blog/goodfire-olmo)

## Continue exploring

- [Open intelligence](https://brightaifuture.com/worlds/open)
- [Frontier](https://brightaifuture.com/worlds/frontier)
- [Where can human judgment go with a new instrument?](https://brightaifuture.com/threads/discovery)
- [What changes when powerful models become open-weight?](https://brightaifuture.com/threads/open)
- [Someone builds on it](https://brightaifuture.com/open-intelligence)