# OpenAI’s COBOL AI gains put legacy software in focus

Agent contract: 1.2.0

New coding results invite a serious look at AI-assisted maintenance. Bright examines the evidence, the limits and what safer modernization would require.

Canonical: https://brightaifuture.com/discoveries/openai-cobol-ai-legacy-software-multilangbench
Format: discovery
Source publication: 2026-10-19
Bright publication: 2026-10-19
Substantive update: None recorded
Evidence and review: Demonstrated; confidence: unassessed; approved; ai-assisted. Owner authorized separate publication of complete reviewed article. Company-reported benchmark results; not independently reproduced by Bright. Source review is recorded in the supplied approved article. No benchmark was run. Requested SEO title: OpenAI COBOL AI Results: What Legacy Software Teams Should Know. Requested meta description: OpenAI’s COBOL and MultiLangBench results raise hopes for legacy software. Bright explains the evidence, limits and practical steps for engineering teams.

## The story

Some of the most useful advances in AI may arrive far from a new chatbot interface: inside the software that an organization needs to keep running tomorrow morning.

OpenAI’s October 19 research update gives that possibility a concrete test case. Its company-run evaluations report stronger coding results in languages with comparatively little training material, including COBOL and Fortran. The findings deserve attention, but they leave the operational question open: can an engineering team turn better generated code into a dependable improvement to an existing service?

Bright’s view is that this is a promising direction precisely because maintenance is consequential, unglamorous work. Progress here should ultimately be measured in reliable services, understandable systems and changes that people can safely maintain.

## What OpenAI’s COBOL and MultiLangBench results show

OpenAI reports two different measurements:

• COBOL migration: GPT-6.1 Sol achieved 86.6% accuracy at low reasoning effort, versus GPT-5.6 Sol’s 5.4% at the same setting, using 39% less API cost. Higher reasoning effort reached 97.9%. Success required passing every hidden test with byte-for-byte output agreement when translating to C++, Java, Kotlin, Python or Rust.

• MultiLangBench: At maximum reasoning effort, GPT-6 Astra averaged an 87.4% test pass rate across 20 languages, against GPT-5.6 Sol’s 26.8%. The task is building a SQLite-compatible database engine. [Source: OpenAI’s research announcement.](https://openai.com/index/low-resource-programming-languages/)

Those percentages have different denominators. One counts migrations meeting a complete test requirement; the other measures tests passed by database implementations. Neither should be presented as the probability that an organization’s migration will succeed.

## Why older programming languages still matter

There is a practical reason to care about this territory beyond a leaderboard. In a July 2025 review, the US Government Accountability Office identified 11 particularly critical federal legacy systems needing modernization. Eight used outdated languages; seven had known cybersecurity vulnerabilities. The selected systems supported missions including tax processing, health care and national security. GAO also identified shrinking availability of skills for the COBOL and assembly languages used by two Treasury systems. These are findings about the reviewed systems, not a census of all older software. [Source: GAO’s legacy systems review.](https://www.gao.gov/products/gao-25-107795)

Fortran offers another useful corrective to the idea that an old language means abandoned software. The Unified Forecast System’s weather-model documentation includes Fortran configuration and variables, while its public repository includes atmospheric, ocean, land and wave components alongside regression and unit-testing infrastructure. This is an actively maintained scientific software ecosystem. [Sources: UFS model documentation](https://ufs-weather-model.readthedocs.io/en/develop/InputsOutputs.html) and [UFS Weather Model repository.](https://github.com/ufs-community/ufs-weather-model)

For Bright, the distinction matters: improving support for an established language can be valuable even when nobody intends to replace it. Helping a team understand, test or repair a component may serve users better than starting a wholesale rewrite.

## What the public benchmark lets developers inspect

The released [MultiLangBench repository](https://github.com/openai/MultiLangBench) targets SQLite 3.40.1. Its tests supply a starting database and query, then use the reference SQLite implementation to check the resulting database. A wrong answer, crash or 60-second timeout fails the test.

The README says roughly 24,000 of SQLite’s 51,000 TCL tests were converted. It recommends evaluating a specified 272-test subset, with larger runs disclosed. Its setup supplies documentation, prohibits internet access and requires removing existing SQLite implementations from the model’s environment. Solutions must use the requested language. The project carries an Apache-2.0 license.

That release makes the task inspectable. It does not establish independent reproduction of the reported scores. Bright reviewed the announcement and repository documentation, but did not run the benchmark.

The subset also deserves attention when comparing future results: teams should report which tests they ran rather than treating every score with the same benchmark name as interchangeable. A useful reproduction should preserve the settings, environment and scoring rules, and make failures available for inspection.

## Passing tests is only part of preserving a system

SQLite itself provides a revealing comparison. Its own testing documentation describes separate test harnesses, simulated memory and input/output failures, crash and power-loss tests, malformed databases, fuzzing and checks across configurations. The production project’s assurance work reaches far beyond answering a set of ordinary queries correctly. [Source: How SQLite is tested.](https://www.sqlite.org/testing.html)

Bright’s interpretation is straightforward: a benchmark can demonstrate a valuable capability without certifying the complete product that capability might help build. A service must also behave acceptably when storage fails, inputs are unexpected or several operations happen together. A team needs to establish those properties for its actual deployment.

The same reasoning applies to modernization. IBM’s COBOL overview notes that translating a language alone leaves platform questions unresolved, including data architecture, runtime replacement and transaction integrity. IBM is a commercial participant in this market, but the distinction is useful when interpreting a code-generation result. [Source: IBM’s COBOL overview.](https://www.ibm.com/think/topics/cobol)

Consider an illustrative payroll change. Matching a normal monthly run would be encouraging; a responsible acceptance process would also ask about corrections, missing records, duplicate submissions and recovery after an interrupted batch. The people responsible for that service must decide what evidence is sufficient. Fluent-looking code cannot make that decision for them.

## The human opportunity is better-supported expertise

These evaluations do not measure jobs eliminated, specialist time saved or the outcome of a live institutional migration. Predicting those consequences from a benchmark would skip the most important intermediate work.

Bright sees a more useful near-term question: could a maintainer use an assistant to make a difficult system more legible, while retaining responsibility for the change? A proposed explanation of an unfamiliar module, a draft test or a candidate translation can become material for expert scrutiny. Each is a hypothesis until checked.

That suggests a constructive role for experienced developers: identifying unusual cases, distinguishing accidental behavior from required behavior, and teaching newer colleagues how to verify both. Teams should test whether assistance actually reduces their total workload, including review and repair. Faster generation that creates a larger checking burden would be a weak outcome.

## A practical starting point for engineering teams

Bright would start with a bounded experiment rather than a migration promise:

1. Choose a low-risk component. Define its purpose and dependencies, and use an isolated environment with appropriately approved data and tools.

2. Establish the reference behavior first. Record expected outputs and important failure cases before asking for changes. Have someone other than the generator scrutinize the tests.

3. Compare like for like. Record the model, reasoning setting, attempts, runtime and API cost. Include human review hours and unresolved discrepancies.

4. Make acceptance a human-owned decision. Require relevant domain review, integration checks, security assessment and a usable rollback plan before any production change.

The most persuasive next evidence would be documented projects showing what changed, how it was validated and whether the system remained dependable afterward. Bright’s [coverage of AI support for power and water cybersecurity](https://brightaifuture.com/discoveries/anthropic-cyber-mission-power-water) raises a related question: does an AI-generated finding become a safely deployed improvement?

For legacy software, that is the standard worth keeping. Better coding capability could widen the options available to maintainers. The public benefit will depend on the quality of the engineering decisions that follow.

## The human problem

Organizations depend on established software that must remain reliable while engineers understand, maintain and modernize it.

## The prior constraint

Established languages can have comparatively little model training material, while modernization also requires domain expertise and operational assurance.

## AI’s actual role

OpenAI evaluates code translation and SQLite-compatible database implementations across programming languages.

## The documented result

OpenAI reports 86.6% COBOL migration accuracy for GPT-6.1 Sol at low reasoning effort and 97.9% at higher effort; GPT-6 Astra averaged an 87.4% MultiLangBench test pass rate at maximum effort across 20 languages. The measurements use different denominators.

## Why it may matter

Better assistance could help maintainers understand, test or repair important software; dependable outcomes still require human engineering decisions.

## Limitations

Company-reported benchmark results; not independently reproduced by Bright.

Benchmark scores do not establish the success probability of a live migration, production assurance, jobs eliminated or specialist time saved.

COBOL migration accuracy and MultiLangBench test pass rates use different denominators.

Teams must disclose test subsets and evaluate platform, transaction, integration and failure behavior for their own deployments.

## Unresolved questions

Can teams demonstrate dependable deployments and a reduction in total workload including review and repair?

## Provenance and history

{
  "dates": {
    "captureDate": "2026-10-19",
    "eventDate": "2026-10-19",
    "lastReviewedDate": "2026-10-19",
    "publicationDate": "2026-10-19"
  },
  "provenance": {
    "origin": "editorial",
    "externalId": "https://openai.com/index/low-resource-programming-languages/"
  },
  "revisions": [
    {
      "id": "openai-cobol-ai-legacy-software-multilangbench-initial",
      "recordedAt": "2026-10-19",
      "sourceIds": [
        "source:openai-cobol-ai-legacy-software-multilangbench:openai",
        "source:openai-cobol-ai-legacy-software-multilangbench:gao",
        "source:openai-cobol-ai-legacy-software-multilangbench:ufs-docs",
        "source:openai-cobol-ai-legacy-software-multilangbench:ufs-repo",
        "source:openai-cobol-ai-legacy-software-multilangbench:benchmark",
        "source:openai-cobol-ai-legacy-software-multilangbench:sqlite",
        "source:openai-cobol-ai-legacy-software-multilangbench:ibm"
      ],
      "summary": "Initial owner-authorized reviewed article on OpenAI COBOL and MultiLangBench results, legacy-software context, benchmark limitations and practical engineering evaluation."
    }
  ],
  "corrections": []
}

## Original sources

- [OpenAI research: low-resource programming languages](https://openai.com/index/low-resource-programming-languages/)
- [GAO legacy systems review](https://www.gao.gov/products/gao-25-107795)
- [UFS Weather Model documentation](https://ufs-weather-model.readthedocs.io/en/develop/InputsOutputs.html)
- [UFS Weather Model repository](https://github.com/ufs-community/ufs-weather-model)
- [OpenAI MultiLangBench repository](https://github.com/openai/MultiLangBench)
- [How SQLite is tested](https://www.sqlite.org/testing.html)
- [IBM COBOL overview](https://www.ibm.com/think/topics/cobol)

## Continue exploring

- [Anthropic brings AI and engineers to the teams protecting power and water](https://brightaifuture.com/discoveries/anthropic-cyber-mission-power-water)