BRIGHT EVIDENCE PACK / Demonstrated
OpenAI’s COBOL AI gains put legacy software in focus
New coding results invite a serious look at AI-assisted maintenance. Bright examines the evidence, the limits and what safer modernization would require.
Canonical Bright record · JSON evidence pack · Key-facts embed
Dates and assessment
- Source published
- 2026-10-19
- Bright published
- 2026-10-19
- Substantive update
- None recorded
- Evidence state
- Demonstrated
- Independent verification
- Not established by this source review
- Last source review
- 2026-10-19
The claim in context
The human problem
Organizations depend on established software that must remain reliable while engineers understand, maintain and modernize it.
The prior constraint
Established languages can have comparatively little model training material, while modernization also requires domain expertise and operational assurance.
AI’s actual role
OpenAI evaluates code translation and SQLite-compatible database implementations across programming languages.
The documented result
OpenAI reports 86.6% COBOL migration accuracy for GPT-6.1 Sol at low reasoning effort and 97.9% at higher effort; GPT-6 Astra averaged an 87.4% MultiLangBench test pass rate at maximum effort across 20 languages. The measurements use different denominators.
Why it may matter
Better assistance could help maintainers understand, test or repair important software; dependable outcomes still require human engineering decisions.
Limitations
- Company-reported benchmark results; not independently reproduced by Bright.
- Benchmark scores do not establish the success probability of a live migration, production assurance, jobs eliminated or specialist time saved.
- COBOL migration accuracy and MultiLangBench test pass rates use different denominators.
- Teams must disclose test subsets and evaluate platform, transaction, integration and failure behavior for their own deployments.
Original evidence
- OpenAI research: low-resource programming languages · institution
- GAO legacy systems review · government
- UFS Weather Model documentation · institution
- UFS Weather Model repository · institution
- OpenAI MultiLangBench repository · institution
- How SQLite is tested · institution
- IBM COBOL overview · institution
Attribution
Credit Bright AI Future and link the canonical Bright record.
- Link to the canonical Bright record.
- Keep material limitations with the claim they qualify.
- Link to the original evidence when repeating a substantive claim.
- Do not describe a source check or organization-reported result as independent verification.
Linked source material, quotations, trademarks and media remain subject to their owners’ terms. No reuse right is granted for third-party media.