Forty percent of the world’s banks still run on COBOL, and the people who know how to maintain it are retiring. OpenAI thinks its latest models can help, and the numbers it shared are good. The caveats are better.
OpenAI said its models are now stronger at building and migrating software in older languages, including COBOL, Fortran and Visual Basic. The company says feedback from developers and enterprise customers shaped the work. Banks face migration bills that run from millions to hundreds of millions of dollars, plus years of effort, and the talent pool is shrinking fast.
To measure progress, OpenAI built an evaluation suite called MultiLangBench. It asks a model to write a SQLite-compatible database engine in a given language. GPT-6 Astra averaged an 87.4 percent pass rate across 20 languages. GPT-5.6 Sol managed 26.8 percent. That’s a huge jump, though it’s OpenAI’s own test, so treat it as a directional signal and not an independent verdict. COBOL was still the hardest language, with Astra landing at 72 percent.
The more relevant test is migration. OpenAI had its models convert COBOL programs into C++, Java, Kotlin, Python or Rust while preserving behavior. That means matching details like how the original rounds numbers with extra decimal digits and how it handles missing records. In banking, those quirks are the business logic. GPT-6.1 Sol reached 97.9 percent accuracy with high reasoning effort and 86.6 percent at low effort.
So what do you do with 97.9 percent? In a bank, not much by itself. A single wrong rounding rule across millions of transactions is a real problem. OpenAI says as much, noting that a migration succeeds only if all tests pass, and that production use still needs expert review and testing against business rules and dependencies. The model is a accelerator for engineers, not a replacement for them.
This also fits a wider push. Anthropic has been pitching Claude for legacy modernization, and IBM sells watsonx Code Assistant for Z specifically for COBOL-to-Java work. Amazon and Microsoft offer their own mainframe modernization tooling. OpenAI is stepping into a crowded enterprise niche where the buyers are conservative and the stakes are high.
Does it matter? Yes, with limits. Legacy migration is one of the clearest paid uses for coding models, because the pain is real and the budgets are large. But the winner will be whoever proves reliability on messy production systems, not whoever posts the best benchmark. Watch for customer case studies. Those will tell you far more than a pass rate.
