Earlier this year, Mantle faced a common challenge.
The next-generation equity management platform had finished the prototype for a new product, and needed to get it ready for production.
“We developed an approach that reduced the scope of work by two-thirds and saved months of developer time.”
Dwayne Forde, CTO of Mantle
They had built the prototype in a specific coding language that was perfect for speedy interaction in response to feedback from customers.
The problem was, it was different from the code used in their production tech stack.
To ship the product, Mantle would need to convert the codebase from one language to another, an onerous task that is regularly faced by software teams and enterprises.
“The effort is justified, but the process is painful,” said Dwayne Forde, Mantle co-founder and CTO. “Instead of moving a customer-facing roadmap forward, you are now going to spend a significant portion of valuable engineering time recreating existing functionality.”
Forde wondered if AI could help.
A trusted industry leader with more than 20 years of engineering experience in roles with companies including VMware and Xtreme Labs, he chronicled the process recently in a blog post on Mantle called “Working with AI: Code Conversion.”
He hopes the case study will serve as a useful resource to other tech teams, helping them save time and effort.
It is the second in a series of instructional guides Forde has written for technical teams, [Read about “Part One: How to Tame Your LLM”] as part of an effort to advance the collective interests of the sector by showing how AI can accelerate and enhance their work.
“Our goal wasn’t to achieve 100% perfectly crafted code,” Forde noted. “The goal was to get 80% of the boilerplate and repeated patterns out of the way so that engineers could focus on high-value validation and verification and we could ship the product.”
Capacity building
Not too long ago, it wasn’t possible for LLMs (Large Language Models) to rewrite code.
Each LLM has a token limit, which establishes how many words it can absorb and apply.
With lower token limits, the models are unable to absorb the amount of information required to perform complex tasks like code conversions.
But with rapid advancements in LLM software came higher token limits, and Forde realized his team had exciting new options in front of them. Higher limits meant that models could increase their reasoning, perform more complex math and inference, and input and output context in dramatically larger sizes.
One million tokens means, according to Medium, that a model can do the equivalent of reading 20 novels or 1000 legal case briefs.
Forde and his team understood that this dramatically larger token limit would allow them to feed entire coding languages into an LLM, essentially teaching it to be bilingual.
Because converting code is extremely labour-intensive, Mantle knew that having an LLM convert even small amounts of code from one language to another would be hugely beneficial to the delivery time of the engineering project.
“We developed an approach that reduced the scope of work by two-thirds and saved months of developer time,” Forde wrote in his post.
Show don’t tell
Converting the Mantle prototype project into a new code language would have normally taken months of manual labour.
Instead, Forde said his engineers focused their time experimenting with how to best prompt an LLM to do much of the work for them.
It wasn’t just as simple as feeding the code languages into the LLM and asking it to translate.
Under Forde’s watch, the Mantle team went through a process of innovation and discovery to figure out the best instructions, context and guidance to provide the LLM in its work.
They fed the model code snippets from their prototype source language, as well as existing production code patterns, descriptions of their target architecture, and provided the LLM with context about specific libraries and utilities used in Mantle’s own tech stack.
“We have certain libraries that we prefer, so adding a section of context was very helpful to make sure the LLM output code was compatible with what we use,” said Forde.
The team even fed the LLM screenshots to demonstrate how they wanted the information to be presented, something that would not be obvious to AI from the code language alone.
“Screenshots of the existing application give the LLM a visual layout of the application,” said Forde. “The context and direction you provide don’t have to be all verbal. You can use visual reference points as well to get the output you’re after.”
Use your time wisely
In his blog post, Forde breaks down the step-by-step process Mantle used to convert their code. The process is innovative, iterative and – at times – playful.
At one point, the Mantle team instructed the LLM to “act like a software engineer who could only answer in source code.”
The Mantle team asked the LLM to convert only small sections of code at a time, checked its work, corrected any misinterpretations, and then moved on.
The step-by-step experimentation allowed the Mantle team to refine and improve its work over time, and create an effective process that can now be replicated in future projects.
“Once the file was generated, our team either reviewed and adjusted the output manually or adjusted the prompt to recreate the output,” said Forde. “It’s important to catch any patterns early on before they compound and multiply.”
For Mantle, the code conversion exercise was a game changer, streamlining and optimizing a complex software development process, while still putting the time and effort of their team members to good use.
Forde hopes that the technical guide he published on the process saves other tech companies time and effort.
“As token windows continue to grow and models become more adept at understanding and generating code, code conversion will continue to improve in speed and quality,” he said. “This will undoubtedly lead to more cost-effective software development, which will benefit the whole ecosystem.”
PRESENTED BY
For more detailed information about how to use AI to convert a prototype codebase, read Forde’s blog post here.
Feature image courtesy Unsplash. Photo by Sigmund.