How I used AI to translate an app to 100+ languages
Figuring out the most efficient, reliable and cost-effective way to localise an app
Okay, now that you have opened up the article because of the clickbait-y nature of the word “AI”, I hope I can spend some time convincing you initially on why localisation is important. After that, I promise I will go into how I used AI and all the different experiments I ran to get this done.
Why localisation?
Wait, first, what is localisation? Localisation or Internationalisation is the practice of ensuring that the content of an app is available in multiple languages. So, why is this important?
Some of us are privileged and good with English, we perhaps even love English and for all intents and purposes, it’s probably the language in which we communicate the most and even think internally. However, for a lot of us, that’s probably not the case. According to some statistics on the internet, if you just consider India which has some of the highest amount of English speakers in the world because of the huge amount of tech employees, only 15% of the total population can speak or read English. In comparison, around 46% of the Indian population owns a smartphone.
This shows that there is a huge gap between the number of people who own a phone versus the number of people who actually understand and are comfortable with the content they read on the phone. Even amongst the people who understand English, not everyone might be comfortable reading all their content in English.
Technology was created to enable everyone to do great things. All the people around the world who don’t understand English still deserve the same great smartphone experiences that all of us know and love. If everyone can enjoy the world in their own unique way, this planet becomes a happier place. This is why it’s important that we localise the content of our app and reach everyone who wants to use the tools we create.
Not to be a capitalist, but it’s also good for business. Localised apps enable you to target a demographic that might not have been previously accessible to you. You probably have a great app out there, but you are missing out on some lucrative revenue opportunities if you are not localising it for the rest of the world.
Understanding localisation on iOS
If you are not an iOS developer, feel free to skip this section and jump into the next one where I talk about how I translated content using AI!
If you are an iOS developer, you might have previously come across the `.strings` file. The strings file was a way to add all the strings within the app in one place and then access them in your code like a variable. If you started an app from scratch with localisation in mind, this would not really be a bad approach. However, most of us don’t think about it in the early stages of development. By the time you know it, your app is filled with strings in all kinds of obscure places. When you finally decide to implement localisation, you realise that you need to collect all the strings from different files and put them in one place which is a long, lonely and arduous task. I remember a couple of years ago when I was trying to localise an app that was already in production, it took me more than a week just to collect all the strings in one place.
And the task does not end there, for every language you want to localise in, you need to manually copy paste all strings into a new string file and then localise all of the content. This becomes even more cumbersome when you decide to add new text because you need to ensure that the key is added to each language along with its corresponding value. This was very prone to errors because chances were that you would miss out on adding a couple of keys and eventually realise everything was out of sync.
If you are already giving up on implementing localisation after reading till here, fret not, things have really changed now for the better. In 2023, Apple launched String Catalogs, a smart, easy and modern way to manage localisable strings.
To start, all you need is to add a String Catalog file from template. What’s really neat with the String Catalog file is that it shows you all the different languages you support in one place. So you need not maintain different files for different languages. The file also has a very neat representation of the languages supported and the various key-value pairs within Xcode so working through this is a breeze.
So let’s say you add a new string in the future, the key is immediately added for all the languages so that you don’t miss out on a particular language translation. What’s even neater is that there’s a percentage text right next to each language name which will indicate if you have forgotten to translate a key into a particular language
This is great, but wait, do you still have to do the groundwork of manually adding all the bad debt of hardcoded strings across the app? That could be days or weeks of work right?
The good news is that String Catalogs work very nicely with both UIKit and SwiftUI strings and pull them in automatically! Yep! All you need to do is build your app once after creating a String Catalog file and all the content will be pulled in.
This is only for the hardcoded strings you might have used in views. For data stored in variables, you will have to change the type to a localised string.
So something like var welcomeText = "Hello World" should be written as var welcomeText = String(localized: "Hello World") . Adding the string this way ensures that these also get pulled into the String Catalog file.
For most apps that aren’t very text-heavy (and by that I mean apps that don’t hold a lot of text on the app’s data itself) a single String Catalog file is a good start. But if your app is super text-heavy and none of this text is coming from a server, you might want to consider splitting the String Catalog file into multiple files.
If you are planning to manage multiple String Catalog files, you can reference text within variables as part of different files using something like var welcomeText = String(localized: "Hello World", table: "welcomeStrings") The table name should correspond to the name of the String Catalog file.
One final thought before we move to the next section - while the interface for String Catalogs in Xcode is pretty and easy to manage, right click on the String Catalog file and click Open As - Source Code. This will show you how the file really looks. At its barebones, it’s a huge JSON object that contains certain specialised keys and language codes to help the compiler understand the language values for each string.
This is important to understand because once you get an idea of how your content needs to be structured within a string catalog file, you can easily start manipulating the file using code to generate large amounts of text.
Welcome AI
Once a codebase is all set for localisation comes the hard part - actually localising the content. In the past, the best way to do this was to hire translators who understand the context of the strings used and translate it into the meaningful equivalent in a particular language.
Even today, for a large business targeting at a scale of billions of people (like Apple), it’s a good approach to take because regional nuances are best understood only by real people. However, if you are a small business or an indie developer, hiring translators can become super expensive and time consuming.
One approach I have seen people take is asking their own user base to contribute translations. This works well too but it takes time to find the right people who are ready to help.
With the pace of AI today, you can use large language models to generate translations for your content and make your app instantly available across a wide range of languages. The beauty of large language models is that they are good at understanding text which means they are also good at understanding nuance, something that a tool like Google Translate might not do well. With the right prompts, you can get translations that are largely accurate and meaningful. While AI can still get a few things wrong, your users can report those and help you correct it. AI still helps you get the first step done of releasing your app in multiple languages.
There are a lot of great tools already available in the market that help you translate text into multiple languages. There’s Smartcat or Lokalise, reasonably priced tools that get the job done for most people. There are a lot of other similar tools as well to translate content in minutes using AI.
In my particular case, the idea was to translate a huge body of text within the app into multiple languages. This was for an indie app so it was important to keep expenses to a minimum. We also set the ambitious goal of making the language available in 100+ languages. The problem with the tools I listed above was that they were inexpensive only when you wanted to translate a small amount of text into a few languages (say, around 20-40). If your app doesn’t have a large amount of text, I would definitely recommend trying out one of those tools. For the use case I was trying to solve, these tools weren’t an option because translating a large body of text into 100 languages was either not possible (limit in languages) or too expensive (because of the huge amount of text). So instead of relying on built-in tools, I decided to make it work on my own.
Approach 1 - Just use the ChatGPT App, duh!
I believe in Occam’s Razor, the simplest approach is probably the right approach. So before I tried something too dramatic, I thought, why not feed structured data into the ChatGPT app and ask it to translate all the content into the relevant languages. At best I would run into my daily limits which I felt I could manage by doing it across multiple days.
So I formatted the data into a neat JSON structure, wrote a prompt that best described the purpose of the app, its niche and how I want the content to be localised. I then fed the JSON file as input to the app and waited for it to generate an output.
Within a couple of minutes, the app generated an output file. I was elated thinking that it had worked. But when I downloaded the file, I realised ChatGPT would simply give up after translating a single sentence into 2-3 languages. After that, it would just write empty strings for the rest. To combat this, I tried different prompting techniques, broke the task down into simpler components, begged, pleaded and even scolded the model but nothing worked.
Approach 2 - The GPT API, but the hard, expensive way
Next I decided to try the GPT API. I realised that the app probably had some limits around how much content it could handle. But when trying with the API, I realised that feeding the whole JSON file and expecting it to read the keys and translate the string into a 100 languages would be too much of an overhead. So I broke it down into a simpler request with a Python script.
I wrote a script that would read the JSON file (xcstrings file) that Xcode used (see section above) and then break it down into each string key that I wanted to translate. I also maintained a list of languages I wanted to translate the string in as a dictionary within my Python script. For each string key, I would make a GPT API request with the right prompt content giving the model enough context and nuance it needed to work on the right translation.
So did it work?
YES! But……
Firstly, the process was super expensive. I had about 150 strings (each about a couple of sentences long) to be translated into 100 languages and my Python script to the GPT API was synchronous. This meant that the script took about a full day to execute. But I also realised making individual requests to the API also costed a lot. I ended up using 12 USD worth of credits to translate the content. It isn’t bad per se compared to the cost of the other tools but I still had another 800 strings to be translated! I realized I would end up spending a lot more if I used the same approach.
That’s why they say measure twice, cut once I guess?
Approach 3 - The Golden Egg - The GPT Batch API
As I mused over how I can best optimise my code, one thing was clear to me. I wanted to make the process async so that I wouldn’t have to spend a whole day waiting for the code to execute and so that I wouldn’t have to pass strings serially to the GPT API. That’s when I realised that GPT also had a Batch API
The very first line caught my eye because they specifically mentioned 50% lower costs. So I set about optimising my Python script to feed data into the Batch API instead of individual requests to the normal API. I also realised that I could play around with how many languages and how many items I wanted to translate as part of a single API request. I tried a few sample API requests to find the sweet spot of what worked best and then constructed the script to translate all of my content. I had to write two scripts - one for sending data to the Batch API and because it was asynchronous, another script that could then fetch data once the processing was done and then piece it altogether in the format that I wanted.
Within the next 24 hours, in a span of a few batches, I ended up translating all the 800 strings that were remaining in 100 languages. The Batch API worked beautifully and because I had optimised the script with the best language to sentences ratio, my cost had also dramatically come down.
I ended up spending only 8 USD for the rest of my translations. So overall, for just 20 USD, I had translated a huge body of text into 100 different languages. It could have been even cheaper if I had used the right script earlier.
Localise it - for everyone
So, how was the accuracy? Now, I obviously don’t know 100 languages, so it was not something I could manually verify but in the languages that I do know - Hindi and Tamil, I was able to validate that the strings had been translated in the right context. This is also a very niche app, so I know that the context this required is different from the day to day usage of language so I would say that the API did a fantastic job in maintaining cultural context and nuance across its translations.
I am sure it isn’t 100% perfect. Chances are that in a few languages, there would be one or more errors. But it’s still a great first step. Once users start using the app in different languages, they can always let us know if they run into issues of bad translations which can then be rectified with input from real people.
Honestly, with the amazing tools at our disposal today, there’s no reason not to localise our apps. Everyone deserves a great mobile experience and now is the right time to deliver it.
If you liked this essay, also check out my last post about how iOS manages to figure out who’s calling you in the most privacy sensitive way -
