YouTube will help its creators reach an international audience as the platform plans to introduce a new AI-powered dubbing tool for translating videos into other languages.
Announced at VidCon 2023, the goal of this latest effort is to provide a quick and easy way for creators to translate “for free” their content into languages they don’t speak. This helps smaller channels because they don’t have the resources to hire human translators. To make all this possible, Amjad Hanif, vice president of Creator Products at YouTube, revealed that the tool will use Made by Google Aloud In addition, the platform will bring the team behind the AI from Area 120, a division of the parent company that often works on experimental technology.
Easy translation
The way the translation system works, according to Aloud’s official website, is for AI to first transcribe a video into a script. You then edit the transcription to remove any errors, make clarifications, or highlight text “where time is critical.” From there, you feed the edited script back to Aloud where it will automatically translate your video into the language of your choice. Once done, you can publish the newly dubbed content by uploading any new audio tracks to their original video.
A Google representative told us that “creators don’t have to [actually] understand any of the languages they’re dubbing in.” Power will handle all the heavy lifting surrounding complex tasks such as “interpretation, timing, and speech synthesis.” Again, all you need to do is double check the transcription.
Future changes
It is not known when the Aloud update will be rolled out. However, YouTube is already working on expanding AI beyond what is currently possible. Currently, Aloud can only translate English content into Spanish or Portuguese. But there are plans to expand to other languages from Hindi to Indonesian plus support for different dialects.
Later down the line, the platform will introduce various features such as “voice preservation, better emotion transfer, and even lip change” to improve pronunciation. In addition, YouTube will establish some safeguards to ensure that only creators can “dub their own content”.
The same Google representative from earlier also told us that the platform is testing Aloud AI with “hundreds [YouTube] creators” with plans to add more over time. As of June 2023, more than 10,000 videos have been dubbed in more than 70 languages.
You can join the early access program by filling out the official Google Docs form. If you want to know what an Aloud dub sounds like, go for it watch the channel trailer for the Amoeba Sisters channel on YouTube. Click the gear icon, go to Audio Track, then select Spanish. The robotic voice you hear is what the AI will do.
