Live AI Speech Translations
Clevercast leverages cutting-edge AI technologies to add accurate speech translations to your global live streams. Our multilingual video player lets viewers select their preferred language and listen to natural-sounding AI voices in real-time.
Unlock the power of live AI speech translations
In addition to multilingual live closed captions, our award-winning AI technology supports adding speech translations to live streams. It uses synthetic voices that sound almost like humans, available in more than 50 different languages. This powerful solution, also known as live audio dubbing, significantly improves the viewing experience of international audiences. You can rely on AI to add 99+% accurate speech translations to your live stream. You can even reach 100% accuracy with minimal effort, by using our real-time correction interface in the cloud. We can also provide real-time correctors for you.
Automatic speech recognition and audio translation
Choose female/male voices and regional accents
Very easy to use
Choose your audio languages, embed our player on your site or platform, and start broadcasting. Clevercast ensures that your viewers can see your live stream and listen to crystal clear audio in their own language.
Using the best-in-class language models
Most accurate solution on the market
Clevercast leverages the latency that comes with the HTTP Live Streaming protocol. By slightly increasing it, we’re able to send more context to the ASR engine, which leads to a more accurate speech to text conversion.
Enhance AI with human intervention
Trusted by global brands and companies
Enhanced AI: improve the accuracy of live AI speech translations
You have two options for using live AI speech translation. First, you can use fully automatic translation. Alternatively, you can enhance translation quality by creating a vocabulary of key terms and performing real-time corrections on the text generated from the AI speech-to-text conversion. You can also hire real-time correctors.
AI without human intervention
In basic terms, live AI speech translation works like this: a language model first converts the broadcasted audio into text, then the text is translated by AI and converted to speech again. At each step of this process, pre- and post-processing operations increase the accuracy and quality of the result. This way, live AI speech translation can be used fully automatically.
Enhanced AI for more accuracy
Enhanced AI lets you improve the quality and accuracy, through human intervention. For a live stream with many languages, this is more cost-effective than using human interpreters. What’s more, you get AI closed captions for free. By default, closed captions are available in the player for every AI speech translation language. You can turn this on and off yourself.
Interface for human correction in real-time
Clevercast provides an intuitive web application that lets you edit the transcript of the broadcasted audio in (near) real-time. This way, errors can be kept out of the source for translations.
The application is designed for both first-time users and experienced editors, using a normal keyboard and mouse. Simple actions include making text corrections and temporarily stop captions from appearing. More experienced users can move text to other lines and use shortcuts to update vocabularies.
Due to the high quality of our language models, only a limited number of corrections will be needed in most cases.
This tutorial shows how to use AI vocabularies. Alternatively, you can use Clevercast as a managed service.
Using AI Vocabularies
Clevercast uses smart AI vocabularies to increase accuracy at every step of the process. They guarantee a correct interpretation of specific terms (e.g. names, acronyms, industry jargon, technical phrases) and may contain custom translations in different languages.
AI Vocabularies can be created and updated before and during the live stream. Real-time correctors can use them to relieve their workload by adding frequent terms, so they have to correct them only once.
AI Vocabularies also let you filter profanity and disfluency words.
Frequently Asked Questions
What is the accuracy of automatic speech-to-text conversion?
The accuracy of speech-to-text conversion has improved drastically, thanks to the use of the best AI and ASR technology on the market.
The result of the speech-to-text conversion, which is translated and then converted to speech, is 99+% accurate for commonly used languages like English, Spanish, French, German, Italian, Portuguese, Dutch, Japanese and others. For less common languages, the accuracy will be somewhat lower.
Factors such as speaking speed, articulation and dialect of the speaker or word usage like jargon and acronyms only reduce accuracy in extreme cases (and only to a very limited extent).
Even though the accuracy is very high, there is always room for improvement. You can do this by using a human operator to make real-time corrections to the source text. The operator doesn’t have to be a professional or someone with experience in the matter.
We expect the accuracy of speech-to-text to continue to improve in the near future. The best-of-class ASR technology used by Clevercast is constantly evolving.
How many AI languages are possible?
Unlimited. In practice, it depends on your plan.
What is the quality of the live AI speech translations?
The quality of the voices is excellent: they are clear and sound natural, almost human. If you have specific requirements that differ from our available voices, please contact us.
Sentence segmentation, fluency and intonation are generally very good. For specific types of content, such as conversations, deploying a real-time editor can further enhance this.
Is it possible to combine audio translations with closed captions in the same live stream?
Yes.
Do live streams with AI speech have a delay?
When using AI-generated languages, you can expect the live stream to have a delay of approximately 120 seconds. We are working on a low-latency version of our current solution.
No matter what the delay is, translations are always in sync with the video of the live stream.
Are the AI translations recorded? Can they be downloaded afterwards?
Yes, all AI translations are recorded in the cloud. You can download them afterwards as part of a single MP4 and as separate AAC files. Or you can publish a Video on-Demand with audio translations, hosted by Clevercast.
Can I use AI to generate speech translations for Video on-Demand or Simulive streaming?
Yes, after you have uploaded or recorded a video, and it has been processed, you can use AI to generate speech translations (audio dubbing) in any number of languages.
You can use Clevercast player to embed the video with speech translations on a site or platform of your choice, either as VoD or as a simulive stream. You can also export a multilingual video into separate single-language MP4s (e.g., for use in social media).
What are the costs? How can I order?
AI hours cost €120/hour, which include streaming in up to 8 languages (= the floor audio plus 7 AI speech translations, as well as 8 AI caption languages). For more info, see our pricing page.
What are 'AI hours'? How are they calculated?
AI hours are used when closed captions or audio translations are generated by speech-to-text conversion, text-to-text translation, or text-to-speech conversion. Usage depends on every set of 8 AI languages. For example, if you broadcast during 1 hour to a single streaming server and have 3 AI languages that are automatically generated, you will use 1 AI hour. If you stream with 10 AI languages, you will consume 2 AI hours.
Please note that this is based on the number of hours you broadcast to Clevercast. So AI hours will also count while your event status is ‘preview’ or ‘paused’.
Clevercast as a Managed Service or SaaS Solution
Clevercast can be used as a SaaS platform. For those who prefer it, we also offer it as a managed service. We partner with leading language service providers to source professional real-time correctors. We can provide them for most languages and subjects, if requested in a timely manner.
Self-Service Solution
Clevercast is a SaaS platform, allowing you to to use our AI solutions independently. You can also hire real-time correctors yourself. We can offer premium support for a guaranteed response time and service level.
Managed Service
We can source real-time correctors, help you manage the event and provide assistance during the live stream. This way, we ensure an optimal viewing experience with closed captions of the best possible quality.
Why choose Clevercast?
Extensive feature set
Clevercast has all necessary features for live and on-demand video streaming, management, distribution, monetization and analytics. Whatever your project needs are, we’ve got you covered.
Our customizable HTML5 player can be easily embedded into any device and platform. Just copy the embed code from Clevercast.
Combine with Remote Simultaneous Interpretation and Closed Captions
AI speech translations can be combined with Translate@Home, our solution for RSI. You can choose to use interpreters for some languages, and AI audio translations for other ones. Closed captions are automatically available for each AI speech translation language.
Branded multilingual video player
Our responsive HTML5 player can be styled as desired. It allows you to display a poster image before the livestream, show interactive messages in an overlay, and much more. Works perfectly in any browser on desktop and mobile.
Full live stream redundancy
Clevercast supports a fully redundant set-up. Our player automatically detects if the main stream becomes unavailable and switches to the backup stream. This way, the live stream won’t drop out if there is an encoder or local network issue.
Cloud recording
Clevercast makes a server-side recording of the multilingual live stream, which can be downloaded. All speech translations can be downloaded as audio streams in a single MP4 and as separate audio-only AAC files. Clevercast can also transcode the recording into single-language MP4 files.
Limit stream accessibility
You can determine who can watch your live stream by configuring white and blacklists for countries, domains and IP addresses. Different settings are possible for each live stream.
Detailed analytics
Our dashboard informs you in real time how many viewers are watching and from which country. After the live stream ends, it provides detailed insights into the behaviour of your viewers.
Conversion to Video on-Demand
The cloud recording of your live stream can easily be converted to Video on-Demand. The VoD player with closed captions can be added to your site or platform by just copying the embed code from Clevercast.
Get Started Now
Start live streaming today with a solution of choice. No credit card required.
Or contact us for more info.