With Apple releasing macOS 12.1 with SharePlay support, developer Jordi Bruin just launched Navi, a universal app that helps people with hearing impairment and other disabilities to easily engage in a FaceTime call using subtitles and live translations.
Bruin, who recently created Cibo, an iOS app that helps you scan and translate restaurant menus when you’re traveling abroad, tells 9to5Mac how he created Navi:
In April I participated in a hackathon and I played around with the mixToTelephonyUplink API to allow mute people to speak on phone calls with Text to Speech (…) During WWDC I watched the SharePlay session and started imaging ways to make the transcription idea real again.
With Navi, the FaceTime client helps users to:
- Instant closed captioning for your FaceTime Calls
- Translate the incoming subtitles in real time to over 20 languages
- Experience build specifically around Picture in Picture
- On macOS the subtitles are overlaid directly on the video window
- Subtitles are end-to-end encrypted through Apple’s servers
Here are the 20 languages available to translate: Polish, Turkish, Thai, Ukranian, German (Austria), French (Switzerland, France, and Canada), Finnish, Vietnamese, Italian, Swedish, German, Japanese, Portuguese (Brazil and Portugal), Spanish (Mexico and Spain), Norwegian, Danish, Croatian, Chinese (Traditional and Simplified), Dutch, and English (Canada, South Africa, Philippines, and United Kingdom), and Arabic (Saudi Arabia).
Although Bruin created this app to help people with hearing impairment and other disabilities that prevent them from engaging easily in a video call, the app is also helpful for people who don’t speak the same language but still need to engage in a real-time conversation:
SharePlay adds a very simple API for developers to pass information between people on a FaceTime call, all encrypted and with very low latency. (…)! I had more than 500 people on the TestFlight, but since testing would require me to call all of them I had to find better ways, so in the end, I had to FaceTime myself a lot from different devices. (…) And I’m still blown away at times when I use the live translation feature and can have a conversation with someone whose language I do not speak. It really feels like a science fiction movie at times.
With these two publics in mind, the developer still wants to keep improving Navi for people with hearing impairment and other disabilities, since Navi just launched:
“I heard back from some of the hearing impaired users on the TestFlight that it’s already super helpful, so foremost I want to hear from them if there is anything I can add to make the app work better for their specific needs.“
To use Navi, just start a FaceTime call, open Navi, tap the “Enable Subtitles” button to invite the other people on the FaceTime call to Navi, and then they just need to accept the SharePlay invitation.
Navi is available for free on the App Store here. Until the end of the year, users can try the subtitles features at no additional cost. In 2022, people will get five free calls, then they’ll have to make a one-time payment of $3.99. To use the Live Translation feature, users need to buy a pack to make calls, which starts a $5.99 and offers 100,000 characters, which should be about an hour of translation depending on how fast people talk.
Check out 9to5Mac on YouTube for more Apple news:
No comments:
Post a Comment