Rayan
complete
Y Miyata
I am not at all technical, so I am just going to write what I would like to do. I'd like to be able to do a voiceover in Japanese using AI voice developed in English. So a movie with English-speaking actors in it will have voiceovers in those actors AI generated voice but in Japanese. Is something like that possible?
Herman Near
Y Miyata: Theoretically, it should be. I've done the reverse, a Japanese voice set that speaks surprisingly fluent English. However, this all depends on the syllables and words and if they pronounce enough compatible phonemes to train on for reproduction.
Y Miyata
Herman Near: Thank you - that is interesting & great news. I am looking forward to being able to do that! I learned in my previous job that people generally liked to hear famous people's voice (rather than voice overs), while they also preferred not have to read the subtitles. Thanks for sharing the knowledge.
Herman Near
VITS of HuggingFace currently has this, but the emotion renderings are quite terrible without extensive training. With Eleven Labs' extensive accuracy in emotions, this implementation will really change the game not just for the anime/doujin scene.