Indonesian - Training with Mozilla Datasets

[tansilandre]

Introduction

Hello, I am Tansil from Indonesia.
I am a newbie here, just want to test if this TTS working or not and its working!!!

First thing first, I want to say thank you for @thorstenMueller for all the guide he had uploaded on Youtube, very helpful! Good work!!!
After hours of hours getting back to back reading documentation here and watching Thorsten video, I have tried 2 different system to do a training:

1. Google Collab, it is pretty easy and straight forward also free, but the limitation on GPU getting me nowhere near the model that I expected
2. I had a training with my PC (with 3060 GPU) on windows but it's getting so many crashes and not stable
3. So the solution is I am using WSL2 with windows, with CUDA activated (trust me it's hard to setup but I can give a tip)

Dataset

Since I have a bad microphone, and I want to test the 'water' first, I am using Mozilla datasets. I also aim to use Indonesian language because that's what I needed. So what I done are:

1. Download the Indonesian speaker datasets on Mozilla
2. Create a python script to group and filter a good datasets
3. I found 3 different speaker with 1500+ recorded audio and I use audio with the up_vote > 0 and down_vote < 1 (up_vote and down_vote is on mozilla datasets)
4. Unfortunately from the 3 different speaker, I only could get 1 good model (minimum 50k step)

Difficulties

1. When I first create a Model, I wasn't aware there are 2 Model needed to generate a good voice (TTS & Vocoder)
2. With only TTS Model I could generate a reasonable voice but sound robotic
3. I want to try to do a training on Vocoder but no luck to find a documentation about it
4. Gaining more knowledge, I decided to do training again with VITS (it's combined TTS model + Vocoder model)

Results

The good news is its working as I expected! Here are some output that I want to share:

1. only TTS model with no Vocoder

TTSModelNonVocoder.mp4

2. with VITS training (TTS&Vocoder)

VITS.mp4

3. long version of VITS

test_podcast.mp4

Questions

1. How can we change the voice in the model? if so, how many minutes/ sentences datasets required to change it?
2. I have trained TTS model but I cannot do further Vocoder training, could anyone share how to do a Vocoder Training? Because the TTS 7.0 is not relatable with the current documentation

[erogol]

Why 0.7.0 is not relatable with the current docs?

[tansilandre]

on Vocoder Training, it's very different between training_vocoder.py and documentation on Trainin a Vocoder section

[tansilandre]

I also cannot find on docs folder a relatable documentation about vocoder

[kin0303]

Hi, Mr. Tansil, I'm Dhea from Indonesia too, may I know what model did you train with? I'm trying to use tacotron2 but I always meet problems.

Hai mas tansil, aku dhea dari indonesia juga boleh tau nggak kamu training nya pake model apa? Aku lagi nyoba pake tacotron2 tapi selalu bermasalah.

[erogol]

@tansilandre check the recipes for vocoder training.

[tansilandre]

@BlackMamba1122 pake glowTTS lumayan bagus tapi kaya robot, kalo mau pake VITS. DM aja Dhea kalo emg mau detailnya ke [email protected]

[ardha27]

itu ditrain berapa lama kak?

[kin0303]

itu ditrain berapa lama kak?

Halo, aku juga sempet nge training vits untuk multilanguage. Untuk trainnya aku lumayan lama, sebulanan lebih hasilnya seperti ini:
https://drive.google.com/drive/folders/1qOa8jfzKRRDToN7bItJ20fL6ZNhu8LC4?usp=sharing

[BayuSasongko]

Halo @kin0303 bagaimana langkah2 cara melakukan training pada suara kita sendiri ?
saya ada project voice clone jika boleh bsa join ke [email protected]. Thx u

[jerichosiahaya]

@kin0303 how many steps did it take from you? I've trained using 4k steps and the result is not really good.

[irfanamal]

@kin0303 Boleh tau training sebulanan lebih itu pake device (GPU) apa?

[hnfarif]

itu ditrain berapa lama kak?

Halo, aku juga sempet nge training vits untuk multilanguage. Untuk trainnya aku lumayan lama, sebulanan lebih hasilnya seperti ini: https://drive.google.com/drive/folders/1qOa8jfzKRRDToN7bItJ20fL6ZNhu8LC4?usp=sharing

@kin0303 itu pakai GPU apa kak?

[Ryuodan]

can you show us the graphs from tensorboard and how many steps did it take from you? I'm training a new model and facing some problems.

[habibandpatia]

Saya rasa ini adalah sesuatu yang sangat luar biasa, saya sangat antusis dalam hal ini, sayang sekali keahlian saya sebatas di audio dan video grafhi, besar harapan saya bisa berkontribusi, namu apa daya. saya sudah banyak melakukan pengujian terhadap tts, sejauh ini saya berharap dari indonesia bisa tersedian dengan luas dan baik, saay ini ada, namun tidak banyak yang cukup baik