Compare commits

...

12 Commits
V1 ... main

Author SHA1 Message Date
Dave Niewinski
fe9f6229f6 Minor update to README 2025-07-27 11:59:47 -04:00
Dave's Armoury
03af89b67f
Update README.md 2024-07-10 09:27:00 -04:00
Dave's Armoury
db6784b227
Update requirements.txt 2024-05-14 11:23:43 -04:00
Dave Niewinski
36b11ee1e9 Updated README 2024-03-05 11:48:42 -05:00
Dave Niewinski
d6c905b730 Added GLaDOS blender model and all (i think) models for 3d printing 2024-03-02 20:18:37 -05:00
Dave Niewinski
7ae00833d3 Updated openai 2024-02-20 14:12:30 -05:00
Dave Niewinski
a7030ca8ac Consolidating 2024-02-14 12:54:03 -05:00
Dave Niewinski
5e76654e1d Cleaned up training 2024-02-14 12:52:30 -05:00
Dave Niewinski
855a69cfdd Making training more generic 2024-02-14 11:02:28 -05:00
Dave Niewinski
d67690c1eb Minor QOL fixes 2024-02-13 09:10:57 -05:00
Dave Niewinski
768ea9a014 Mostly config files 2024-02-12 20:58:10 -05:00
Dave Niewinski
d5ea36b7af Adding wheatley notebook. Lots of other files need cleanup first 2024-02-12 20:50:47 -05:00
84 changed files with 408 additions and 87 deletions

View File

@ -1,23 +1,89 @@
# GLaDOS
[<img src="https://img.youtube.com/vi/yNcKTZsHyfA/0.jpg">](https://youtu.be/yNcKTZsHyfA)
## LLM
git clone --depth=1 https://github.com/dusty-nv/jetson-containers
cd jetson-containers
./run.sh --workdir=/opt/llama.cpp/bin $(./autotag llama_cpp) /bin/bash -c './server --model $(huggingface-downloader TheBloke/openchat_3.5-GGUF/openchat_3.5.Q4_K_S.gguf) --n-gpu-layers 999 --threads $(nproc) -c 2048'
## RIVA
## Voice
Look here [https://docs.nvidia.com/deeplearning/riva/user-guide/docs/quick-start-guide.html](https://docs.nvidia.com/deeplearning/riva/user-guide/docs/quick-start-guide.html)
### Training Your Own Voice
Install Jupyter Notebook
pip install notebook
Run jupyter notebook and open [Train_NeMo.ipynb](https://github.com/davesarmoury/GLaDOS/blob/main/TrainNemo/Train_NeMo.ipynb). Uncommenting the install lines in the first cell *should* install everything needed, but that isn't guaranteed. Run one cell at a time until everything is trained. This should be run on a computer with a decent GPU, not a Jetson.
### Setup NGC
You will first need to setup [NGC](https://org.ngc.nvidia.com/setup). Setup the CLI and also login to nvcr.io in Docker
### Using the GLaDOS Voice from Python
You can use the models directly from Python.
### Using the GLaDOS Voice with RIVA
#### Riva Files
You can download the Riva files from [HERE](https://huggingface.co/DavesArmoury/GLaDOS_TTS) and skip this step, or generate your own using the commands below. These commands need to be don on the computer that trained the models, not the computer that will be running them (ie. Jetson). **Note**: All of the filepaths here are for my computer. Just create empty folders to start.
pip3 install whl
pip3 install nemo2riva==2.13.1
nemo2riva --out hifigan.riva hifigan.nemo --key tlt_encode
nemo2riva --out fastpitch.riva fastpitch.nemo --key tlt_encode
#### Deploy Models
Transfer the .riva files from the step above (or you downloaded) to whatever machine will be running the voice. These commands are run on the Riva machine. **Note**: All of the filepaths here are for my computer. Just create empty folders to start.
docker run --gpus all -it --rm \
-v /home/davesarmoury/RIVA/artifacts:/servicemaker-dev \
-v /home/davesarmoury/RIVA/riva_repo:/data \
--entrypoint="/bin/bash" \
nvcr.io/nvidia/riva/riva-speech:2.13.1-servicemaker-l4t-aarch64
riva-build speech_synthesis \
/servicemaker-dev/glados.rmir:tlt_encode \
/servicemaker-dev/glados_fastpitch.riva:tlt_encode \
/servicemaker-dev/glados_hifigan.riva:tlt_encode \
--voice_name=GLaDOS \
--sample_rate 22050
riva-deploy /servicemaker-dev/glados.rmir:tlt_encode /data/models
exit
These commands will create a bunch of folders inside the "artifacts" directory.
#### Riva Setup
Then, get the Quickstart for Riva. This example is for Jetson (arm64), but will change for different architectures
ngc registry resource download-version nvidia/riva/riva_quickstart_arm64:2.13.1
cd riva_quickstart_arm64_v2.13.1
Edit the config.sh file and change *service_enabled_nlp* and *service_enabled_nmt* to false. This isn't necessary, but will speed things up if you aren't using them. Then, download all of the models with the command below
bash riva_init.sh
# See NeMo information for what to do here
Take the folders that were created in the previous step and copy them into the *riva_quickstart_arm64_v2.13.1/model_repository/models*. That should be all you need to setup Riva. Run it with the command below
riva_start.sh
#### Using Riva TTS
To verify that everything is working properly, use the riva python clients. Follow the install instructions [HERE](https://github.com/nvidia-riva/python-clients?tab=readme-ov-file#installation)
Once that is setup, run the tts.py file under riva/client. Pass in "--voice GLaDOS" as parameters and whatever string you want her to say. You can choose to save the audio as a file, or play it directly.
## ASR
Check device number

View File

@ -10,12 +10,25 @@
"#!apt install sox libsndfile1 ffmpeg\n",
"#!pip3 install wheel\n",
"#!pip3 install -r requirements.txt\n",
"#!pip3 install wget unidecode\n",
"#!pip3 install resampy==0.3.1\n",
"#!pip3 install nemo_toolkit[all]\n",
"#!pip3 install numba==0.48\n",
"#!pip3 install librosa==0.8.1\n",
"#!pip3 install pynini"
"#!pip3 install nemo_toolkit[all]"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "ab5977d6-964e-4ebf-bbd8-e5418fdf67ea",
"metadata": {},
"outputs": [],
"source": [
"## GLaDOS ##\n",
"blocklist = [\"potato\", \"_ding_\", \"00_part1_entry-6\", \"_escape_\"]\n",
"sources = [\"https://theportalwiki.com/wiki/GLaDOS_voice_lines_(Portal)\", \"https://theportalwiki.com/wiki/GLaDOS_voice_lines_(Portal_2)\", \"https://theportalwiki.com/wiki/GLaDOS_voice_lines_(Other)\"]\n",
"base_dir=\"glados_out\"\n",
"\n",
"## Wheatley ##\n",
"#blocklist = [\"jailbreakdooropens08\", \"jailbreakdooropens05\", \"jailbreakdooropens07\", \"jailbreakdooropens01\", \"sp_a1_intro7_pickupnags07\", \"sp_a1_wakeup_panic01\", \"sp_a1_wakeup_hacking09\", \"sp_a1_wakeup_hacking12\", \"sp_a1_wakeup_hacking10\", \"sp_a1_wakeup_hacking12\", \"sp_trust_flingalt08\", \"nanobotow03\", \"sp_a2_wheatley_ows\", \"bw_a4_2nd_first_test_solve_nags\", \"bw_sp_a2_core_actually05\", \"bw_sp_a2_core_actually01\", \"bw_sp_a2_core_potato04\", \"bw_sp_a4_tb_trust_drop_solve05\", \"bw_sp_a4_tb_trust_drop_impatient02\", \"bw_sp_a4_tb_wall_button_solve09\", \"bw_sp_a4_tb_polarity_solve04\", \"bw_a4_finale04_whitegel_break01\", \"bw_finale04_portal_opens_short05\", \"bw_finale04_portal_opens_short06\", \"bw_finale04_portal_opens_short07\", \"bw_finale04_portal_opens_short08\", \"bw_a4_finale04_wakeupa09\", \"bw_finale04_portal_opens17\", \"bw_finale04_portal_opens15\", \"bw_finale04_portal_opens13\", \"bw_finale04_portal_opens03\", \"openingwallhitone01\", \"demospherepowerup02\"]\n",
"#sources = [\"https://theportalwiki.com/wiki/wheatley_voice_lines\"]\n",
"#base_dir=\"wheatley_out\"\n"
]
},
{
@ -80,7 +93,6 @@
" BOLD = '\\033[1m'\n",
" UNDERLINE = '\\033[4m'\n",
"\n",
"blocklist = [\"potato\", \"_ding_\", \"00_part1_entry-6\", \"_escape_\"]\n",
"audio_dir = 'audio'\n",
"download_threads = 64\n",
"\n",
@ -131,34 +143,35 @@
" print(bcolors.FAIL + \"[\" + u'\\u2715' + \"] \" + bcolors.ENDC + result[0])\n",
"\n",
"def main():\n",
" r = requests.get(\"https://theportalwiki.com/wiki/GLaDOS_voice_lines\", allow_redirects=False)\n",
"\n",
" urls = []\n",
" filenames = []\n",
" texts = []\n",
"\n",
" soup = BeautifulSoup(r.text.encode('utf-8').decode('ascii', 'ignore'), 'html.parser')\n",
" for link_item in soup.find_all('a'):\n",
" url = link_item.get(\"href\", None)\n",
" if url:\n",
" if \"https:\" in url and \".wav\" in url:\n",
" list_item = link_item.find_parent(\"li\")\n",
" ital_item = list_item.find_all('i')\n",
" if ital_item:\n",
" text = ital_item[0].text\n",
" text = text.replace('\"', '')\n",
" filename = url[url.rindex(\"/\")+1:]\n",
"\n",
" if \"[\" not in text and \"]\" not in text and \"$\" not in text:\n",
" if url not in urls:\n",
" for s in blocklist:\n",
" if s in url:\n",
" break\n",
" else:\n",
" urls.append(url)\n",
" filenames.append(filename)\n",
" text = text.replace('*', '')\n",
" texts.append(text)\n",
" for s in sources:\n",
" r = requests.get(s, allow_redirects=False)\n",
" \n",
" soup = BeautifulSoup(r.text.encode('utf-8').decode('ascii', 'ignore'), 'html.parser')\n",
" for link_item in soup.find_all('a'):\n",
" url = link_item.get(\"href\", None)\n",
" if url:\n",
" if \"https:\" in url and \".wav\" in url:\n",
" list_item = link_item.find_parent(\"li\")\n",
" ital_item = list_item.find_all('i')\n",
" if ital_item:\n",
" text = ital_item[0].text\n",
" text = text.replace('\"', '')\n",
" filename = url[url.rindex(\"/\")+1:]\n",
" \n",
" if \"[\" not in text and \"]\" not in text and \"$\" not in text:\n",
" if url not in urls:\n",
" for s in blocklist:\n",
" if s in url:\n",
" break\n",
" else:\n",
" urls.append(url)\n",
" filenames.append(filename)\n",
" text = text.replace('*', '')\n",
" texts.append(text)\n",
"\n",
" print(\"Found \" + str(len(urls)) + \" urls\")\n",
"\n",
@ -261,13 +274,14 @@
"metadata": {},
"outputs": [],
"source": [
"#!wget https://raw.githubusercontent.com/nvidia/NeMo/r1.14.0/examples/tts/fastpitch_finetune.py\n",
"#!wget https://raw.githubusercontent.com/NVIDIA/NeMo/r1.14.0/examples/tts/hifigan_finetune.py\n",
"# \n",
"#!wget https://raw.githubusercontent.com/nvidia/NeMo/r1.23.0/examples/tts/fastpitch_finetune.py\n",
"#!wget https://raw.githubusercontent.com/NVIDIA/NeMo/r1.23.0/examples/tts/hifigan_finetune.py\n",
" \n",
"#!mkdir -p conf\n",
"#!cd conf \\\n",
"#&& wget https://raw.githubusercontent.com/nvidia/NeMo/r1.14.0/examples/tts/conf/fastpitch_align_v1.05.yaml \\\n",
"#&& wget https://raw.githubusercontent.com/NVIDIA/NeMo/r1.14.0/examples/tts/conf/hifigan/hifigan.yaml \\\n",
"#&& wget https://raw.githubusercontent.com/nvidia/NeMo/r1.23.0/examples/tts/conf/fastpitch_align_v1.05.yaml \\\n",
"#&& wget https://raw.githubusercontent.com/NVIDIA/NeMo/r1.23.0/examples/tts/conf/hifigan/hifigan.yaml \\\n",
"#&& wget https://raw.githubusercontent.com/NVIDIA/NeMo/r1.23.0/scripts/dataset_processing/tts/ljspeech/ds_conf/ds_for_fastpitch_align.yaml \\\n",
"#&& cd .."
]
},
@ -280,12 +294,74 @@
"source": [
"# additional files\n",
"#!mkdir -p tts_dataset_files && cd tts_dataset_files \\\n",
"#&& wget https://raw.githubusercontent.com/NVIDIA/NeMo/r1.14.0/scripts/tts_dataset_files/cmudict-0.7b_nv22.10 \\\n",
"#&& wget https://raw.githubusercontent.com/NVIDIA/NeMo/r1.14.0/scripts/tts_dataset_files/heteronyms-052722 \\\n",
"#&& wget https://raw.githubusercontent.com/NVIDIA/NeMo/r1.14.0/nemo_text_processing/text_normalization/en/data/whitelist/lj_speech.tsv \\\n",
"#&& wget https://raw.githubusercontent.com/NVIDIA/NeMo/r1.23.0/scripts/tts_dataset_files/cmudict-0.7b_nv22.10 \\\n",
"#&& wget https://raw.githubusercontent.com/NVIDIA/NeMo/r1.23.0/scripts/tts_dataset_files/heteronyms-052722 \\\n",
"#&& wget https://raw.githubusercontent.com/NVIDIA/NeMo/r1.23.0/scripts/dataset_processing/tts/ljspeech/lj_speech.tsv \\\n",
"#&& cd .."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "bc0a5478-1f03-42a5-a001-709f80f66083",
"metadata": {},
"outputs": [],
"source": [
"!wget https://raw.githubusercontent.com/nvidia/NeMo/r1.23.0/scripts/dataset_processing/tts/extract_sup_data.py\n",
"!wget https://raw.githubusercontent.com/nvidia/NeMo/r1.23.0/scripts/dataset_processing/tts/compute_speaker_stats.py"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "8880536e-ca5c-4cce-8856-607277ffd145",
"metadata": {},
"outputs": [],
"source": [
"sup_data_path = './fastpitch_sup_data'\n",
"pitch_stats_path = './pitch_stats.json'\n",
"\n",
"# The script extract_sup_data.py writes the pitch mean and pitch std in the commandline. We will parse it to get the pitch mean and std\n",
"#cmd_str_list = \n",
"cmd_str_list = !python3 extract_sup_data.py --config-path \"conf\" manifest_filepath=\"./manifest_train.json\" sup_data_path={sup_data_path} phoneme_dict_path=\"./tts_dataset_files/cmudict-0.7b_nv22.10\" heteronyms_path=\"./tts_dataset_files/heteronyms-052722\"\n",
"\n",
"# Select only the line that contains PITCH_MEAN\n",
"mean_str = [c for c in cmd_str_list if \"PITCH_MEAN\" in c][0]\n",
"min_str = [c for c in cmd_str_list if \"PITCH_MIN\" in c][0]"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "f912ce1b-f5cf-4e23-88ec-be2a4782cf3f",
"metadata": {},
"outputs": [],
"source": [
"pitch_mean_str = mean_str.split('=')[1]\n",
"pitch_mean = float(pitch_mean_str.split(',')[0])\n",
"pitch_std_str = mean_str.split('=')[2]\n",
"pitch_std = float(pitch_std_str)\n",
"pitch_mean, pitch_std\n",
"\n",
"os.environ[\"pitch_mean\"] = str(pitch_mean)\n",
"os.environ[\"pitch_std\"] = str(pitch_std)\n",
"\n",
"print(f\"pitch mean: {pitch_mean}\")\n",
"print(f\"pitch std: {pitch_std}\")\n",
"\n",
"pitch_min_str = min_str.split('=')[1]\n",
"pitch_min = float(pitch_mean_str.split(',')[0])\n",
"pitch_max_str = min_str.split('=')[2]\n",
"pitch_max = float(pitch_max_str)\n",
"pitch_min, pitch_max\n",
"\n",
"os.environ[\"pitch_min\"] = str(pitch_min)\n",
"os.environ[\"pitch_max\"] = str(pitch_max)\n",
"\n",
"print(f\"pitch min: {pitch_min}\")\n",
"print(f\"pitch max: {pitch_max}\")"
]
},
{
"cell_type": "code",
"execution_count": null,
@ -299,16 +375,18 @@
" sup_data_path=./fastpitch_sup_data \\\n",
" phoneme_dict_path=tts_dataset_files/cmudict-0.7b_nv22.10 \\\n",
" heteronyms_path=tts_dataset_files/heteronyms-052722 \\\n",
" whitelist_path=tts_dataset_files/lj_speech.tsv \\\n",
" exp_manager.exp_dir=./glados_out \\\n",
" +whitelist_path=tts_dataset_files/lj_speech.tsv \\\n",
" exp_manager.exp_dir=./{base_dir} \\\n",
" +init_from_nemo_model=./tts_en_fastpitch_align.nemo \\\n",
" trainer.max_epochs=150 \\\n",
" trainer.max_epochs=175 \\\n",
" trainer.check_val_every_n_epoch=25 \\\n",
" model.train_ds.dataloader_params.batch_size=12 model.validation_ds.dataloader_params.batch_size=12 \\\n",
" model.n_speakers=1 model.pitch_mean=165.458 model.pitch_std=40.1891 \\\n",
" model.pitch_fmin=80.0 model.pitch_fmax=2048.0 model.optim.lr=2e-4 \\\n",
" model.train_ds.dataloader_params.batch_size=6 model.validation_ds.dataloader_params.batch_size=6 \\\n",
" model.n_speakers=1 model.pitch_mean={os.environ[\"pitch_mean\"]} model.pitch_std={os.environ[\"pitch_std\"]} \\\n",
" model.pitch_fmin={os.environ[\"pitch_min\"]} model.pitch_fmax={os.environ[\"pitch_max\"]} model.optim.lr=2e-4 \\\n",
" ~model.optim.sched model.optim.name=adam trainer.devices=1 trainer.strategy=null \\\n",
" +model.text_tokenizer.add_blank_at=true \\\n",
" trainer.devices=1 \\\n",
" trainer.strategy=auto \\\n",
")"
]
},
@ -322,7 +400,7 @@
"from nemo.collections.tts.models import HifiGanModel\n",
"from nemo.collections.tts.models import FastPitchModel\n",
"\n",
"vocoder = HifiGanModel.from_pretrained(\"tts_hifigan\")\n",
"vocoder = HifiGanModel.from_pretrained(\"tts_en_hifigan\")\n",
"vocoder = vocoder.eval().cuda()"
]
},
@ -365,7 +443,7 @@
" return spectrogram, audio\n",
"\n",
"def get_best_ckpt_from_last_run(\n",
" base_dir=\"./glados_out\", \n",
" base_dir=\"./\" + base_dir, \n",
" model_name=\"FastPitch\"\n",
" ): \n",
" \n",
@ -430,8 +508,10 @@
"import soundfile as sf\n",
"\n",
"from pathlib import Path\n",
"\n",
"from nemo.collections.tts.torch.helpers import BetaBinomialInterpolator\n",
"from nemo.collections.tts.parts.utils.tts_dataset_utils import (\n",
" BetaBinomialInterpolator,\n",
" beta_binomial_prior_distribution,\n",
")\n",
"\n",
"def load_wav(audio_file, target_sr=None):\n",
" with sf.SoundFile(audio_file, 'r') as f:\n",
@ -522,8 +602,8 @@
" ~model.optim.sched \\\n",
" train_dataset=./hifigan_manifest_train.json \\\n",
" validation_datasets=./hifigan_manifest_validation.json \\\n",
" exp_manager.exp_dir=./glados_out \\\n",
" +init_from_pretrained_model=tts_hifigan \\\n",
" exp_manager.exp_dir=./{base_dir} \\\n",
" +init_from_pretrained_model=tts_en_hifigan \\\n",
" trainer.check_val_every_n_epoch=10 \\\n",
" model/train_ds=train_ds_finetune \\\n",
" model/validation_ds=val_ds_finetune)"
@ -546,8 +626,6 @@
"# Only need to set speaker_id if there is more than one speaker\n",
"speaker_id = None\n",
"\n",
"base_dir=\"glados_out\"\n",
"\n",
"last_fastpitch_ckpt = get_best_ckpt_from_last_run(base_dir=base_dir, model_name=\"FastPitch\")\n",
"last_hifigan_ckpt = get_best_ckpt_from_last_run(base_dir=base_dir, model_name=\"HifiGan\")\n",
"\n",
@ -600,12 +678,14 @@
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "63485879",
"cell_type": "raw",
"id": "9706d079-fd76-49f6-bce9-1a902d930699",
"metadata": {},
"outputs": [],
"source": [
"#### NAMING WILL CHANGE DEPENDING ON CHARACTER ####\n",
"#### NAMING WILL CHANGE DEPENDING ON CHARACTER ####\n",
"#### NAMING WILL CHANGE DEPENDING ON CHARACTER ####\n",
"\n",
"# pip3 install whl\n",
"# pip3 install nemo2riva\n",
"\n",
@ -624,27 +704,35 @@
" nvcr.io/nvidia/riva/riva-speech:2.13.1-servicemaker-l4t-aarch64\n",
"\n",
"riva-build speech_synthesis \\\n",
" /servicemaker-dev/glados.rmir:tlt_encode \\\n",
" /servicemaker-dev/glados_fastpitch.riva:tlt_encode \\\n",
" /servicemaker-dev/glados_hifigan.riva:tlt_encode \\\n",
" --voice_name=GLaDOS \\\n",
" /servicemaker-dev/wheatley.rmir:tlt_encode \\\n",
" /servicemaker-dev/wheatley_fastpitch.riva:tlt_encode \\\n",
" /servicemaker-dev/wheatley_hifigan.riva:tlt_encode \\\n",
" --voice_name=Wheatley \\\n",
" --sample_rate 22050\n",
"\n",
"riva-deploy /servicemaker-dev/glados.rmir:tlt_encode /data/models\n",
"riva-deploy /servicemaker-dev/wheatley.rmir:tlt_encode /data/models\n",
"\n",
"# Exit docker\n",
"\n",
"ngc registry resource download-version nvidia/riva/riva_quickstart\n",
"cd riva_quickstart\n",
"bash riva_init.sh\n",
"# copy glados.riva files into riva models dir\n",
"# copy wheatley.riva files into riva models dir\n",
"bash riva_start.sh"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "61ed1415-104f-4627-af89-15d6583e2b81",
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},

View File

@ -0,0 +1,49 @@
name: "ds_for_fastpitch_align"
manifest_filepath: "train_manifest.json"
sup_data_path: "sup_data"
sup_data_types: [ "align_prior_matrix", "pitch" ]
phoneme_dict_path: "scripts/tts_dataset_files/cmudict-0.7b_nv22.10"
heteronyms_path: "scripts/tts_dataset_files/heteronyms-052722"
dataset:
_target_: nemo.collections.tts.data.dataset.TTSDataset
manifest_filepath: ${manifest_filepath}
sample_rate: 22050
sup_data_path: ${sup_data_path}
sup_data_types: ${sup_data_types}
n_fft: 1024
win_length: 1024
hop_length: 256
window: "hann"
n_mels: 80
lowfreq: 0
highfreq: 8000
max_duration: null
min_duration: 0.1
ignore_file: null
trim: false
pitch_fmin: 65.40639132514966
pitch_fmax: 2093.004522404789
text_normalizer:
_target_: nemo_text_processing.text_normalization.normalize.Normalizer
lang: en
input_case: cased
text_normalizer_call_kwargs:
verbose: false
punct_pre_process: true
punct_post_process: true
text_tokenizer:
_target_: nemo.collections.common.tokenizers.text_to_speech.tts_tokenizers.EnglishPhonemesTokenizer
punct: true
stresses: true
chars: true
apostrophe: true
pad_with_space: true
g2p:
_target_: nemo.collections.tts.g2p.models.en_us_arpabet.EnglishG2p
phoneme_dict: ${phoneme_dict_path}
heteronyms: ${heteronyms_path}

View File

@ -30,7 +30,6 @@ window: hann
phoneme_dict_path: "scripts/tts_dataset_files/cmudict-0.7b_nv22.10"
heteronyms_path: "scripts/tts_dataset_files/heteronyms-052722"
whitelist_path: "nemo_text_processing/text_normalization/en/data/whitelist/lj_speech.tsv"
model:
learn_alignment: true
@ -60,7 +59,6 @@ model:
_target_: nemo_text_processing.text_normalization.normalize.Normalizer
lang: en
input_case: cased
whitelist: ${whitelist_path}
text_normalizer_call_kwargs:
verbose: false
@ -75,14 +73,14 @@ model:
apostrophe: true
pad_with_space: true
g2p:
_target_: nemo_text_processing.g2p.modules.EnglishG2p
_target_: nemo.collections.tts.g2p.models.en_us_arpabet.EnglishG2p
phoneme_dict: ${phoneme_dict_path}
heteronyms: ${heteronyms_path}
phoneme_probability: 0.5
train_ds:
dataset:
_target_: nemo.collections.tts.torch.data.TTSDataset
_target_: nemo.collections.tts.data.dataset.TTSDataset
manifest_filepath: ${train_dataset}
sample_rate: ${model.sample_rate}
sup_data_path: ${sup_data_path}
@ -114,7 +112,7 @@ model:
validation_ds:
dataset:
_target_: nemo.collections.tts.torch.data.TTSDataset
_target_: nemo.collections.tts.data.dataset.TTSDataset
manifest_filepath: ${validation_datasets}
sample_rate: ${model.sample_rate}
sup_data_path: ${sup_data_path}

View File

@ -72,7 +72,7 @@ trainer:
num_nodes: 1
devices: 1
accelerator: gpu
strategy: ddp
strategy: ddp_find_unused_parameters_true
precision: 32
max_steps: ${model.max_steps}
accumulate_grad_batches: 1

View File

@ -72,7 +72,7 @@ trainer:
num_nodes: 1
devices: 1
accelerator: gpu
strategy: ddp
strategy: ddp_find_unused_parameters_true
precision: 32
max_steps: ${model.max_steps}
accumulate_grad_batches: 1

View File

@ -0,0 +1,99 @@
# This config contains the default values for training HiFi-GAN model on HiFi-TTS dataset.
# If you want to train model on other dataset, you can change config values according to your dataset.
# Most dataset-specific arguments are in the head of the config file, see below.
name: "HifiGan"
train_dataset: ???
validation_datasets: ???
# Default values for dataset with sample_rate=44100
sample_rate: 44100
n_mel_channels: 80
n_window_size: 2048
n_window_stride: 512
n_fft: 2048
lowfreq: 0
highfreq: null
window: hann
train_n_segments: 16384
train_max_duration: null # change to null to include longer audios.
train_min_duration: 0.75
val_n_segments: 131072
val_max_duration: null
val_min_duration: 3
defaults:
- model/generator: v1_44100
- model/train_ds: train_ds
- model/validation_ds: val_ds
model:
preprocessor:
_target_: nemo.collections.asr.parts.preprocessing.features.FilterbankFeatures
nfilt: ${n_mel_channels}
lowfreq: ${lowfreq}
highfreq: ${highfreq}
n_fft: ${n_fft}
n_window_size: ${n_window_size}
n_window_stride: ${n_window_stride}
pad_to: 0
pad_value: -11.52
sample_rate: ${sample_rate}
window: ${window}
normalize: null
preemph: null
dither: 0.0
frame_splicing: 1
log: true
log_zero_guard_type: clamp
log_zero_guard_value: 1e-05
mag_power: 1.0
use_grads: false
exact_pad: true
optim:
_target_: torch.optim.AdamW
lr: 0.0002
betas: [0.8, 0.99]
sched:
name: CosineAnnealing
min_lr: 1e-5
warmup_ratio: 0.02
max_steps: 2500000
l1_loss_factor: 45
denoise_strength: 0.0025
trainer:
num_nodes: 1
devices: -1
accelerator: gpu
strategy: ddp_find_unused_parameters_true
precision: 16
max_steps: ${model.max_steps}
accumulate_grad_batches: 1
enable_checkpointing: false # Provided by exp_manager
logger: false # Provided by exp_manager
log_every_n_steps: 100
check_val_every_n_epoch: 10
benchmark: false
exp_manager:
exp_dir: null
name: ${name}
create_tensorboard_logger: true
create_checkpoint_callback: true
checkpoint_callback_params:
monitor: val_loss
mode: min
create_wandb_logger: false
wandb_logger_kwargs:
name: null
project: null
entity: null
resume_if_exists: false
resume_ignore_no_checkpoint: false

View File

@ -0,0 +1,7 @@
_target_: nemo.collections.tts.modules.hifigan_modules.Generator
resblock: 1
upsample_rates: [8,8,4,2]
upsample_kernel_sizes: [16,16,4,4]
upsample_initial_channel: 512
resblock_kernel_sizes: [3,7,11]
resblock_dilation_sizes: [[1,3,5], [1,3,5], [1,3,5]]

View File

@ -0,0 +1,7 @@
_target_: nemo.collections.tts.modules.hifigan_modules.Generator
resblock: 1
upsample_rates: [8,8,2,2]
upsample_kernel_sizes: [16,16,4,4]
upsample_initial_channel: 128
resblock_kernel_sizes: [3,7,11]
resblock_dilation_sizes: [[1,3,5], [1,3,5], [1,3,5]]

View File

@ -0,0 +1,7 @@
_target_: nemo.collections.tts.modules.hifigan_modules.Generator
resblock: 2
upsample_rates: [8,8,4]
upsample_kernel_sizes: [16,16,8]
upsample_initial_channel: 256
resblock_kernel_sizes: [3,5,7]
resblock_dilation_sizes: [[1,2], [2,6], [3,12]]

View File

@ -1,5 +1,5 @@
dataset:
_target_: "nemo.collections.tts.torch.data.VocoderDataset"
_target_: "nemo.collections.tts.data.dataset.VocoderDataset"
manifest_filepath: ${train_dataset}
sample_rate: ${sample_rate}
n_segments: ${train_n_segments}

View File

@ -1,5 +1,5 @@
dataset:
_target_: "nemo.collections.tts.torch.data.VocoderDataset"
_target_: "nemo.collections.tts.data.dataset.VocoderDataset"
manifest_filepath: ${train_dataset}
sample_rate: ${sample_rate}
n_segments: ${train_n_segments}

View File

@ -1,5 +1,5 @@
dataset:
_target_: "nemo.collections.tts.torch.data.VocoderDataset"
_target_: "nemo.collections.tts.data.dataset.VocoderDataset"
manifest_filepath: ${validation_datasets}
sample_rate: ${sample_rate}
n_segments: ${val_n_segments}

View File

@ -1,5 +1,5 @@
dataset:
_target_: "nemo.collections.tts.torch.data.VocoderDataset"
_target_: "nemo.collections.tts.data.dataset.VocoderDataset"
manifest_filepath: ${validation_datasets}
sample_rate: ${sample_rate}
n_segments: ${val_n_segments}

View File

@ -1,9 +1,9 @@
beautifulsoup4==4.11.1
ipython==8.10.0
matplotlib==3.1.2
nemo==4.3.2
nemo_toolkit==1.13.0
num2words==0.5.12
requests==2.31.0
soundfile==0.11.0
torch==1.13.1
beautifulsoup4
ipython
matplotlib
nemo_toolkit
num2words
requests
soundfile
torch
cython

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

@ -1 +1 @@
Subproject commit ed9a8cd5faabc7f82490f44aba20468ef50c04b8
Subproject commit 9da6630a295bd2cdb77ff159fb15fc9d9e44e717