A history of samples of my singing voice synthesis system.

Code: https://github.com/r9y9/nnsvs

NNSVS: DNN-based singing voice synthesis samples

TEAM AUXILIARYの新しいプロジェクト「SEVEN(No7) for NNSVS/ENUNU」を発表！
経験を積むための研究プロジェクトで能力を見させてくだらい！
プレアルファ版ですが、高機能があります。
今はまだ調査中から完成品をお待ちください！

TEAM AUXILIARY is proud to present our first project: SEVEN(No7) for NNSVS/ENUNU!
This is an independent research project to gain experience and to show we can do!
The current model is a pre-pre-alpha but we believe the program has strong capabilities.
We are excited to do more research and can't wait to share the finished result with everyone!

Special thanks to:
No.7製作委員会/No.7 Production Committee
    https://voiceseven.com/
混乱Pさん/KonranP 
    https://twitter.com/konran_p https://www.youtube.com/channel/UCLDUyE4pBcGNfKcp6763Lpg
admkさん
    (no homepage)
R9y9/山本 龍一様/Ryuichi Yamamoto
    https://r9y9.github.io/ https://twitter.com/r9y9
Crazyさん
    https://github.com/oatsu-gh https://twitter.com/crazy_toho https://crazy.fanbox.cc/
カノンさん/Canon
     https://twitter.com/@canon_73 https://canon-voice.com/
ちかのさん/Chikano
    https://twitter.com/chikano1582
Jeremy
    (no homepage)

No7 (SEVEN) NNSVS PORT DEMONSTRATION

https://github.com/r9y9/nnsvs

Test on a Neural source filter (NSF) model
NOTE: Training is still in progress (80k steps out of 400k)
NOTE: Haven't done any optimization yet

ust: www.nicovideo.jp/watch/sm39319737

Statistical parametric SVS (sampling rate: 48000)
Time-lag model: {
    "_target_": "nnsvs.model.MDN",
    "in_dim": 331,
    "out_dim": 1,
    "hidden_dim": 32,
    "num_layers": 3,
    "dropout": 0.5,
    "num_gaussians": 1
}
Duration model: {
    "_target_": "nnsvs.model.MDN",
    "in_dim": 331,
    "out_dim": 1,
    "hidden_dim": 256,
    "num_layers": 3,
    "dropout": 0.5,
    "num_gaussians": 1
}
Acoustic model: {
    "_target_": "nnsvs.model.ResSkipF0FFConvLSTM",
    "in_dim": 335,
    "out_dim": 206,
    "ff_hidden_dim": 1024,
    "conv_hidden_dim": 512,
    "lstm_hidden_dim": 256,
    "dropout": 0.0,
    "bidirectional": true,
    "in_lf0_idx": 286,
    "out_lf0_idx": 180,
    "in_lf0_min": 5.162578582763672,
    "in_lf0_max": 6.491110801696777,
    "out_lf0_mean": 5.885598413281433,
    "out_lf0_scale": 0.23067350503371858
}
Vocoder model: {
    "generator_type": "HnSincNSF",
    "generator_params": {
        "aux_context_window": 2,
        "cin_channels": 67,
        "cnn_kernel_s": 5,
        "cnn_num_in_block": 10,
        "filter_block_num": 5,
        "harmonic_num": 7,
        "hidden_dim": 64,
        "noise_std": 0.003,
        "out_channels": 1,
        "out_lf0_idx": 60,
        "out_lf0_mean": 5.885598413281433,
        "out_lf0_scale": 0.23067350503371858,
        "out_vuv_idx": 61,
        "sample_rate": 48000,
        "sinc_order": 31,
        "sine_amp": 0.1,
        "upsample_net": "ConvInUpsampleNetwork",
        "upsample_params": {
            "upsample_scales": [
                2,
                3,
                4,
                10
            ],
            "aux_channels": 67,
            "aux_context_window": 2
        },
        "upsample_rate": 240
    }
}

2022/04/05 Kiritan test SVS w/ NSF 80k steps Roshin Yuukai

「波音リツ」歌声データベースVer2.zip を利用しました
https://www.youtube.com/watch?v=K0NuZK0IiGA

GAN-based post-filter ありバージョンです。過剰平滑化問題が緩和されて、クリアになっていま（一方、現状、若干ノイズっぽくなってしまいます。気になる？

モデルの詳細:
Statistical parametric SVS (sampling rate: 48000)
Time-lag model: {
    "_target_": "nnsvs.model.MDN",
    "in_dim": 337,
    "out_dim": 1,
    "hidden_dim": 32,
    "num_layers": 3,
    "num_gaussians": 1
}
Duration model: {
    "_target_": "nnsvs.model.MDN",
    "in_dim": 337,
    "out_dim": 1,
    "hidden_dim": 256,
    "num_layers": 3,
    "num_gaussians": 1
}
Acoustic model: {
    "_target_": "nnsvs.model.ResSkipF0FFConvLSTM",
    "in_dim": 341,
    "out_dim": 206,
    "ff_hidden_dim": 1024,
    "conv_hidden_dim": 512,
    "lstm_hidden_dim": 256,
    "dropout": 0.0,
    "num_lstm_layers": 2,
    "bidirectional": true,
    "init_type": "kaiming_normal",
    "use_mdn": false,
    "in_lf0_idx": 292,
    "out_lf0_idx": 180,
    "in_lf0_min": 4.989291667938232,
    "in_lf0_max": 6.895446300506592,
    "out_lf0_mean": 5.922958879360765,
    "out_lf0_scale": 0.2911988159887554
}
Post-filter model: {
    "_target_": "nnsvs.postfilters.MultistreamPostFilter",
    "stream_sizes": [
        60,
        1,
        1,
        5,
        2,
        1
    ],
    "mgc_postfilter": {
        "_target_": "nnsvs.postfilters.Conv2dPostFilter",
        "in_dim": 58,
        "channels": 64,
        "kernel_size": [
            5,
            5
        ],
        "use_tadn": false,
        "init_type": "kaiming_normal",
        "scale": 1.0
    },
    "mgc_offset": 2,
    "bap_postfilter": {
        "_target_": "nnsvs.postfilters.Conv2dPostFilter",
        "in_dim": 5,
        "channels": 32,
        "kernel_size": [
            5,
            1
        ],
        "use_tadn": false,
        "init_type": "kaiming_normal",
        "scale": 1.0
    },
    "lf0_postfilter": null
}
Vocoder model: WORLD

2022/05/21  Ritsu Sample1 GAN-Postfilter

https://github.com/r9y9/nnsvs

vocoder: WORLD

ust: www.nicovideo.jp/watch/sm39319737

Statistical parametric SVS (sampling rate: 48000)
Time-lag model: {
    "_target_": "nnsvs.model.MDN",
    "in_dim": 331,
    "out_dim": 1,
    "hidden_dim": 32,
    "num_layers": 3,
    "num_gaussians": 1
}
Duration model: {
    "_target_": "nnsvs.model.MDN",
    "in_dim": 331,
    "out_dim": 1,
    "hidden_dim": 256,
    "num_layers": 3,
    "num_gaussians": 1
}
Acoustic model: {
    "_target_": "nnsvs.model.ResSkipF0FFConvLSTM",
    "in_dim": 335,
    "out_dim": 206,
    "ff_hidden_dim": 1024,
    "conv_hidden_dim": 512,
    "lstm_hidden_dim": 256,
    "dropout": 0.0,
    "bidirectional": true,
    "in_lf0_idx": 286,
    "out_lf0_idx": 180,
    "in_lf0_min": 5.162578582763672,
    "in_lf0_max": 6.491110801696777,
    "out_lf0_mean": 5.885598413281433,
    "out_lf0_scale": 0.23067350503371858,
    "init_type": "kaiming_normal"
}
Post-filter model: {
    "_target_": "nnsvs.postfilters.MultistreamPostFilter",
    "stream_sizes": [
        60,
        1,
        1,
        5,
        2,
        1
    ],
    "mgc_postfilter": {
        "_target_": "nnsvs.postfilters.Conv2dPostFilter",
        "in_dim": 58,
        "channels": 64,
        "kernel_size": [
            5,
            5
        ],
        "use_tadn": false,
        "init_type": "kaiming_normal",
        "scale": 1.0
    },
    "mgc_offset": 2,
    "bap_postfilter": {
        "_target_": "nnsvs.postfilters.Conv2dPostFilter",
        "in_dim": 5,
        "channels": 32,
        "kernel_size": [
            5,
            3
        ],
        "use_tadn": false,
        "init_type": "kaiming_normal",
        "scale": 1.0
    },
    "lf0_postfilter": null
}
Vocoder model: {
    "generator_type": "HnSincNSF",
    "generator_params": {
        "aux_context_window": 0,
        "cin_channels": 67,
        "cnn_kernel_s": 5,
        "cnn_num_in_block": 10,
        "filter_block_num": 5,
        "harmonic_num": 7,
        "hidden_dim": 64,
        "noise_std": 0.003,
        "out_channels": 1,
        "out_lf0_idx": 60,
        "out_lf0_mean": 5.885598413281433,
        "out_lf0_scale": 0.23067350503371858,
        "out_vuv_idx": 61,
        "sample_rate": 48000,
        "sinc_order": 31,
        "sine_amp": 0.1,
        "upsample_net": null,
        "upsample_params": {
            "upsample_scales": [
                2,
                3,
                4,
                10
            ],
            "aux_channels": 67,
            "aux_context_window": 0
        },
        "upsample_rate": 240
    }
}

2022/05/22 Kiritan test SVS Roshin Yuukai Full GAN-postfilter

本家: https://www.nicovideo.jp/watch/sm6119955
UST: https://www.youtube.com/watch?v=szFcxJwd5i0
Inst音源: https://www.youtube.com/watch?v=rxb7yCHOw7g

nnsvsのテストです https://github.com/r9y9/nnsvs
調声一切無

転載などはお控えください。音源、USTなど、研究用途として使用しています

2022/05/30 Kiritan Ruka Test

Credit:
Database:「波音リツ」歌声データベースVer2.zip
www.youtube.com/watch?v=K0NuZK0IiGA
UST: https://bowlroll.net/file/193938
原曲: https://www.nicovideo.jp/watch/sm31413058

てるみい、の最後のサビの部分だけ。てるみい最高

開発中のモデルです。以前よりもビブラートがきれいに出ているはず？
無調声、NNSVS の raw output です。IA＆OИE の2パート、どちらも波音リツです

NNSVS: https://nnsvs.github.io/

2022/08/02 NNSVS Ritsu Test Tellme

This demo shows the strong capability of the autoregressive F0 model. 

NOTE:
- Namine Ritsu's database [1] is used to train a backbone model.
- NIT-song 070 database is used for fine-tuning
- WORLD vocoder is used instead of a neural vocoder (to save my time to train a vocoder)
- Sampling rate: 48kHz

[1] Canon, [NamineRitsu] Blue (YOASOBI) [ENUNU model Ver.2, Singing DBVer.2 release], https://www.youtube.com/watch?v=pKeo9IE_L1I, Accessed: 2022.10.06.

2022/11/22 NIT-SONG070 Nitech Jp Song070 AR-F0 model

ルカルカ★ナイトフィーバーのカラオケ音源です。
オリジナル音源だと大手を振っての使用はしづらいかなと思い、作成に至りました。
良かったら歌ってくださいね！

i made karaoke of LUKALUKA★NIGHTFEVER.

i uploaded same track for youtube.
that's subed with romaji.
hope you sing and enjoy^^

https://www.youtube.com/watch?v=vAZPHZaQ2U4

2022/04/05 Kiritan test SVS w/ NSF 80k steps Roshin Yuukai

r9y9

Related tracks