pip install -r requirements.txt

If you want to use apex for AMP training, please clone the apex source code from the repository at github.com to install.

We provide the pre-trained checkpoint of our model at huggingface.co. To reproduce our result in the paper, please first download the pre-processed fine-tuning data (be available soon), then run scripts/finetune.sh

To pretrain our model from scratch, please first download our processed pretraining dataset (be available soon), then download pre-trained WavLM and RoBERTa models from huggingface.co (optional), and run scripts/train-960.sh