MTDNN v1.1.0 introduces components to build a full pipeline

MTDNN v1.1.0 incorporates reusable components to build end to end NLU systems. The new additions include MTDNNDataBuilder and MTDNNTokenizer.

Useful Links

• Find installation instructions and steps here.
• For instructions on how to use the package, please follow this link.
• To see MTDNN v1.1.0 in action, a Jupyter notebook is provided to show a runnable example using the MNLI dataset here.

Summary of Feature/v1.1.0 (#6)

• Add Data Builder Component
• Add Data Tokenizer Component
• Update Process with Data Builder and Tokenizer
• Create new process pipeline for the data
• Add support for transformers
• Add sample MNLI data versioned by Git LFS
• Add support for MLM
• Update README instructions
• Add: component governance files
• Fix component governance dependency alerts
• update dep for component governance alerts
• Create Github Page with and set theme jekyll-theme-cayman
• Update text classification example
• Update generate_requirements_txt.py

In this release, we package MT-DNN as a pip installable library ready to be absorbed in your NLU applications directly from this repository.

Multi-Task Deep Neural Networks for Natural Language Understanding (MT-DNN), an open-source natural language understanding (NLU) toolkit that makes it easy for researchers and developers to train customized deep learning models. Built upon PyTorch and Transformers, MT-DNN is designed to facilitate rapid customization for a broad spectrum of NLU tasks, using a variety of objectives (classification, regression, structured prediction) and text encoders (e.g., RNNs, BERT, RoBERTa, UniLM).

A unique feature of MT-DNN is its built-in support for robust and transferable learning using the adversarial multi-task learning paradigm. To enable efficient production deployment, MT-DNN supports multi-task knowledge distillation, which can substantially compress a deep neural model without significant performance drop. We demonstrate the effectiveness of MT-DNN on a wide range of NLU applications across general and biomedical domains.

Installation steps

For instructions on installing this repo, please refer to this link: Pip install Package

How to use steps

For instructions on how to use the components provided in this library for your task, please refer to this: How To Use

Learning more about MT-DNN

For more information about MT-DNN, please refer to the following papers:

• Multi-Task Deep Neural Networks for Natural Language Understanding - ACL 2019 - Xiaodong Liu, Pengcheng He, Weizhu Chen and Jianfeng Gao

• Improving Multi-Task Deep Neural Networks via Knowledge Distillation for Natural Language Understanding - Xiaodong Liu, Pengcheng He, Weizhu Chen and Jianfeng Gao

• Hybrid Neural Network Model for Commonsense Reasoning - Pengcheng He, Xiaodong Liu, Weizhu Chen and Jianfeng Gao

• On the Variance of the Adaptive Learning Rate and Beyond - Liyuan Liu, Haoming Jiang, Pengcheng He, Weizhu Chen, Xiaodong Liu, Jianfeng Gao and Jiawei Han

• SMART: Robust and Efficient Fine-Tuning for Pre-trained Natural Language Models through Principled Regularized Optimization - Haoming Jiang, Pengcheng He, Weizhu Chen, Xiaodong Liu, Jianfeng Gao and Tuo Zhao

• The Microsoft Toolkit of Multi-Task Deep Neural Networks for Natural Language Understanding - Xiaodong Liu, Yu Wang, Jianshu Ji, Hao Cheng, Xueyun Zhu, Emmanuel Awa, Pengcheng He, Weizhu Chen, Hoifung Poon, Guihong Cao, Jianfeng Gao

• Adversarial Training for Large Neural Language Models - Xiaodong Liu, Hao Cheng, Pengcheng He, Weizhu Chen, Yu Wang, Hoifung Poon and Jianfeng Gao