Pipeline Categorization MT

Encoder - Decoder

Process language

Related work to Training

Convolutional Neural Networks

[1] A Convolutional Neural Network for Modelling Sentences by Nal Kalchbrenner, 2014.

[2] Molding CNNs for text: non-linear, non-consecutive convolutions by Tao Le, 2015.

[3] Neural Machine Translation in Linear Time by Nal Kalchbrenner, 2016.

[4] A convolutional Encoder Model for Neural Machine Translation by Jonas Gehring, 2016.

[5] Convolutional Sequence to Sequence Learning by Jonas Gehring, 2017.

[6] Online Segment to Segment Neural Transduction by Yu, 2016.

Recurrent Neural Networks

Description

MultiLingual

[1] Multi-Way, Multilingual Neural Machine Translation with a Shared Attention Mechanism by Orhan Firat, 2016.

[2] Multi-source neural translation by Barret Zoph, 2016.

[3] Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation by Yonghui Wu, 2016.

[4] Google's Multilingual Neural Machine Translation System: Enabling Zero-Shot Translation by Yonghui Wu, 2016.

[5] Toward multilingual neural machine translation with universal encoder and decoder by Han, 2015.

[6] Transfer learning for low-resource neural machine translation by Zoph, 2016.

MonoLingual

[1] Improving neural machine translation models with monolingual data by Rico Sennrich, 2016.

[2] Word Translation Without Parallel Data by Conneau, 2017.

Syntax Structure

[1] Improved Neural Machine Translation with a Syntax-Aware Encoder and Decoder by Chen, 2017.

[2] Sequence-to-Dependency Neural Machine Translation by Wu, 2017.

[3] Learning to Parse and Translate Improves Neural Machine Translation by Eriguchi, 2017.

[4] Predicting Target Language CCG Supertags Improves Neural Machine Translation by Nadejde, 2017.

[5] Tree-to-Sequence Attention Neural Machine Translation by Eriguchi, 2016.

[6] Context Gates for Neural Machine Translation by Tu, 2016.

[7] Distilling the knowledge in a neural network by Hinton, 2015.

[8] Does String-Based Neural MT Learn Source Syntax by Shi, 2016.

[9] Linguistic input Features Improve Neural Machine Translation by Eriguchi, 2017.

Attention

[1] Attention is All you Need by Ashish Vaswani, 2017.

Recurrent Neural Networks

[1] Recurrent continuous translation models by Nal Kalchbrenner, 2013.

[2] Learning phrase representations using RNN encoder-decoder for statistical machine translation by Kyunghyun Cho, 2014.

Recursive Neural Networks

Description

Long Short Term Memory

[1] Sequence to Sequence Learning with Neural Networks by Ilya Sustkever, 2014.

[2] Deep Recurrent Models with Fast-Forward Connection for Neural Machine Translation by Zhou, 2016.

Stacked RNN

Description

Gated Recurrent Unit

Description

Bahdanau

[1] Neural Machine Translation by Jointly Learning to Align and Translate by Dzmitry Bahdanau, 2014.

Thang

[1] Effective Approaches to Attention-based Neural Machine Translation by Minh Thang Luong, 2015.

BiLSTM

Description

BiRNN

Description

Neural Machine Translation

Ensembling model

Description

Experimentation

Architecture and hyperparameters

[1] Massive Exploration of Neural Machine Translation Architectures by Denny Britz, 2017.

Vocabulary

[1] On Using Very Large Target Vocabulary for Neural Machine Translation by Sébastien Jean, 2015.

[2] Variable-Length Word Encodings for Neural Translation Models by Rohan Chitnis, 2015.

[3] Achieving Open Vocabulary Neural Machine Translation with Hibrid Word-Character Models by Minh-Thang Luong, 2016.

Rare Words

[1] Neural Machine Translation of Rare Words with Subwords Units by Rico Senrich, 2015.

[2] Addressing the rare word problem in neural machine translation by Minh-Thang Luong, 2015.

Character Word Level

[1] A Character-level Decoder without Explicit Segmentation for Neural Machine Translation by Junyoung Chung, 2016.

Improve to LSTM

[1] Deep Recurrent Models with Fast-Forward Connections for Neural Machine Translation by ie Zhou, 2016.

MultiTask Learning

[1] Multi-task learning for multiple language translation by Daxiang Dong, 2015.

[2] Multi-task Sequence to Sequence Learning by Minh-Thang Luong, 2016.

Improve parameters

Reduce size

[1] Sequence-Level Knowledge Distillation by Yoon Kim, 2016.

[2] Compression of Neural Machine Translation Models via Pruning by Abigail See, 2016.

Visualization

[1]Visualizing and Understanding Neural Machine Translation by Yanzhuo Ding, 2017.

Type of Neural Network

Attention Mechanics

Improving Attention

[1] Modeling Coverage for Neural Machine Translation by Zhaopeng Tu, 2016.

[2] Incorporating structural alignment biases into a attentional neural translation model by Cohn, 2016.

[3] Structured Attention Networks by Kiim, 2017.

[4] Neural Machine Translation with Recurrent Attention Modeling by Yang, 2016.

New Note

Description

New Note

[1] An Actor-Critic Algorithm for Sequence Prediction by Bahdanau, 2016.

[2] Reward augmented maximum likehood for neural structured prediction by Norouzi, 2016.

[3] Minimum risk training for neural machine translation by Shen, 2016.

Maximum Likehood Estimation

NMT systems & research groups

NMT systems

[1] OpenNMT http://www.statmt.org/moses/?n=Moses.Releases

[2] SysTran http://www.systransoft.com/systran/translation-technology/pure-neural-machine-translation/

[3] Moses http://www.statmt.org/moses/

[4] Marian https://github.com/marian-nmt/marian C++

Tree-structured composition functions

New Note

[1] Transition-based dependency parsing with stack long short-term memory by Dyer, 2015.
[2] Recurrent neural network grammars by Dyer, 2016.

[3] Parsing with compositional vector grammars by Socher, 2013.

[4] Parsing natural scenes and natural language with recursive neural networks by Socher, 2011.

Research groups

[1] Machine Translation Group at The Johns Hopkins University http://statmt.org/jhu/?n=NMTWinterSchool.HomePage

[2] Universidad Pompeu Fabra TALN https://www.upf.edu/web/taln

[3] Special Interest Group for Machine Translation(SIGMT) http://sigmt.org/?n=Main.HomePage

Multidomain

[1] Neural vs. Phrase-Based Machine Translation in a Multi-Domain Scenario by Farajian, 2017.

DataSets

[1] http://opus.lingfil.uu.se/

[2] http://www.statmt.org/wmt13/translation-task.html

[3] http://www.statmt.org/wmt12/translation-task.html

[4] http://www.statmt.org/wmt11/translation-task.html

[5] http://www.statmt.org/wmt10/translation-task.html

[6] Linguistic Data Consortium https://catalog.ldc.upenn.edu/LDC2014T23

[7] UN Parallel Text (Spanish) https://catalog.ldc.upenn.edu/LDC94T4B-3

Online Contest

[1] http://matrix.statmt.org/matrix