Reusing a Pretrained Language Model on Languages with Limited Corpora for Unsupervised NMT, Chronopoulou et al., 2020
The reused monolingual LM is fine-tuned on both languages and then used to initialize a UNMT model. We propose a new vocabulary extension method, RE-LM, that outperforms a competitive cross-lingual pretraining model and also improves translations on a low-resource supervised setup.