ICLR 2017 paper by Hakan Inan, Khashayar Khosravi, Richard Socher, presented by Waleed.

The paper presents some tricks for training better language models. 
 It introduces a modified loss function for language modeling, where producing a word that is similar to the target word is not penalized as much as producing a word that is very different to the target (I've seen this in other places, e.g., image classification, but not in language modeling).  They also give theoretical and empirical justification for tying input and output embeddings.

https://www.semanticscholar.org/paper/Tying-Word-Vectors-and-Word-Classifiers-A-Loss-Fra-Inan-Khosravi/424aef7340ee618132cc3314669400e23ad910ba

31 - Tying Word Vectors and Word Classifiers: A Loss Framework for Language Modeling

**The podcast is currently on hiatus. For more active NLP content, check out the Holistic Intelligence Podcast linked below.**

Welcome to the NLP highlights podcast, where we invite researchers to talk about their work in various areas in natural language processing. All views expressed belong to the hosts/guests, and do not represent their employers.

31 - Tying Word Vectors and Word Classifiers: A Loss Framework for Language Modeling

NLP Highlights