NAACL 2018 paper, by Sergey Edunov, Myle Ott, Michael Auli, David Grangier, and Marc'Aurelio Ranzato, from Facebook AI Research

In this episode we continue our theme from last episode on structured prediction, talking with Sergey and Myle about their paper.  They did a comprehensive set of experiments comparing many prior structured learning losses, applied to neural seq2seq models.  We talk about the motivation for their work, what turned out to work well, and some details about some of their loss functions.  They introduced a notion of a "pseudo reference", replacing the target output sequence with the highest scoring output on the beam during decoding, and we talk about some of the implications there.  It also turns out the minimizing expected risk was the best overall training procedure that they found for these structured models.

https://www.semanticscholar.org/paper/Classical-Structured-Prediction-Losses-for-Sequence-Edunov-Ott/20ae11c08c6b0cd567c486ba20f44bc677f2ed23

53 - Classical Structured Prediction Losses for Sequence to Sequence Learning, with Sergey and Myle

**The podcast is currently on hiatus. For more active NLP content, check out the Holistic Intelligence Podcast linked below.**

Welcome to the NLP highlights podcast, where we invite researchers to talk about their work in various areas in natural language processing. All views expressed belong to the hosts/guests, and do not represent their employers.

53 - Classical Structured Prediction Losses for Sequence to Sequence Learning, with Sergey and Myle

NLP Highlights