NAACL 2018 paper by James Thorne, Andreas Vlachos, Christos Christodoulopoulos, and Arpit Mittal

James tells us about his paper, where they created a dataset for fact checking.  We talk about how this dataset relates to other datasets, why a new one was needed, how it was built, and how well the initial baseline does on this task.  There are some interesting side notes on bias in dataset construction, and on how "fact checking" relates to "fake news" ("fake news" could mean that an article is actively trying to deceive or mislead you; "fact checking" here is just determining if a single claim is true or false given a corpus of assumed-correct reference material).  The baseline system does quite poorly, and the lowest-hanging fruit seems to be in improving the retrieval component that finds relevant supporting evidence for claims.

There's a workshop and shared task coming up on this dataset: http://fever.ai/.  The shared task test period starts on July 24th - get your systems ready!

https://www.semanticscholar.org/paper/FEVER%3A-a-Large-scale-Dataset-for-Fact-Extraction-Thorne-Vlachos/7b1f840ecfafb94d2d9e6e926696dba7fad0bb88

Episode 60

60 - FEVER: a large-scale dataset for Fact Extraction and VERification, with James Thorne

**The podcast is currently on hiatus. For more active NLP content, check out the Holistic Intelligence Podcast linked below.**

Welcome to the NLP highlights podcast, where we invite researchers to talk about their work in various areas in natural language processing. All views expressed belong to the hosts/guests, and do not represent their employers.

60 - FEVER: a large-scale dataset for Fact Extraction and VERification, with James Thorne

NLP Highlights