John Schulman is a Research Scientist and co-founder of Open AI. John co-leads the reinforcement learning team, researching algorithms that safely and efficiently learn by trial and error and by imitating humans.

His PhD thesis is titled "Optimizing Expectations: From Deep Reinforcement Learning to Stochastic Computation Graphs", which he completed in 2016 at Berkeley. We talk about his work on stochastic computation graphs and TRPO, how it evolved to PPO and how it's used in large-scale applications like Open AI Five, as well as his recent work on generalization in RL. 

Episode notes: https://cs.nyu.edu/~welleck/episode7.html

Follow the Thesis Review (@thesisreview) and Sean Welleck (@wellecks) on Twitter, and find out more info about the show at https://cs.nyu.edu/~welleck/podcast.html

Support The Thesis Review at www.buymeacoffee.com/thesisreview

[07] John Schulman - Optimizing Expectations: From Deep RL to Stochastic Computation Graphs

Each episode of The Thesis Review is a conversation centered around a researcher's PhD thesis, giving insight into their history, revisiting older ideas, and providing a valuable perspective on how their research has evolved (or stayed the same) since.