linear digression

There is no known learning algorithm that is more flexible and powerful than the human brain. That's quite inspirational, if you think about it--to level up machine learning, maybe we should be going back to biology and letting millions of year of evolution guide the structure of our algorithms. 

This is the idea behind neural nets, which mock up the structure of the brain and are some of the most studied and powerful algorithms out there. In this episode, we’ll lay out the building blocks of the neural net (called neurons, naturally) and the networks that are built out of them. 

We’ll also explore the results that neural nets get when used to do object recognition in photographs.

Neural Nets (Part 1)

In the last episode, we zipped through neural nets and got a quick idea of how they work and why they can be so powerful. Here’s the real payoff of that work:

In this episode, we’ll talk about a brand-new pair of results, one from Stanford and one from Google, that use neural nets to perform automated picture captioning. One neural net does the object and relationship recognition of the image, a second neural net handles the natural language processing required to express that in an English sentence, and when you put them together you get an automated captioning tool. Two heads are better than one indeed...

Neural Nets (Part 2)

When you sleep, the neural pathways in your brain take the "white noise" of your resting brain, mix in your experiences and imagination, and the result is dreams (that is a highly unscientific explanation, but you get the idea).  What happens when neural nets are put through the same process?  Train a neural net to recognize pictures, and then send through an image of white noise, and it will start to see some weird (but cool!) stuff.

Neural Net Inception

Ever tried to visualize a cluster of data points in 40 dimensions?  Or even 4, for that matter?  We prefer to stick to 2, or maybe 3 if we're feeling well-caffeinated.  The t-SNE algorithm is one of the best tools on the market for doing dimensionality reduction when you have clustering in mind.

Relevant links:
https://www.youtube.com/watch?v=RJVL80Gg3lA

t-SNE: Reduce Your Dimensions, Keep Your Clusters

First up in this episode: a crash course in natural language processing, and important steps if you want to use machine learning techniques on text data.  Then we'll take that NLP know-how and talk about a really cool analysis of State of the Union text, which analyzes the topics and word choices of every President from Washington to Obama.

Relevant link:
https://civisanalytics.com/blog/data-science/2016/01/15/data-science-on-state-of-the-union-addresses/

Text Analysis on the State Of The Union

The reason that neural nets are taking over the world right now is because they can be efficiently trained with the backpropagation algorithm.  In short, backprop allows you to adjust the weights of the neural net based on how good of a job the neural net is doing at classifying training examples, thereby getting better and better at making predictions.  In this episode: we talk backpropagation, and how it makes it possible to train the neural nets we know and love.

Backpropagation

Machine learning on imbalanced classes: surprisingly tricky.  Many (most?) algorithms tend to just assign the majority class label to all the data and call it a day.  SMOTE is an algorithm for manufacturing new minority class examples for yourself, to help your algorithm better identify them in the wild.

Relevant links:
https://www.jair.org/media/953/live-953-2037-jair.pdf

SMOTE: makin' yourself some fake minority data

If modeling is about predicting the unknown, optimization tries to answer the question of what to do, what decision to make, to get the best results out of a given situation.  Sometimes that's straightforward, but sometimes... not so much.  What makes an optimization problem easy or hard, and what are some of the methods for finding optimal solutions to problems?  Glad you asked!  May we recommend our latest podcast episode to you?

Optimization Problems

You might sometimes find that it's hard to get started doing something, but once you're going, it gets easier.  Turns out machine learning algorithms, and especially recommendation engines, feel the same way.  The more they "know" about a user, like what movies they watch and how they rate them, the better they do at suggesting new movies, which is great until you realize that you have to start somewhere.  The "cold start" problem will be our focus in this episode, both the heuristic solutions that help deal with it and a bit of realism about the importance of skepticism when someone claims a great solution to cold starts.

Relevant links:
http://repository.upenn.edu/cgi/viewcontent.cgi?article=1141&context=cis_papers

The Cold Start Problem

This week, we're doing a crash course in recurrent neural networks--what the structural pieces are that make a neural net recurrent, how that structure helps RNNs solve certain time series problems, and the importance of forgetfulness in RNNs.  

Relevant links: 
http://colah.github.io/posts/2015-08-Understanding-LSTMs/

Recurrent Neural Nets

Competing in a machine learning competition on Kaggle is a kind of rite of passage for data scientists.  Losing unexpectedly at the very end of the contest is also something that a lot of us have experienced.  It's not just bad luck: a very specific combination of overfitting on popular competitions can take someone who is in the top few spots in the final days of a contest and bump them down hundreds of slots in the final tally.

How to Lose at Kaggle

Today we are delighted to bring you an interview with Matt Might, computer scientist and medical researcher extraordinaire and architect of President Obama's Precision Medicine Initiative.  As the Obama Administration winds down, we're talking with Matt about the goals and accomplishments of precision medicine (and related projects like the Cancer Moonshot) and what he foresees as the future marriage of data and medicine.  Many thanks to Matt, our friends over at Partially Derivative (hi, Jonathon!) and the White House for arranging this opportunity to chat.  Enjoy!

Data and the Future of Medicine : Interview with Precision Medicine Initiative researcher Matt Might

If one machine learning model is good, are two models better?  In a lot of cases, the answer is yes.  If you build many ok models, and then bring them all together and use them in combination to make your final predictions, you've just created an ensemble model.  It feels a little bit like cheating, like you just got something for nothing, but the results don't like: algorithms like Random Forests and Gradient Boosting Trees (two types of ensemble algorithms) are some of the strongest out-of-the-box algorithms for classic supervised classification problems.  What makes a Random Forest random, and what does it mean to gradient boost a tree?  Have a listen and find out.

Ensemble Algorithms

This week: everybody's favorite WWII-era classifier metric!  But it's not just for winning wars, it's a fantastic go-to metric for all your classifier quality needs.

Rock the ROC Curve

This week's episode is just in time for JupyterCon in NYC, August 22-25...

Jupyter notebooks are probably familiar to a lot of data nerds out there as a great open-source tool for exploring data, doing quick visualizations, and packaging code snippets with explanations for sharing your work with others.  If you're not a data person, or you are but you haven't tried out Jupyter notebooks yet, here's your nudge to go give them a try.  In this episode we'll go back to the old days, before notebooks, and talk about all the ways that data scientists like to work that wasn't particularly well-suited to the command line + text editor setup, and talk about how notebooks have evolved over their lifetime to become even more powerful and well-suited to the data scientist's workflow.

Jupyter Notebooks

As data science matures as a field, it's becoming clearer what attributes a data science team needs to have to elevate their work to the next level.  Most of our episodes are about the cool work being done by other people, but this one summarizes some thinking Katie's been doing herself around how to guide data science teams toward more mature, effective practices.  We'll go through five key characteristics of great data science teams, which we collectively refer to as "disciplined data science," and why they matter.

Disciplined Data Science

The Fourier transform is one of the handiest tools in signal processing for dealing with periodic time series data. Using a Fourier transform, you can break apart a complex periodic function into a bunch of sine and cosine waves, and figure out what the amplitude, frequency and offset of those component waves are. It's a really handy way of re-expressing periodic data--you'll never look at a time series graph the same way again.

The Fourier Transform

If you've done image recognition or computer vision tasks with a neural network, you've probably used a convolutional neural net. This episode is all about the architecture and implementation details of convolutional networks, and the tricks that make them so good at image tasks.

Convolutional Neural Nets

A huge part of the ascent of deep learning in the last few years is related to advances in computer hardware that makes it possible to do the computational heavy lifting required to build models with thousands or even millions of tunable parameters. This week we'll pretend to be electrical engineers and talk about how modern machine learning is enabled by hardware.

CPUs, GPUs, TPUs: Hardware for Deep Learning

If you've been in data science for more than a year or two, chances are you've noticed changes in the field as it's grown and matured. And if you're newer to the field, you may feel like there's a disconnect between lots of different stories about what data scientists should know, or do, or expect from their job. This week, we cover two thought pieces, one that arose from interviews with 35(!) data scientists speaking about what their jobs actually are (and aren't), and one from the head of data science at AirBnb organizing core data science work into three main specialties.

Relevant links:
https://hbr.org/2018/08/what-data-scientists-really-do-according-to-35-data-scientists
https://www.linkedin.com/pulse/one-data-science-job-doesnt-fit-all-elena-grewal

linear digression

Yasaman Haghpanah