machine Learning

To hear

Machine Learning & Data Science

Data Science

Linear Digressions

Good stuff

Many see willpower as something you develop like a muscle, something you can strengthen through practice and mental exercise, but the latest research suggests willpower runs on an internal battery, one that can be drained after heavy use, but recharges after rest and reward. Once you've used it up, you much recharge it or else you'll be unable to keep your hand out of the cookie jar. Speaking of cookies...we also explore in this episode how psychologists have used cookies in novel ways to uncover the secrets of our minds. 

032 - Ego Depletion

What motivates you to keep going, to reach for your dreams, to persist and endure? Psychology has, over the last 40 years, learned a great deal about human motivation and drive. In this episode we ask Daniel Pink, author of Drive, how we can better put that knowledge to use in our lives, and in our workplaces and institutions.

037 - Motivation - Daniel Pink



Before Pope Francis, Fr. James Martin was perhaps the best loved Jesuit in American life. He’s followed the calling of the founder of the Jesuit order, St. Ignatius of Loyola, to “find God in all things” – and in 21st century forms. To delve into Fr. Martin’s way of being in the world is to discover the "spiritual exercises" St. Ignatius designed to be accessible to everyone more than six centuries ago. Also, "un-taming" Christmas.

See more at www.onbeing.org/program/james-martin-finding-god-in-all-things/7121

James Martin — Finding God in All Things

Now that we know about eigenfaces (if you don't, listen to the previous episode), let's talk about how it breaks down. 

Variations that are trivial to humans when identifying faces can really mess up computer-driven facial ID--expressions, lighting, and angle are a few. Something that can easily happen is an algorithm can optimize to identify one of those traits, rather than the underlying trait of whether the person is the same (for example, if the training image is me smiling, you may reject an image of me frowning but accidentally approve an image of another woman smiling). 

Fisherfaces uses a fisher linear discriminant to distinguish based on the dimension in the data that shows the smallest inter-class distance, rather than maximizing the variation overall (we'll unpack this statement), and it is much more robust than our pal eigenfaces when there's shadows, cut-off images, expressions, etc.

http://www.cs.columbia.edu/~belhumeur/journal/fisherface-pami97.pdf

Better Facial Recognition with Fisherfaces

Courage is borne out of vulnerability, not strength. This finding of Brené Brown’s research on shame and "wholeheartedness" shook the perfectionist ground beneath her own feet. And now it’s inspiring millions to reconsider the way they live, parent, and navigate relations with members of the opposite gender.

See more at www.onbeing.org/program/brene-brown-on-vulnerability/4928

Brené Brown — The Courage to Be Vulnerable

Ray Kurzweil‘s impact on my life in general but especially on what I have been doing for the past 3 or 4 years is hard to exaggerate. It is a simple fact that, if I haven’t read his seminal book The Singularity is Near, I would be neither blogging nor podcasting about exponential technologies, not to mention going to Singularity University. And so it was with great excitement and some trepidation that I went to interview Dr. Kurzweil in his office in Boston.

Part of my trepidation came from some technical concerns: I wish I could buy a better camera. I wish I could hire a team of audio and video professionals so that I can focus on the interview itself. I wish I did a better job with the set up. I wish I had noticed that Ray’s lavalier mic has slipped out of its holder… The list is exponential. Still, if there is one thing that I’ve learned since I started podcasting is that we do get better. But it takes time. Meanwhile, my consolation is that eventually I will have to do another interview with Ray Kurzweil just so I get it right that time. For now, however, I hope that the content of this one will make up for its technical deficiencies.

During our conversation with Dr. Kurzweil we cover a wide variety of topics such as: how and why at age 5 Ray decided to become an inventor; his unique background of being born to Jewish parents but brought up in a Unitarian Church; his early interest in issues such as religious tolerance, poverty, social inequality and justice; 3D printing, open source, patents, progress and intellectual property rights; Watson, artificial intelligence, the Turing Test and human rights for AI, the technological singularity and some criticism thereof; his upcoming book How To Create A Mind and his Pattern Recognition Theory of Mind; the evolutionary advantages of intelligence; the benefits of reverse-engineering the human brain for the creation of AI and whether the latter would be interested in pondering and solving humanity’s greatest problems.

My favorite quote that I will take away from this interview with Ray Kurzweil is:

“Don’t be too concerned about what’s practical. Follow your passion and be who you would like to be.”

Ray Kurzweil on the Singularity: Be Who You Would Like To Be

Gary Marcus is not only a professor in psychology but also a computer scientist, programmer, AI researcher and best selling author. He recently wrote a critical article titled Ray Kurzweil’s Dubious New Theory of Mind that was publsihed by The New Yorker. After reading his blog post I thought that it will be interesting to invite Gary on Singularity 1 on 1 so that he can talk more about his argument.   

During our conversation with Gary we cover a wide variety of topics such as: what is psychology and how he got interested in it; his theory of mind in general and the idea that the mind is a kluge in particular; why the best approach to creating AI is at the bridge between neuroscience and psychology; other projects such as Henry Markram‘s Blue Brain, Randal Koene‘s Whole Brain Emulation, and Dharmendra Modha‘s SyNAPSE; Ray Kurzweil’s Patern Recognition Theory of Mind; Deep Blue, Watson and the lessons thereof; his take on the technological singularity and the ethics surrounding the creation of friendly AI…

Gary Marcus: How do we bridge the mind with the brain?!…

After the Challenger exploded in 1986, killing all 7 astronauts aboard, an investigation into the cause was immediately launched. 

In the cold temperatures the night before the launch, the o-rings that seal off the fuel tanks from the rocket boosters became inflexible, so they did not seal properly, which led to the fuel tank explosion. NASA knew that there could be o-ring problems, but performed the analysis of their data incorrectly and ended up massively underestimating the risk associated with the cold temperatures. 

In this episode, we'll unpack the mistakes they made. We'll talk about how they excluded data points that they thought were irrelevant but which actually were critical to recognizing a fatal pattern.

Statistical Mistakes and the Challenger Disaster

Steeped in the cutting edge of research around the social lives of networked teens, danah boyd demystifies technology while being wise about the changes it’s making to life and relationship. She has intriguing advice on the technologically-fueled generation gaps of our age — that our children’s immersion in social media may offer a kind of respite from their over-structured, overscheduled analog lives. And that cyber-bullying is an online reflection of the offline world, and blaming technology is missing the point.

See more at www.onbeing.org/program/danah-boyd-online-reflections-of-our-offline-lives/7449

danah boyd — Online Reflections of Our Offline Lives

This episode is inspired by one of our projects for Intro to Machine Learning: given a writing sample, can you use machine learning to identify who wrote it? Turns out that the answer is yes, a person’s writing style is as distinctive as their vocal inflection or their gait when they walk. 

By tracing the vocabulary used in a given piece, and comparing the word choices to the word choices in writing samples where we know the author, it can be surprisingly clear who is the more likely author of a given piece of text. 

We’ll use a seminal paper from the 1960’s as our example here, where the Naive Bayes algorithm was used to determine whether Alexander Hamilton or James Madison was the more likely author of a number of anonymous Federalist Papers.

Inferring Authorship (Part 1)

Now that we’re up to speed on the classic author ID problem (who wrote the unsigned Federalist Papers?), we move onto a couple more contemporary examples. 

First, J.K. Rowling was famously outed using computational linguistics (and Twitter) when she wrote a book under the pseudonym Robert Galbraith. 

Second, we’ll talk about a mystery that still endures--who is Satoshi Nakamoto? Satoshi is the mysterious person (or people) behind an extremely lucrative cryptocurrency (aka internet money) called Bitcoin; no one knows who he, she or they are, but we have plenty of writing samples in the form of whitepapers and Bitcoin forum posts. We’ll discuss some attempts to link Satoshi Nakamoto with a cryptocurrency expert and computer scientist named Nick Szabo; the links are tantalizing, but not a smoking gun. “Who is Satoshi” remains an example of attempted author identification where the threads are tangled, the conclusions inconclusive and the stakes high.

Inferring Authorship (Part 2)

Let’s talk money. As a “hot” career right now, data science can pay pretty well. But for an individual person matched with a specific job or industry, how much should someone expect to make? 

Since Katie was on the job market lately, this was something she’s been researching, and it turns out that data science itself (in particular linear regressions) has some answers. 

In this episode, we go through a survey of hundreds of data scientists, who report on their job duties, industry, skills, education, location, etc. along with their salaries, and then talk about how this data was fed into a linear regression so that you (yes, you!) can use the patterns in the data to know what kind of salary any particular kind of data scientist might expect.

Careers in Data Science

Come get a little "out there" with us this week, as we use a meta-study of extrasensory perception (or ESP, often used in the same sentence as "psychics") to chat about Bayesian vs. frequentist statistics.

Bayesian Psychics

This machine learning algorithm beat the human champions at Jeopardy.  What is... Watson?

Watson

Doing some science, and want to know if you might have found something?  Or maybe you've just accomplished the scientific equivalent of going fishing and reeling in an old boot?  Frequentist p-values can help you distinguish between "eh" and "oooh interesting".  Also, there's a lot of physics in this episode, nerds.

PFun with P Values

O'Reilly's Ben Lorica chats with Mike Cafarella, assistant professor of computer science at the University of Michigan and co-founder of both Hadoop and Nutch.

From Search to Distributed Computing to Large-scale Information Extraction

In this special holiday episode of the O’Reilly Data Show, I look back at two conversations I had earlier this year at the Spark Summit in San Francisco. The first segment is an on-stage fireside chat with Ben Horowitz, co-founder of Andreessen Horowitz and author of The Hard Thing About Hard Things.

In the second segment, Reynold Xin, one of the architects of Apache Spark, explains the rise of Apache Spark in China.

Investing in Big Data Technologies

The Google AdWords algorithm is (famously) an auction system for allocating a massive amount of online ad space in real time--with that fascinating use case in mind, this episode is part one in a two-part series all about auctions.  We dive into the theory of auctions, and what makes a "good" auction.   

Relevant links:
https://en.wikipedia.org/wiki/English_auction
http://people.ischool.berkeley.edu/~hal/Papers/2006/position.pdf
http://www.benedelman.org/publications/gsp-060801.pdf 

Going Once, Going Twice: Auctions (Part 1)

The Google ads auction is a special kind of auction, one you might not know as well as the famous English auction (which we talked about in the last episode).  But if it's what Google uses to sell billions of dollars of ad space in real time, you know it must be pretty cool.

Relevant links:
https://en.wikipedia.org/wiki/English_auction
http://people.ischool.berkeley.edu/~hal/Papers/2006/position.pdf
http://www.benedelman.org/publications/gsp-060801.pdf 

Sold!  Auctions (Part 2)

Artificial intelligence includes a number of different strategies for how to make machines more intelligent, and often more human-like, in their ability to learn and solve problems.  An ambitious group of researchers is working right now to classify all the approaches to AI, perhaps as a first step toward unifying these approaches and move closer to strong AI.  In this episode, we'll touch on some of the most provocative work in many different subfields of artificial intelligence, and their strengths and weaknesses.

Relevant links:
https://www.technologyreview.com/s/544606/can-this-man-make-aimore-human/
https://www.youtube.com/watch?v=B8J4uefCQMc
http://venturebeat.com/2013/11/29/sentient-code-an-inside-look-at-stephen-wolframs-utterly-new-insanely-ambitious-computational-paradigm/
http://www.slate.com/articles/technology/bitwise/2014/03/stephen_wolfram_s_new_programming_language_can_he_make_the_world_computable.html

Paradigms in Artificial Intelligence

Competing in a machine learning competition on Kaggle is a kind of rite of passage for data scientists.  Losing unexpectedly at the very end of the contest is also something that a lot of us have experienced.  It's not just bad luck: a very specific combination of overfitting on popular competitions can take someone who is in the top few spots in the final days of a contest and bump them down hundreds of slots in the final tally.

How to Lose at Kaggle

Open source software is ubiquitous throughout data science, and enables the work of nearly every data scientist in some way or another. Open source projects, however, are disproportionately maintained by a small number of individuals, some of whom are institutionally supported, but many of whom do this maintenance on a purely volunteer basis. The health of the data science ecosystem depends on the support of open source projects, on an individual and institutional level.

https://hdsr.mitpress.mit.edu/pub/xsrt4zs2/release/2

The Data Science Open Source Ecosystem

This is a re-release of an episode that originally ran in October 2019.

If you’re trying to manage a project that serves up analytics data for a few very distinct uses, you’d be wise to consider having custom solutions for each use case that are optimized for the needs and constraints of that use cases. You also wouldn’t be YouTube, which found themselves with this problem (gigantic data needs and several very different use cases of what they needed to do with that data) and went a different way: they built one analytics data system to serve them all. Procella, the system they built, is the topic of our episode today: by deconstructing the system, we dig into the four motivating uses of this system, the complexity they had to introduce to service all four uses simultaneously, and the impressive engineering that has to go into building something that “just works.”

Procella: YouTube's super-system for analytics data storage

A few weeks ago, we put out a call for data scientists interested in issues of race and racism, or people studying how those topics can be studied with data science methods, should get in touch to come talk to our audience about their work. This week we’re excited to bring on Todd Hendricks, Bay Area data scientist and a volunteer who reached out to tell us about his studies with the Stanford Open Policing dataset.

A Data Science Take on Open Policing Data

The data science and artificial intelligence community has made amazing strides in the past few years to algorithmically automate portions of the healthcare process. This episode looks at two computer vision algorithms, one that diagnoses diabetic retinopathy and another that classifies liver cancer, and asks the question—are patients now getting better care, and achieving better outcomes, with these algorithms in the mix? The answer isn’t no, exactly, but it’s not a resounding yes, because these algorithms interact with a very complex system (the healthcare system) and other shortcomings of that system are proving hard to automate away. Getting a faster diagnosis from an image might not be an improvement if the image is now harder to capture (because of strict data quality requirements associated with the algorithm that wouldn’t stop a human doing the same job). Likewise, an algorithm getting a prediction mostly correct might not be an overall benefit if it introduces more dramatic failures when the prediction happens to be wrong. For every data scientist whose work is deployed into some kind of product, and is being used to solve real-world problems, these papers underscore how important and difficult it is to consider all the context around those problems.

A Reality Check on AI-Driven Medical Assistants

All good things must come to an end, including this podcast. This is the last episode we plan to release, and it doesn’t cover data science—it’s mostly reminiscing, thanking our wonderful audience (that’s you!), and marveling at how this thing that started out as a side project grew into a huge part of our lives for over 5 years.

It’s been a ride, and a real pleasure and privilege to talk to you each week. Thanks, best wishes, and good night!

—Katie and Ben