Data Science

Overfitting to your training data can be avoided by evaluating your machine learning algorithm on a holdout test dataset, but what about overfitting to the test data?  Turns out it can be done, easily, and you have to be very careful to avoid it.  But an algorithm from the field of privacy research shows promise for keeping your test data safe from accidental overfitting

Thresholdout: Down with Overfitting

We're back again with friend of the pod Walt, former software engineer extraordinaire and current data scientist extraordinaire, to talk about some best practices from software engineering that are ready to jump the fence over to data science.  If last week's episode was for software engineers who are interested in becoming more like data scientists, then this week's episode is for data scientists who are looking to improve their game with best practices from software engineering.

What Data Scientists Can Learn from Software Engineers

It's been a busy hurricane season in the Southeastern United States, with millions of people making life-or-death decisions based on the forecasts around where the hurricanes will hit and with what intensity.  In this episode we'll deconstruct those models, talking about the different types of models, the theory behind them, and how they've evolved through the years.

Hurricane Forecasting

This week, we've got a fun paper by our friends at Google about the hidden costs of maintaining machine learning workflows.  If you've worked in software before, you're probably familiar with the idea of technical debt, which are inefficiencies that crop up in the code when you're trying to go fast.  You take shortcuts, hard-code variable values, skimp on the documentation, and generally write not-that-great code in order to get something done quickly, and then end up paying for it later on.  This is technical debt, and it's particularly easy to accrue with machine learning workflows.  That's the premise of this episode's paper.

Machine Learning: The High Interest Credit Card of Technical Debt

How will you prepare yourself for imperfection going forward? 

If you enjoyed this episode, check out show notes, resources, and more at www.superdatascience.com/246

The Super Data Science Podcast: Big Data | Analytics Careers | Mentors | Success

SDS 246: Boost Your Self-Confidence

I hope that by the end of this episode of the AI in Industry podcast, you'll not only be able to hire better data scientists who will be a fit for your business problems and build better data science teams, but also pick the AI applications and use cases that you should bring into your business versus those that you shouldn't. This episode, we interview Brooke Wenig, the machine learning practice lead at Databricks. Databricks was founded by the folks who created Apache Spark. Those of you who are technically savvy with AI will be familiar with Apache Spark as an open source language for artificial intelligence and distributed computing. Wenig works with a lot of companies with Databricks. Databricks is now close to 700 folks&nbsp;and helps implement AI applications into, oftentimes, large enterprise environments. Wenig speaks with us this week about what to look for in an actual data scientist and how to find data science folks with the right skills to be able to communicate to business people, not just to work with models. What should people be capable of; how should they be capable of thinking? Hopefully, some of you will have better interview questions by the end of this podcast. In addition, we ask Brooke about what the value of covering the cutting edge applications of AI is, looking at what's working in industry. How does that help us in our own business make better decisions? Read the full article on Emerj.com

Why Executives Should Keep Up with AI Trends in Business

In this episode of the AI in Industry podcast, we speak with Marshall Choy, VP of Product at SambaNova, an AI hardware firm based in the Bay Area. SambaNova was founded by a number of Oracle and Sun Micro Systems alumni. We speak with Choy on two fundamental questions:  How will business models fundamentally change with respect to new AI hardware capabilities? How can business leaders think about their AI hardware needs?  SambaNova is one of many firms that's going to be advertising at the Kisaco Research AI Hardware Summit in Beijing June 4th and 5th.

Data Science

Sebastian Cariño