Linear Digressions - Old to New

Genomics Backdrop

AI & ML

Linear Digressions

test

A few weeks ago we recorded a conversation with George Church, a Harvard University geneticist and one of the founders of modern genomics. In the resulting podcast, you'll hear Church offer his thoughts on the coming transformation of medicine, whether genes should be patentable, and whether the public is prepared to deal with genetic data.

Here's how Church characterizes the state of genomics:

"It's kind of like '93 on the Web. In fact, in a certain sense, it's more sophisticated than electronics because we have inherited three billion years of amazing technology that was just like a spaceship that was parked in our back yard and we're just reverse-engineering and probably not fully utilizing even the stuff that we've discovered so far."

For more about this episode and links related to the things we discussed, visit http://radar.oreilly.com/2013/07/podcast-george-church-genomics-spaceship-parked-in-our-back-yard.html

[Thumbnail image courtesy Harvard University Center for the Environment]

More on this episode

George Church on genomics: "Like a spaceship that was parked in our back yard"

Machine learning and particle physics go together like peanut butter and jelly--but this is a relatively new development.  

For many decades, physicists looked through their fairly large datasets using the laws of physics to guide their exploration; that tradition continues today, but as ever-larger datasets get made, machine learning becomes a more tractable way to deal with the deluge.  

With this in mind, ATLAS (one of the major experiments at CERN, the European Center for Nuclear Research and home laboratory of the recently discovered Higgs boson) ran a machine learning contest over the summer, to see what advances could be found from opening up the dataset to non-physicists.  

The results were impressive--physicists are smart folks, but there’s clearly lots of advances yet to make as machine learning and physics learn from one another.  And who knows--maybe more Nobel prizes to win as well!

https://www.kaggle.com/c/higgs-boson

Hunting for the Higgs

In the 1850s, there were a lot of things we didn’t know yet: how to create an airplane, how to split an atom, or how to control the spread of a common but deadly disease: cholera.  

When a cholera outbreak in London killed scores of people, a doctor named John Snow used it as a chance to study whether the cause might be very small organisms that were spreading through the water supply (the prevailing theory at the time was miasma, or “bad air”).  By tracing the geography of all the deaths from the outbreak, Snow was practicing elementary data science--and stumbled upon one of history’s most famous outliers.  

In this episode, we’ll tell you more about this single data point, a case of cholera that cracked the case wide open for Snow and provided critical validation for the germ theory of disease.

http://en.wikipedia.org/wiki/1854_Broad_Street_cholera_outbreak

How Outliers Helped Defeat Cholera

Computers are capable of many impressive feats, but making you laugh is usually not one of them. Or could it be? This episode will talk about a custom-built machine learning algorithm that searches through text and writes jokes based on what it finds. 

The jokes are formulaic: they're all of the form "I like my X like I like my Y: Z" where X and Y are nouns, and Z is an adjective that can describe both X and Y. For (dumb) example, "I like my men like I like my coffee: steaming hot." The joke is funny when ZX and ZY are both very common phrases, but X and Y are rarely seen together. 

So, given a large enough corpus of text, the algorithm looks for triplets of words that fit this description and writes jokes based on them. Are the jokes funny? You be the judge...

http://homepages.inf.ed.ac.uk/s0894589/petrovic13unsupervised.pdf

Computers Try to Tell Jokes

Baseball is characterized by a high level of equality between teams; even the best teams might only have 55% win percentages (contrast this with college football, where teams go undefeated pretty regularly). In this regime, where 2 outcomes (Giants win/Giants lose) are approximately equally likely, we can model the win/loss chances with a binomial distribution. 

Using the binomial distribution, we can calculate an interesting little result: what's the chance of the world series going to only 4 games? 5? 6? All the way to 7? Then we can compare to decades' worth of world series data, to see how well the data follows the binomial assumption. 

The result tells us a lot about sports psychology--if each game is independent of the others, 4/5/6/7 game series are equally likely. The data shows a different trend: 4 and 7 game series are significantly more likely than 5 or 6. There's a powerful psychological effect at play--everybody loves the 7th game of the world series, or a good sweep. And it turns out that the baseball teams, whether they intend it or not, oblige our love of short (4) and long (7) world series!

http://blog.philbirnbaum.com/2007/06/winning-world-series-in-x-games.html

Stats of World Series Streaks

A true classic topic in ML: Facial recognition is very high-dimensional, meaning that each picture can have millions of pixels, each of which can be a single feature. It's computationally expensive to deal with all these features, and invites overfitting problems. PCA (principal components analysis) is a classic dimensionality reduction tool that compresses these many dimensions into the few that contain the most variation in the data, and those principal components are often then fed into a classic ML algorithm like and SVM. 

One of the best thing about eigenfaces is the great example code that you can find in sklearn--you can distinguish pictures of world leaders yourself in just a few minutes!

http://scikit-learn.org/stable/auto_examples/applications/face_recognition.html

Facial Recognition with Eigenfaces

It can take dozens of back-and-forth emails just to set up a quick meeting. Now, an artificial intelligence personal assistant can do it for you. Dennis Mortensen, CEO of x.ai, explains why you shouldn’t be afraid.

Goodbye to Personal Assistants

In science-fiction, you’ll encounter a lot of computers that can feel... HAL 9000, Deckard, WALL-E. But what about computers that can really sense your emotions? MIT Professor Rosalind Picard takes a look at this artificial-feeling future.

For more innovative content, follow us:
twitter.com/IHubRadio
www.facebook.com/InnovationHubradio

Computers that Know You're Mad

What is A.I. or artificial intelligence but the 'space of possible minds', argues Murray Shanahan, scientific advisor on the movie Ex Machina and Professor of Cognitive Robotics at Imperial College London.

In this special episode of the a16z Podcast brought to you on the ground from London, Shanahan -- along with journalist-turned-entrepreneur Azeem Azhar (who also curates The Exponential View newsletter on AI and more) and The Economist Deputy Editor Tom Standage (the author of several tech history books) -- we discuss the past, present, and future of A.I. ... as well as how it fits (or doesn't fit) with machine learning and deep learning.

But where are we now in the A.I. evolution? What players do we think will lead, if not win, the current race? And how should we think about issues such as ethics and automation of jobs without descending into obvious extremes? All this and more, including a surprise easter egg in Ex Machina shared by Shanahan, whose work influenced the movie.

a16z Podcast: Artificial Intelligence and the 'Space of Possible Minds'

Genomics pioneer Craig Venter revisits his predictions for the field from a SciFri conversation in 2003.

Mapping Out the Future of Genomics

Genomics data is some of the biggest #bigdata, and doing machine learning on it is unlocking new ways of thinking about evolution, genomic diseases like cancer, and what really makes each of us different for everyone else.  This episode touches on some of the things that make machine learning on genomics data so challenging, and the algorithms designed to do it anyway.

Machine Learning for Genomics

Once we sequenced the human genome, we'd know the cause of -- and therefore be able to help cure -- all diseases... Or so we thought. Turns out, 20,000 genes (and counting) didn't really explain why disease occurred. Sure, some could be explained by mutations in a single genome, but most, like cancer, are too damn complex. And while the focused, singular approach to understanding disease did yield some useful therapeutics, it's now reached its limits. It hasn't helped much on the diagnostics (and early detection) front, either.

That's where a systems approach to bio comes in, drawing on machine learning techniques as well as a sort of "Moore's Law" for genomics that's driving costs down, and fast. We're now focusing on the 99% of the genome that hasn't really been understood yet in terms of how they affect the human body and disease. But what will it take for such an approach to succeed? For one thing, it involves building an applications layer on top of the sequencing layer -- so can we borrow lessons from how the computing industry (from chips to apps) evolved here? What are some of the constraints unique to the healthcare system?

In this episode of the a16z Podcast, Freenome CEO and co-founder Gabriel Otte and a16z bio fund partners Vijay Pande and Malinka Walaliyadde (in conversation with Sonal Chokshi) talk all things genomics and disease from science to business, also covering recent news like Illumina to what's next beyond human genomics to future trends. Including what the ultimate, Elysium-like magical diagnostic machine is (hint: the magical is mundane!).

a16z Podcast: On the Genomics of Disease, From Science to Business

With 100 trillion cells in the human body, bacteria outnumber our own human cells 2 to 1. These bacteria make up one’s microbiome, and particularly bacteria in our guts affect all our key organ functions. They play a role in our health, development and wellness, including endurance, recovery and mental aptitude.

In this episode of Disruptive, Wyss Core Faculty member George Church and Wyss Postdoctoral Fellow Jonathan Scheiman discuss collecting and sequencing gut bacteria of elite athletes to produce customized probiotics - and the potential that these probiotics could give recipients access to some of the biological advantages that make those athletes elite.

Transcript: https://s3-us-west-2.amazonaws.com/wyss-prod/app/uploads/2017/08/08154753/Ep010_Disruptive_Sports_Genomics.pdf

Disruptive: Sports Genomics

What do you get when you cross a support vector machine with matrix factorization?  You get a factorization machine, and a darn fine algorithm for recommendation engines.

Factorization Machines

If you've been in data science for more than a year or two, chances are you've noticed changes in the field as it's grown and matured. And if you're newer to the field, you may feel like there's a disconnect between lots of different stories about what data scientists should know, or do, or expect from their job. This week, we cover two thought pieces, one that arose from interviews with 35(!) data scientists speaking about what their jobs actually are (and aren't), and one from the head of data science at AirBnb organizing core data science work into three main specialties.

Relevant links:
https://hbr.org/2018/08/what-data-scientists-really-do-according-to-35-data-scientists
https://www.linkedin.com/pulse/one-data-science-job-doesnt-fit-all-elena-grewal

The Three Types of Data Scientists, and What They Actually Do

Linear regression is a great tool if you want to make predictions about the mean value that an outcome will have given certain values for the inputs. But what if you want to predict the median? Or the 10th percentile? Or the 90th percentile. You need quantile regression, which is similar to ordinary least squares regression in some ways but with some really interesting twists that make it unique. This week, we’ll go over the concept of quantile regression, and also a bit about how it works and when you might use it.

Relevant links:
https://www.aeaweb.org/articles?id=10.1257/jep.15.4.143
https://eng.uber.com/analyzing-experiment-outcomes/

Quantile Regression

It’s not news that data scientists are expected to be capable in many different areas (writing software, designing experiments, analyzing data, talking to non-technical stakeholders). One thing that has been changing, though, as the field becomes a bit older and more mature, is our ideas about what data scientists should focus on to stay relevant. Should they specialize in a particular area (if so, which one)? Should they instead stay general and work across many different areas? In either case, what are the costs and benefits?

This question has prompted a number of think pieces lately, which are sometimes advocating for specializing, and sometimes pointing out the benefits of generalists. In short, if you’re trying to figure out what to actually do, you might be hearing some conflicting opinions. In this episode, we break apart the arguments both ways, and maybe (hopefully?) reach a little resolution about where to go from here.

The Great Data Science Specialist vs. Generalist Debate

We often hear from folks wondering what advice we can give them as they search for their first job in data science. What does a hiring manager look for? Should someone focus on taking classes online, doing a bootcamp, reading books, something else? How can they stand out in a crowd? 

There’s no single answer, because so much depends on the person asking in the first place, but that doesn’t stop us from giving some perspective. So in this episode we’re sharing that advice out more widely, so hopefully more of you can benefit from it.

Advice to those trying to get a first job in data science

Lunch Hour Lectures - Spring 2013

Prof Aroon HIngorani, Director, UCL Institute of Cardiovascular Science

Greater understanding of how genetic differences influence disease susceptibility and drug response has potentially important healthcare applications. This lecture, marking Heart Awareness Month, will focus on some of the opportunities and challenges of using genomic information to improve personal and public health, using cardiovascular disease as an example.

Listen to more Lunch Hour Lectures from this series:
https://soundcloud.com/uclsound/sets/lunch-hour-lectures-spring-2013

UCL is consistently ranked as one of the world's top universities. Across all disciplines our faculties are known for their research-intensive approaches, academic excellence and engagement with global challenges. This is the basis of our world-renowned degree programmes. Visit us at ucl.ac.uk.

Genomics and Healthcare (28 Feb 2013)

Open source software is ubiquitous throughout data science, and enables the work of nearly every data scientist in some way or another. Open source projects, however, are disproportionately maintained by a small number of individuals, some of whom are institutionally supported, but many of whom do this maintenance on a purely volunteer basis. The health of the data science ecosystem depends on the support of open source projects, on an individual and institutional level.

https://hdsr.mitpress.mit.edu/pub/xsrt4zs2/release/2

The Data Science Open Source Ecosystem

This is a re-release of an episode that originally ran in October 2019.

If you’re trying to manage a project that serves up analytics data for a few very distinct uses, you’d be wise to consider having custom solutions for each use case that are optimized for the needs and constraints of that use cases. You also wouldn’t be YouTube, which found themselves with this problem (gigantic data needs and several very different use cases of what they needed to do with that data) and went a different way: they built one analytics data system to serve them all. Procella, the system they built, is the topic of our episode today: by deconstructing the system, we dig into the four motivating uses of this system, the complexity they had to introduce to service all four uses simultaneously, and the impressive engineering that has to go into building something that “just works.”

Procella: YouTube's super-system for analytics data storage

A few weeks ago, we put out a call for data scientists interested in issues of race and racism, or people studying how those topics can be studied with data science methods, should get in touch to come talk to our audience about their work. This week we’re excited to bring on Todd Hendricks, Bay Area data scientist and a volunteer who reached out to tell us about his studies with the Stanford Open Policing dataset.

A Data Science Take on Open Policing Data

The data science and artificial intelligence community has made amazing strides in the past few years to algorithmically automate portions of the healthcare process. This episode looks at two computer vision algorithms, one that diagnoses diabetic retinopathy and another that classifies liver cancer, and asks the question—are patients now getting better care, and achieving better outcomes, with these algorithms in the mix? The answer isn’t no, exactly, but it’s not a resounding yes, because these algorithms interact with a very complex system (the healthcare system) and other shortcomings of that system are proving hard to automate away. Getting a faster diagnosis from an image might not be an improvement if the image is now harder to capture (because of strict data quality requirements associated with the algorithm that wouldn’t stop a human doing the same job). Likewise, an algorithm getting a prediction mostly correct might not be an overall benefit if it introduces more dramatic failures when the prediction happens to be wrong. For every data scientist whose work is deployed into some kind of product, and is being used to solve real-world problems, these papers underscore how important and difficult it is to consider all the context around those problems.

A Reality Check on AI-Driven Medical Assistants

All good things must come to an end, including this podcast. This is the last episode we plan to release, and it doesn’t cover data science—it’s mostly reminiscing, thanking our wonderful audience (that’s you!), and marveling at how this thing that started out as a side project grew into a huge part of our lives for over 5 years.

It’s been a ride, and a real pleasure and privilege to talk to you each week. Thanks, best wishes, and good night!

—Katie and Ben