Linear Digressions

LinearDegresion

queue

test

What do you get when you cross a support vector machine with matrix factorization?  You get a factorization machine, and a darn fine algorithm for recommendation engines.

Factorization Machines

If you've been in data science for more than a year or two, chances are you've noticed changes in the field as it's grown and matured. And if you're newer to the field, you may feel like there's a disconnect between lots of different stories about what data scientists should know, or do, or expect from their job. This week, we cover two thought pieces, one that arose from interviews with 35(!) data scientists speaking about what their jobs actually are (and aren't), and one from the head of data science at AirBnb organizing core data science work into three main specialties.

Relevant links:
https://hbr.org/2018/08/what-data-scientists-really-do-according-to-35-data-scientists
https://www.linkedin.com/pulse/one-data-science-job-doesnt-fit-all-elena-grewal

The Three Types of Data Scientists, and What They Actually Do

The Poisson distribution is a probability distribution function used to for events that happen in time or space. It’s super handy because it’s pretty simple to use and is applicable for tons of things—there are a lot of interesting processes that boil down to “events that happen in time or space.” This episode is a quick introduction to the distribution, and then a focus on two of our favorite applications: using the Poisson distribution to identify supernovas and study army deaths from horse kicks.

Better Know a Distribution: The Poisson Distribution

When you think about it, it’s pretty amazing that we can draw conclusions about huge populations, even the whole world, based on datasets that are comparatively very small (a few thousand, or a few hundred, or even sometimes a few dozen). That’s the power of statistics, though. This episode is kind of a two-for-one but we’re excited about it—first we’ll talk about the Normal or Gaussian distribution, which is maybe the most famous probability distribution function out there, and then turn to the Central Limit Theorem, which is one of the foundational tenets of statistics and the real reason why the Normal distribution is so important.

The Normal Distribution and the Central Limit Theorem

Bringing you another old classic this week, as we gear up for 2019! See you next week with new content.

Word2Vec is probably the go-to algorithm for vectorizing text data these days.  Which makes sense, because it is wicked cool.  Word2Vec has it all: neural networks, skip-grams and bag-of-words implementations, a multiclass classifier that gets swapped out for a binary classifier, made-up dummy words, and a model that isn't actually used to predict anything (usually).  And all that's before we get to the part about how Word2Vec allows you to do algebra with text.  Seriously, this stuff is cool.

Re-release: Word2Vec

with James J. Collins, Vijay Pande (@vijaypande), and Hanne Tidnam (@omnivorousread)

The idea of 'designing biology' -- once science fiction -- has over the last 20 years become just... science. In this episode, a16z bio general partner Vijay Pande with Hanne Tidnam talk all about the field of synthetic biology with James J. Collins, professor of bioengineering at MIT. Collins, whose work in synthetic biology and systems biology pioneered the field, has also launched a number of companies and received numerous awards and honors (including a MacArthur "Genius" Award, an NIH Director's Pioneer Award, and Sanofi-Institut Pasteur Award).

This wide-ranging conversation about the birth of synthetic biology covers everything from the founding story of the discipline to what "engineering and designing" biology really looks like in action -- when instead of engineering electrons, you are engineering toggle switches for genes -- to the disciplinary differences (and synergies) between how biologists and engineers see the world. What are the engineering and design principles, techniques, approaches that work best when applied to science? How does building a company in this new space look different, in terms of platforms and products? And how is this new field changing education in science, all the way down to kits that allow you to play with the machinery of a cell... at home... and even in middle school?

a16z Podcast: All About Synthetic Biology

When you build a model for natural language processing (NLP), such as a recurrent neural network, it helps a ton if you’re not starting from zero. In other words, if you can draw upon other datasets for building your understanding of word meanings, and then use your training dataset just for subject-specific refinements, you’ll get farther than just using your training dataset for everything. This idea of starting with some pre-trained resources has an analogue in computer vision, where initializations from ImageNet used for the first few layers of a CNN have become the new standard. There’s a similar progression under way in NLP, where simple(r) embeddings like word2vec are giving way to more advanced pre-processing methods that aim to capture more sophisticated understanding of word meanings, contexts, language structure, and more.

Relevant links:
https://thegradient.pub/nlp-imagenet/

Pre-training language models for natural language processing problems

with Susannah Fox (@susannahfox), Anil Sethi (@anilsethiusa / @ciitizencorp), Vijay Pande (@vijaypande), and Sonal Chokshi (@smc90) 

The problem of "dark data" in healthcare isn't just a feel-good empowerment thing, but a structural issue that leads to miscommunication and extra friction, different players in the entire healthcare system not being able to collaborate with each other, and just major missed opportunities all round. And yes, it also leads to lack of empowerment for patients, not to mention doctors too (who often have less than 30 minutes on site to do their jobs).

But we already know all that. What's not clear is WHY and HOW is this the case, when the very point of HIPAA -- the Health Insurance Portability and Accountability Act (of 1996!) -- is to make data portable, not private. That is, IF patients know to ask for it... and can easily get it. So what if we could have a sort of permissioned "permissionless innovation" for healthcare data, not only bringing all that dark data to light, but more importantly -- borrowing from the history of internet innovation -- letting all sorts of expected and unexpected uses be built on top as a result? What happens when data and entities can talk to each other (à la APIs) through patients at the center of the circle of data?

From the Dr. Google problem (or opportunity!) to clinical trials and even the opioid crisis, we -- Susannah Fox (former CTO of the U.S. Department of Health and Human Services); Anil Sethi (CEO and founder of Ciitizen); and a16z bio general partner Vijay Pande; in conversation with Sonal Chokshi -- explore all this and more in this episode of the a16z Podcast. Let there be light!

a16z Podcast: Dark Data in Healthcare

When data scientists use a linear regression to look for causal relationships between a treatment and an outcome, what they’re usually finding is the so-called average treatment effect. In other words, on average, here’s what the treatment does in terms of making a certain outcome more or less likely to happen. But there’s more to life than averages: sometimes the relationship works one way in some cases, and another way in other cases, such that the average isn’t giving you the whole story. In that case, you want to start thinking about heterogeneous treatment effects, and this is the podcast episode for you.

Relevant links:
https://eng.uber.com/analyzing-experiment-outcomes/
https://multithreaded.stitchfix.com/blog/2018/11/08/bandits/
https://www.locallyoptimistic.com/post/against-ab-tests/

Heterogeneous Treatment Effects

Veterinary oncology can inform human oncology, and vice versa -- providing a better model for looking at drug performance, interrelationships, and more. Especially when you add in data (there's no "doggy HIPAA!") and networks to get a "living laboratory at scale".

Or so argues Amy Abernethy (Chief Medical and Chief Scientific Officer at Flatiron Health and advisor to One Health), who was recently named the new Principal Deputy Commissioner of the FDA, pending ethics clearance; and Christina Lopes, CEO and co-founder of One Health; in conversation with a16z bio general partner Jorge Conde. Dogs -- as a species, as pets, as companions, as family members -- evolved alongside humans, so are actually more similar to us... not just genetically and in terms of the biologic pathways that may cause cancer, but also in exposure to similar environmental factors as well.

But what does this all mean when it comes to thinking about real-world evidence in science, human clinical trials, and more broadly, building a bio company? How can product designers -- of all kinds -- backwards-architect their product roadmap for data network effects? And how can bio founders keep both a big-picture roadmap in mind while also focusing on specific milestones, and while working across unconnected disciplines as well? We cover all this and more in this special episode of the a16z Podcast, recorded during the recent J.P.M. healthcare conference in San Francisco.

---

The views expressed here are those of the individual AH Capital Management, L.L.C. (“a16z”) personnel quoted and are not the views of a16z or its affiliates. Certain information contained in here has been obtained from third-party sources, including from portfolio companies of funds managed by a16z. While taken from sources believed to be reliable, a16z has not independently verified such information and makes no representations about the enduring accuracy of the information or its appropriateness for a given situation.
 
This content is provided for informational purposes only, and should not be relied upon as legal, business, investment, or tax advice. You should consult your own advisers as to those matters. References to any securities or digital assets are for illustrative purposes only, and do not constitute an investment recommendation or offer to provide investment advisory services. Furthermore, this content is not directed at nor intended for use by any investors or prospective investors, and may not under any circumstances be relied upon when making a decision to invest in any fund managed by a16z. (An offering to invest in an a16z fund will be made only by the private placement memorandum, subscription agreement, and other relevant documentation of any such fund and should be read in their entirety.) Any investments or portfolio companies mentioned, referred to, or described are not representative of all investments in vehicles managed by a16z, and there can be no assurance that the investments will be profitable or that other investments made in the future will have similar characteristics or results. A list of investments made by funds managed by Andreessen Horowitz (excluding investments and certain publicly traded cryptocurrencies/ digital assets for which the issuer has not provided permission for a16z to disclose publicly) is available at https://a16z.com/investments/.

Charts and graphs provided within are for informational purposes solely and should not be relied upon when making any investment decision. Past performance is not indicative of future results. The content speaks only as of the date indicated. Any projections, estimates, forecasts, targets, prospects, and/or opinions expressed in these materials are subject to change without notice and may differ or be contrary to opinions expressed by others. Please see https://a16z.com/disclosures for additional important information.

a16z Podcast: Connecting Hearts, Bodies, and Networks to Cure Cancer

Linear regression is a great tool if you want to make predictions about the mean value that an outcome will have given certain values for the inputs. But what if you want to predict the median? Or the 10th percentile? Or the 90th percentile. You need quantile regression, which is similar to ordinary least squares regression in some ways but with some really interesting twists that make it unique. This week, we’ll go over the concept of quantile regression, and also a bit about how it works and when you might use it.

Relevant links:
https://www.aeaweb.org/articles?id=10.1257/jep.15.4.143
https://eng.uber.com/analyzing-experiment-outcomes/

Quantile Regression

Ordinary least squares (OLS) is often used synonymously with linear regression. If you’re a data scientist, machine learner, or statistician, you bump into it daily. If you haven’t had the opportunity to build up your understanding from the foundations, though, listen up: there are a number of assumptions underlying OLS that you should know and love. They’re interesting, force you to think about data and statistics, and help you know when you’re out of “good” OLS territory and into places where you could run into trouble.

The Assumptions of Ordinary Least Squares

K Nearest Neighbors is an algorithm with secrets. On one hand, the algorithm itself is as straightforward as possible: find the labeled points nearest the point that you need to predict, and make a prediction that’s the average of their answers. On the other hand, what does “nearest” mean when you’re dealing with complex data? How do you decide whether a man and a woman of the same age are “nearer” to each other than two women several years apart? What if you convert all your monetary columns from dollars to cents, your distances from miles to nanometers, your weights from pounds to kilograms? Can your definition of “nearest” hold up under these types of transformations? We’re discussing all this, and more, in this week’s episode.

K Nearest Neighbors

with George Church (@geochurch) and Jorge Conde (@JorgeCondeBio)

Renowned scientist George Church is known for his groundbreaking work and methods used for the first genome sequence, and for his work in genome editing, writing & recoding -- in fact, Church’s innovations have become an essential building block for most of the DNA sequencing methods and companies we see today. In this conversation, a16z bio general partner Jorge Conde -- who also founded a company with Church out of the George Church Lab -- take us on a wild journey into the scientist’s mind and work, starting with what the leading pioneer in the space makes of where we are today with CRISPR (especially given recent news about CRISPR babies in China), to the broader implications of all of this on a cultural level, and finally to what it really takes to go from science fiction, to lab, to reality.

a16z Podcast: What's in the Water at the George Church Lab?

In this episode of the a16z Podcast -- which originally aired as a video on YouTube -- general partner Alex Rampell (and former fintech entrepreneur as the CEO and co-founder of TrialPay) talks with operating partner Frank Chen about the quickly changing fintech landscape and, even more importantly, why the landscape is changing now.

Should the incumbents be nervous? About what, exactly? And most importantly, what should big companies do about all of this change? But the conversation from both sides of the table begins from the perspective of the hungry and fast fintech startup sharing lessons learned, and then moves to more concrete advice for the execs in the hot seat at established companies.

---

The views expressed here are those of the individual AH Capital Management, L.L.C. (“a16z”) personnel quoted and are not the views of a16z or its affiliates.

This content is provided for informational purposes only, and should not be relied upon as legal, business, investment, or tax advice. You should consult your own advisers as to those matters. References to any securities or digital assets are for illustrative purposes only and do not constitute an investment recommendation or offer to provide investment advisory services. Furthermore, this content is not directed at nor intended for use by any investor or prospective investor, and may not under any circumstances be relied upon when making a decision to invest in any fund managed by a16z. (An offering to invest in an a16z fund will be made only by the private placement memorandum, subscription agreement, and other relevant documentation of any such fund which should be read in their entirety.)

Past performance is not indicative of future results. Any charts and graphs provided within are for informational purposes solely and should not be relied upon when making any investment decision. 

Please see https://a16z.com/disclosures for additional important information.

a16z Podcast: Fintech for Startups and Incumbents

It’s not news that data scientists are expected to be capable in many different areas (writing software, designing experiments, analyzing data, talking to non-technical stakeholders). One thing that has been changing, though, as the field becomes a bit older and more mature, is our ideas about what data scientists should focus on to stay relevant. Should they specialize in a particular area (if so, which one)? Should they instead stay general and work across many different areas? In either case, what are the costs and benefits?

This question has prompted a number of think pieces lately, which are sometimes advocating for specializing, and sometimes pointing out the benefits of generalists. In short, if you’re trying to figure out what to actually do, you might be hearing some conflicting opinions. In this episode, we break apart the arguments both ways, and maybe (hopefully?) reach a little resolution about where to go from here.

The Great Data Science Specialist vs. Generalist Debate

We often hear from folks wondering what advice we can give them as they search for their first job in data science. What does a hiring manager look for? Should someone focus on taking classes online, doing a bootcamp, reading books, something else? How can they stand out in a crowd? 

There’s no single answer, because so much depends on the person asking in the first place, but that doesn’t stop us from giving some perspective. So in this episode we’re sharing that advice out more widely, so hopefully more of you can benefit from it.

Advice to those trying to get a first job in data science

Open source software is ubiquitous throughout data science, and enables the work of nearly every data scientist in some way or another. Open source projects, however, are disproportionately maintained by a small number of individuals, some of whom are institutionally supported, but many of whom do this maintenance on a purely volunteer basis. The health of the data science ecosystem depends on the support of open source projects, on an individual and institutional level.

https://hdsr.mitpress.mit.edu/pub/xsrt4zs2/release/2

The Data Science Open Source Ecosystem

This is a re-release of an episode that originally ran in October 2019.

If you’re trying to manage a project that serves up analytics data for a few very distinct uses, you’d be wise to consider having custom solutions for each use case that are optimized for the needs and constraints of that use cases. You also wouldn’t be YouTube, which found themselves with this problem (gigantic data needs and several very different use cases of what they needed to do with that data) and went a different way: they built one analytics data system to serve them all. Procella, the system they built, is the topic of our episode today: by deconstructing the system, we dig into the four motivating uses of this system, the complexity they had to introduce to service all four uses simultaneously, and the impressive engineering that has to go into building something that “just works.”

Procella: YouTube's super-system for analytics data storage

A few weeks ago, we put out a call for data scientists interested in issues of race and racism, or people studying how those topics can be studied with data science methods, should get in touch to come talk to our audience about their work. This week we’re excited to bring on Todd Hendricks, Bay Area data scientist and a volunteer who reached out to tell us about his studies with the Stanford Open Policing dataset.

A Data Science Take on Open Policing Data

The data science and artificial intelligence community has made amazing strides in the past few years to algorithmically automate portions of the healthcare process. This episode looks at two computer vision algorithms, one that diagnoses diabetic retinopathy and another that classifies liver cancer, and asks the question—are patients now getting better care, and achieving better outcomes, with these algorithms in the mix? The answer isn’t no, exactly, but it’s not a resounding yes, because these algorithms interact with a very complex system (the healthcare system) and other shortcomings of that system are proving hard to automate away. Getting a faster diagnosis from an image might not be an improvement if the image is now harder to capture (because of strict data quality requirements associated with the algorithm that wouldn’t stop a human doing the same job). Likewise, an algorithm getting a prediction mostly correct might not be an overall benefit if it introduces more dramatic failures when the prediction happens to be wrong. For every data scientist whose work is deployed into some kind of product, and is being used to solve real-world problems, these papers underscore how important and difficult it is to consider all the context around those problems.

A Reality Check on AI-Driven Medical Assistants

All good things must come to an end, including this podcast. This is the last episode we plan to release, and it doesn’t cover data science—it’s mostly reminiscing, thanking our wonderful audience (that’s you!), and marveling at how this thing that started out as a side project grew into a huge part of our lives for over 5 years.

It’s been a ride, and a real pleasure and privilege to talk to you each week. Thanks, best wishes, and good night!

—Katie and Ben