Algorithms, the Internet, STEAM, and the Continuing Struggle for Justice

Sometimes when we say an algorithm discriminates, we mean it can tell the difference between two types of items.  But in this episode, we'll talk about another, more troublesome side to discrimination: algorithms can be... racist?  Sexist?  Ageist?  Yes to all of the above.  It's an important thing to be aware of, especially when doing people-centered data science.  We'll discuss how and why this happens, and what solutions are out there (or not).

Relevant Links:
http://www.nytimes.com/2015/07/10/upshot/when-algorithms-discriminate.html
http://techcrunch.com/2015/08/02/machine-learning-and-human-bias-an-uneasy-pair/
http://www.sciencefriday.com/segments/why-machines-discriminate-and-how-to-fix-them/
https://medium.com/@geomblog/when-an-algorithm-isn-t-2b9fe01b9bb5#.auxqi5srz




Discriminatory Algorithms

Machines are making a lot of decisions that used to be made by humans. Computer-driven algorithms can determine everything from which friends we keep up with on Facebook, to the prices that we pay for goods and services, to which neighborhoods get more attention from political candidates. But how do we hold machines accountable for the decisions they make? What’s at stake when it comes to algorithmic injustice and accountability?

Algorithmic Accountability: When Machines Make Our Decisions


Algorithms have become a powerful force in the world, but for all the impressive good they do, they sometimes show some worrying tendencies. Algorithms that discriminate are a problem that nobody's found a solution for yet. This week, we discuss why some algorithms appear to be racist, and whether there's anything that can be done about it.

The Techdirt Podcast

Is Your Algorithm Racist?

In this week's podcast InfoQ’s editor-in-chief Charles Humble talks to Data Scientist Cathy O’Neil. O'Neil is the author of the blog mathbabe.org. She was the former Director of the Lede Program in Data Practices at Columbia University Graduate School of Journalism, Tow Center and was employed as Data Science Consultant at Johnson Research Labs. O'Neil earned a mathematics Ph.D. from Harvard University. Topics discussed include her book “Weapons of Math Destruction,” predictive policing models, the teacher value added model, approaches to auditing algorithms and whether government regulation of the field is needed.

Why listen to this podcast:

 - There is a class of pernicious big data algorithms that are increasingly controlling society but are not open to scrutiny.
 - Flawed data can result in an algorithm that is, for instance, racist and sexist.  For example, the data used in predictive policing models is racist. But people tend to be overly trusting of algorithms because they are mathematical.
 - Data scientists have to make ethical decisions even if they don’t acknowledge it. Often problems stem from an abdication of responsibility.
 - Auditing for algorithms is still a very young field with ongoing academic research exploring approaches.
 - Government regulation of the industry may well be required.

Notes and links can be found on http://bit.ly/2eYVb9q

Weapons of math destruction

0m:43s - The central thesis of the book is that whilst not all algorithms are bad, there is a class of pernicious big data algorithms that are increasingly controlling society.
1m:32s - The classes of algorithm that O'Neil is concerned about - the weapons of math destruction - have three characteristics: they are widespread and impact on important decisions like whether someone can go to college or get a job, they are somehow secret so that the people who are being targeted don’t know they are being scored or don’t understand how their score is computed; and the third characteristic is they are destructive - they ruin lives.
2m:51s - These characteristics undermine the original intention of the algorithm, which is often trying to solve big society problems with the help of data.

More on this:

Quick scan our curated show notes on InfoQ. http://bit.ly/2eYVb9q

You can also subscribe to the InfoQ newsletter to receive weekly updates on the hottest topics from professional software development. bit.ly/24x3IVq

InfoQ Podcast

Show Notes

Cathy O'Neil on Pernicious Machine Learning Algorithms and How to Audit Them

When we covered the Word2Vec algorithm for embedding words, we mentioned parenthetically that the word embeddings it produces can sometimes be a little bit less than ideal--in particular, gender bias from our society can creep into the embeddings and give results that are sexist. For example, occupational words like "doctor" and "nurse" are more highly aligned with "man" or "woman," which can create problems because these word embeddings are used in algorithms that help people find information or make decisions. However, a group of researchers has released a new paper detailing ways to de-bias the embeddings, so we retain gender info that's not particularly problematic (for example, "king" vs. "queen") while correcting bias.

Debiasing Word Embeddings

Writer Sara Wachter-Boettcher explains how racial and sexual bias finds its way from society into digital products (and back into society) as a result of the tech industry's unwillingness to examine its own values, and they way their work engages with and shapes society beyond the mostly white, male corporate campuses of Silicon Valley.

Sara is author of "Technically Wrong: Sexist Apps, Biased Algorithms, and Other Threats of Toxic Tech" from W.W. Norton: http://books.wwnorton.com/books/978-0-393-63463-1/

Social inequality is coded deep within the devices of Silicon Valley.

This is the official channel of the Aspen Ideas Festival, the nation's premier public-gathering place where leaders from around the globe and across many disciplines engage in deep and inquisitive discussion of the ideas and issues that both shape our lives and challenge our times. Some 350 presenters, 200 sessions, and 3000 attendees comprise the annual Festival, launched in 2005, on the Aspen Institute's campus in Aspen, Colorado.

VENUS PELLAGATTI XTRAVAGANZA
ROBERT BROOKS
SAM NORDQUIST
AARON BUSHNELL
ELIJAH MCCLAIN
ANTEO ZAMBONI 
SUMMER TAYLOR
AHMED EREKAT
DOMINIQUE REM'MIE FELLS
SARAH GROSSMAN
JAMES SCURLOCK
DAVID MCATEE
EYAD AL-HALAQ
HECTOR GARCIA MENDOZA
GEORGE FLOYD
BREONNA TAYLOR
VICENTE RAMON GUERRERO
MARCI WISE
JOSEPH ROSENBAUM
ANTHONY HUBER
RIAH MILTON
TONY MCDADE 
EUGUI ROY
AHMAUD ARBERY
ASHLYNN LISBY
OLUWATOYIN SALAU 
VICTORIA SIMS
JOHANNA METZGER
ZEZICO GUAJAJARA
ISABEL CABANILLAS
DAVID GLEN WARD
TONY MCDADE
TIFFANY M. MOFIELD
KENNY RICHIE MARTY "MYLEZ" MITCHELL
NÉHÉMIE JOSEPH
NICOLA SACCO
BARTOLOMEO VANZETTI
ANNA LOPIZZO
ATATIANA KOQUICE JEFFERSON
CARY BALL JR.
ROXANA HERNANDEZ
JAKELIN CAAL MAQUIN
FELIPE ALONZO-GOMEZ
DREASJON "SEAN" REED
MCHALE ROSE
JANA LANGE
WILLIE MCCOY
MARIEE JUAREZ
OSMAR EPIFANIO GONZALEZ-GADBA
JEANCARLO JIMENEZ-JOSEPH
HUY CHI TRAN
KALIEF BROWDER
MI'CHANCE DUNLAP-GITTENS
SERGIO ALONSO LOPEZ
KAMYAR SAMIMI
JOSHUA BROWN
GEMMEL MOORE
TIMOTHY DEAN
SAHAR KHODAYARI
DE'VON BAILEY
TONY TIMPA
MARIA DEL PILAR HURTADO
EMILSEN MANYOMA
MOISÉS MOSQUERA MORENO
TEMISTOCLES MACHADO 
ELEAZAR TEQUÍA VITUCAY
HECTOR WILLIAM MINA
ELIJAH AL-AMIN
NINA POP
JENNIFER LAUDE
LAYLEEN POLANCO
PAMELA TURNER
CHYNAL LINDSEY
JOYCE FIENBERG
RICHARD GOTTFRIED
ROSE MALLINGER 
JERRY RABINOWITZ 
CECIL ROSENTHAL 
DAVID ROSENTHAL 
BERNICE SIMON 
SYLVAN SIMON 
DANIEL STEIN 
MOHAMMED ABU KHUDAIR
MOHAMMAD RAMIZ BAKR
AHED ATEF BAKR
ZAKARIYA AHED BAKR
ISMAIL MAHMOUD BAKR
MELVIN WAX 
TIMOTHY CAUGHMAN
LARNELL BRUCE JR
OSCAR JOSE FIGUERA
EMANTIC “EJ” BRADFORD, JR.
TENIYA JONES 
THURMAN BLEVINS
ALEX NIETO
WALTER LUEBCKE
ARMANDO FRANK
RACHEL ALIENE CORRIE
ANTWON ROSE, JR.
LINUS PHILLIP
DIANTE BUTCHIE YARBER
SAHEED VASSEL
DANA MARTIN
ASHANTI CARMON
CLAIRE LEGATO
RACHEL CORRIE
MUHLAYSIA BOOKER
KENNETH CHAMBERLAIN SR.
ADAMA TRAORE
MICHELLE "TAMIKA" WASHINGTON
JAQUARRIUS HOLLAND
DANNY RAY THOMAS
STEPHON CLARK
MARIELLE FRANCO
TOVEET RADCLIFFE
HEATHER HEYER
FRED HAMPTON
LAVENA LYNN JOHNSON
PHILANDO CASTILE
TAMIR RICE
DANROY HENRY
TRAYVON MARTIN
ERIC GARNER
MICHAEL BROWN
WALTER SCOTT
VIOLA LIUZZO
CLAUDIA PATRICIA GOMEZ GONZALEZ
RICKY JOHN BEST
TALIESIN MYRDDIN NAMKAI MECHE
FREDDIE GRAY 
ALTON STERLING
JORDAN EDWARDS
OSCAR RAMIREZ, JR.
LIGE DANIELS
JESSE WASHINGTON
JOSE ANTONIO ELENA RODRIGUEZ
SAM DUBOSE
HENRY SMITH
JOHN HARTFIELD
ELTON MITCHELL
T.A. ALLEN
THOMAS DEVERT
JOE SPINNER JOHNSON
LUTHER HOLBERT
JESSE THORNTON
WILLIAM LITTLE
JEFF BROWN
KEITH BOWEN
GENERAL LEE
THOMAS MILES
GEORGE DORSEY
DOROTHY DORSEY
TARIKA WILSON
DEAUNTA FARROW
SAM HOSE
PATRICK HARMON
SEAN BELL 
ANTHONY LAMAR SMITH
HENRY GLOVER
RONALD MADISON
TIMOTHY STANSBURY
ALBERTA SPRUILL
DEBORAH DANNER
OUSMANE ZONGO
ORLANDO BARLOW
PRINCE JONES
RONALD BEASLEY
EARL MURRAY
PATRICK DORISMOND
MALCOLM FERGUSON
JONATHAN FERRELL
DONTRE HAMILTON
JOHN CRAWFORD III
EZELL FORD
AIYANA JONES
OSCAR GRANT
DANTE PARKER
TANISHA ANDERSON
TYREE WOODSON
VICTOR WHITE III
YVETTE SMITH
MCKENZIE COCHRAN
JORDAN BAKER
ANDY LOPEZ
MIRIAM CAREY
DEION FLUDD
KIMANI GRAY
REYNALDO QUEVAS
SHANTEL DAVIS
TAMON ROBINSON 
SHEREECE FRANCIS
REKIA BOYD 
TROY DAVIS 
AMADOU DIALLO
RAMARLEY GRAHAM
LAQUAN MCDONALD
KENDRICK JOHNSON
LAVENA JOHNSON
MEDGER EVERS
ANDREW GOODMAN 
MICHAEL SCHWERNER 
JAMES CHANEY 
JOHN EARL REESE 
WILLIE EDWARDS JR 
MACK CHARLES PARKER 
MARSH COOK
JOHN BROWN
HERBERT LEE 
ROMAN DUCKSWORTH JR 
WILLIAM LEWIS MOORE 
ADDIE MAE COLLINS 
DENISE MCNAIR 
CAROLE ROBERTSON 
CYNTHIA WESLEY 
VIRGIL WARE 
HENRY HEZEKIAH DEE 
CHARLES MOORE 
LEMUEL PENN 
JIMMIE LEE JACKSON 
JAMES REEB 
ONEAL MOORE 
WILLIE BREWSTER
JONATHAN DANIELS 
VERNON DAHMER 
BEN CHESTER WHITE 
WHARLEST JACKSON
BENJAMIN BROWN 
SAMUEL HAMMOND JR 
DELANO MIDDLETON 
HENRY EZEKIAL SMITH 
NAT TURNER 
ANDY BURKE 
JOHNSON WHITTAKER 
ROBERT ROBINSON 
FRED ALEXANDER 
ROBERT TRUETT
L.B. REED 
CLINTON BRIGGS 
FRANK LIVINGSTON 
BUD JOHNSON 
LUCIUS MCCARTY 
POWELL GREEN 
EARNEST MCGOWAN 
LEE SNELL 
PRECIOUS GRANT 
HUGH JOHNSON
J.C. FARMER 
EUGENE BELLS 
TIMOTHY HOOD 
JAMES STEPHENSON 
ALONZA BROOKS 
DEJUAN GUILLORY 
ROCRAST MACK JR. 
JUSTIN THOMPSON 
DARRIUS STEWART