Algorithms, the Internet, STEAM, and the Continuing Struggle for Justice

Sometimes when we say an algorithm discriminates, we mean it can tell the difference between two types of items.  But in this episode, we'll talk about another, more troublesome side to discrimination: algorithms can be... racist?  Sexist?  Ageist?  Yes to all of the above.  It's an important thing to be aware of, especially when doing people-centered data science.  We'll discuss how and why this happens, and what solutions are out there (or not).

Relevant Links:
http://www.nytimes.com/2015/07/10/upshot/when-algorithms-discriminate.html
http://techcrunch.com/2015/08/02/machine-learning-and-human-bias-an-uneasy-pair/
http://www.sciencefriday.com/segments/why-machines-discriminate-and-how-to-fix-them/
https://medium.com/@geomblog/when-an-algorithm-isn-t-2b9fe01b9bb5#.auxqi5srz




Discriminatory Algorithms

Josh sits down with Jadzia Butler, Privacy Surveillance & Security Fellow for CDT. Jadzia and Josh delve into the the balance between government security, the needs of corporations and individual rights.
Read the transcript: https://www.ipvanish.com/podcasts/

Technology and Law: The Transition Between Physical & Digital - Jadzia Butler, CDT


Algorithms have become a powerful force in the world, but for all the impressive good they do, they sometimes show some worrying tendencies. Algorithms that discriminate are a problem that nobody's found a solution for yet. This week, we discuss why some algorithms appear to be racist, and whether there's anything that can be done about it.

The Techdirt Podcast

Is Your Algorithm Racist?

In this week's podcast InfoQ’s editor-in-chief Charles Humble talks to Data Scientist Cathy O’Neil. O'Neil is the author of the blog mathbabe.org. She was the former Director of the Lede Program in Data Practices at Columbia University Graduate School of Journalism, Tow Center and was employed as Data Science Consultant at Johnson Research Labs. O'Neil earned a mathematics Ph.D. from Harvard University. Topics discussed include her book “Weapons of Math Destruction,” predictive policing models, the teacher value added model, approaches to auditing algorithms and whether government regulation of the field is needed.

Why listen to this podcast:

 - There is a class of pernicious big data algorithms that are increasingly controlling society but are not open to scrutiny.
 - Flawed data can result in an algorithm that is, for instance, racist and sexist.  For example, the data used in predictive policing models is racist. But people tend to be overly trusting of algorithms because they are mathematical.
 - Data scientists have to make ethical decisions even if they don’t acknowledge it. Often problems stem from an abdication of responsibility.
 - Auditing for algorithms is still a very young field with ongoing academic research exploring approaches.
 - Government regulation of the industry may well be required.

Notes and links can be found on http://bit.ly/2eYVb9q

Weapons of math destruction

0m:43s - The central thesis of the book is that whilst not all algorithms are bad, there is a class of pernicious big data algorithms that are increasingly controlling society.
1m:32s - The classes of algorithm that O'Neil is concerned about - the weapons of math destruction - have three characteristics: they are widespread and impact on important decisions like whether someone can go to college or get a job, they are somehow secret so that the people who are being targeted don’t know they are being scored or don’t understand how their score is computed; and the third characteristic is they are destructive - they ruin lives.
2m:51s - These characteristics undermine the original intention of the algorithm, which is often trying to solve big society problems with the help of data.

More on this:

Quick scan our curated show notes on InfoQ. http://bit.ly/2eYVb9q

You can also subscribe to the InfoQ newsletter to receive weekly updates on the hottest topics from professional software development. bit.ly/24x3IVq

InfoQ Podcast

Show Notes

Cathy O'Neil on Pernicious Machine Learning Algorithms and How to Audit Them

Released on December 8, Episode I features a conversation between host Kirby Ferguson and journalist and science fiction author Cory Doctorow as they examine where copyright is working, where it isn’t, and what changes are needed to catch up to the technological innovations of the digital age.

What Star Wars Can Teach Us About Copyright

Is math biased? That seems like a silly question - in math there is right and wrong, so how could it possibly be biased. But humans use math and develop mathematical models and algorithms, and well, humans can certainly be biased. So, well, maybe math can in fact be biased. 

In her new book, Weapons of Math Destruction, Cathy O’Neil explores how big data is increasing inequality and threatening democracy - that is actually the subtitle for the book.

More on Cathy: https://mathbabe.org/
More info on our host, @ brianwez: bit.ly/cdtbrian

Attribution: sounds used from Psykophobia, Taira Komori, BenKoning, Zabuhailo, bloomypetal, guitarguy1985, bmusic92, and offthesky of freesound.org.

Weapons of Math Destruction –– Talking Tech w/ Cathy O’Neil

GPD's Executive Director interviews Stefaan Verhulst, co-founder and Chief Research and Development Officer of the Governance Laboratory at New York University.

EPISODE 1: Is policymaking stuck in the 19th century? (S1)

GPD's executive director Charles Bradley talks to Emma Llansó, Director of the Free Expression Project at the Center for Democracy and Technology (CDT) about the role of intermediaries like Facebook and Twitter, and how growing pressure on them to remove user content might affect our freedom of expression.

EPISODE 6: Who gets to decide when online content should be removed? (S1)

When we covered the Word2Vec algorithm for embedding words, we mentioned parenthetically that the word embeddings it produces can sometimes be a little bit less than ideal--in particular, gender bias from our society can creep into the embeddings and give results that are sexist. For example, occupational words like "doctor" and "nurse" are more highly aligned with "man" or "woman," which can create problems because these word embeddings are used in algorithms that help people find information or make decisions. However, a group of researchers has released a new paper detailing ways to de-bias the embeddings, so we retain gender info that's not particularly problematic (for example, "king" vs. "queen") while correcting bias.

Debiasing Word Embeddings

Writer Sara Wachter-Boettcher explains how racial and sexual bias finds its way from society into digital products (and back into society) as a result of the tech industry's unwillingness to examine its own values, and they way their work engages with and shapes society beyond the mostly white, male corporate campuses of Silicon Valley.

Sara is author of "Technically Wrong: Sexist Apps, Biased Algorithms, and Other Threats of Toxic Tech" from W.W. Norton: http://books.wwnorton.com/books/978-0-393-63463-1/

Social inequality is coded deep within the devices of Silicon Valley.

Tech Talks are a podcast imagined and created by the folks over at the Center for Democracy and Technology (CDT). We're a 501(c)3 nonpartisan nonprofit organization that fights to put democracy and human rights at the center of the digital revolution. We work to promote democratic values by shaping technology policy and architecture, with a focus on equity and justice. #techtalkcdt 

Support CDT's work with a donation: https://cdt.org/techtalk/

VENUS PELLAGATTI XTRAVAGANZA
ROBERT BROOKS
SAM NORDQUIST
AARON BUSHNELL
ELIJAH MCCLAIN
ANTEO ZAMBONI 
SUMMER TAYLOR
AHMED EREKAT
DOMINIQUE REM'MIE FELLS
SARAH GROSSMAN
JAMES SCURLOCK
DAVID MCATEE
EYAD AL-HALAQ
HECTOR GARCIA MENDOZA
GEORGE FLOYD
BREONNA TAYLOR
VICENTE RAMON GUERRERO
MARCI WISE
JOSEPH ROSENBAUM
ANTHONY HUBER
RIAH MILTON
TONY MCDADE 
EUGUI ROY
AHMAUD ARBERY
ASHLYNN LISBY
OLUWATOYIN SALAU 
VICTORIA SIMS
JOHANNA METZGER
ZEZICO GUAJAJARA
ISABEL CABANILLAS
DAVID GLEN WARD
TONY MCDADE
TIFFANY M. MOFIELD
KENNY RICHIE MARTY "MYLEZ" MITCHELL
NÉHÉMIE JOSEPH
NICOLA SACCO
BARTOLOMEO VANZETTI
ANNA LOPIZZO
ATATIANA KOQUICE JEFFERSON
CARY BALL JR.
ROXANA HERNANDEZ
JAKELIN CAAL MAQUIN
FELIPE ALONZO-GOMEZ
DREASJON "SEAN" REED
MCHALE ROSE
JANA LANGE
WILLIE MCCOY
MARIEE JUAREZ
OSMAR EPIFANIO GONZALEZ-GADBA
JEANCARLO JIMENEZ-JOSEPH
HUY CHI TRAN
KALIEF BROWDER
MI'CHANCE DUNLAP-GITTENS
SERGIO ALONSO LOPEZ
KAMYAR SAMIMI
JOSHUA BROWN
GEMMEL MOORE
TIMOTHY DEAN
SAHAR KHODAYARI
DE'VON BAILEY
TONY TIMPA
MARIA DEL PILAR HURTADO
EMILSEN MANYOMA
MOISÉS MOSQUERA MORENO
TEMISTOCLES MACHADO 
ELEAZAR TEQUÍA VITUCAY
HECTOR WILLIAM MINA
ELIJAH AL-AMIN
NINA POP
JENNIFER LAUDE
LAYLEEN POLANCO
PAMELA TURNER
CHYNAL LINDSEY
JOYCE FIENBERG
RICHARD GOTTFRIED
ROSE MALLINGER 
JERRY RABINOWITZ 
CECIL ROSENTHAL 
DAVID ROSENTHAL 
BERNICE SIMON 
SYLVAN SIMON 
DANIEL STEIN 
MOHAMMED ABU KHUDAIR
MOHAMMAD RAMIZ BAKR
AHED ATEF BAKR
ZAKARIYA AHED BAKR
ISMAIL MAHMOUD BAKR
MELVIN WAX 
TIMOTHY CAUGHMAN
LARNELL BRUCE JR
OSCAR JOSE FIGUERA
EMANTIC “EJ” BRADFORD, JR.
TENIYA JONES 
THURMAN BLEVINS
ALEX NIETO
WALTER LUEBCKE
ARMANDO FRANK
RACHEL ALIENE CORRIE
ANTWON ROSE, JR.
LINUS PHILLIP
DIANTE BUTCHIE YARBER
SAHEED VASSEL
DANA MARTIN
ASHANTI CARMON
CLAIRE LEGATO
RACHEL CORRIE
MUHLAYSIA BOOKER
KENNETH CHAMBERLAIN SR.
ADAMA TRAORE
MICHELLE "TAMIKA" WASHINGTON
JAQUARRIUS HOLLAND
DANNY RAY THOMAS
STEPHON CLARK
MARIELLE FRANCO
TOVEET RADCLIFFE
HEATHER HEYER
FRED HAMPTON
LAVENA LYNN JOHNSON
PHILANDO CASTILE
TAMIR RICE
DANROY HENRY
TRAYVON MARTIN
ERIC GARNER
MICHAEL BROWN
WALTER SCOTT
VIOLA LIUZZO
CLAUDIA PATRICIA GOMEZ GONZALEZ
RICKY JOHN BEST
TALIESIN MYRDDIN NAMKAI MECHE
FREDDIE GRAY 
ALTON STERLING
JORDAN EDWARDS
OSCAR RAMIREZ, JR.
LIGE DANIELS
JESSE WASHINGTON
JOSE ANTONIO ELENA RODRIGUEZ
SAM DUBOSE
HENRY SMITH
JOHN HARTFIELD
ELTON MITCHELL
T.A. ALLEN
THOMAS DEVERT
JOE SPINNER JOHNSON
LUTHER HOLBERT
JESSE THORNTON
WILLIAM LITTLE
JEFF BROWN
KEITH BOWEN
GENERAL LEE
THOMAS MILES
GEORGE DORSEY
DOROTHY DORSEY
TARIKA WILSON
DEAUNTA FARROW
SAM HOSE
PATRICK HARMON
SEAN BELL 
ANTHONY LAMAR SMITH
HENRY GLOVER
RONALD MADISON
TIMOTHY STANSBURY
ALBERTA SPRUILL
DEBORAH DANNER
OUSMANE ZONGO
ORLANDO BARLOW
PRINCE JONES
RONALD BEASLEY
EARL MURRAY
PATRICK DORISMOND
MALCOLM FERGUSON
JONATHAN FERRELL
DONTRE HAMILTON
JOHN CRAWFORD III
EZELL FORD
AIYANA JONES
OSCAR GRANT
DANTE PARKER
TANISHA ANDERSON
TYREE WOODSON
VICTOR WHITE III
YVETTE SMITH
MCKENZIE COCHRAN
JORDAN BAKER
ANDY LOPEZ
MIRIAM CAREY
DEION FLUDD
KIMANI GRAY
REYNALDO QUEVAS
SHANTEL DAVIS
TAMON ROBINSON 
SHEREECE FRANCIS
REKIA BOYD 
TROY DAVIS 
AMADOU DIALLO
RAMARLEY GRAHAM
LAQUAN MCDONALD
KENDRICK JOHNSON
LAVENA JOHNSON
MEDGER EVERS
ANDREW GOODMAN 
MICHAEL SCHWERNER 
JAMES CHANEY 
JOHN EARL REESE 
WILLIE EDWARDS JR 
MACK CHARLES PARKER 
MARSH COOK
JOHN BROWN
HERBERT LEE 
ROMAN DUCKSWORTH JR 
WILLIAM LEWIS MOORE 
ADDIE MAE COLLINS 
DENISE MCNAIR 
CAROLE ROBERTSON 
CYNTHIA WESLEY 
VIRGIL WARE 
HENRY HEZEKIAH DEE 
CHARLES MOORE 
LEMUEL PENN 
JIMMIE LEE JACKSON 
JAMES REEB 
ONEAL MOORE 
WILLIE BREWSTER
JONATHAN DANIELS 
VERNON DAHMER 
BEN CHESTER WHITE 
WHARLEST JACKSON
BENJAMIN BROWN 
SAMUEL HAMMOND JR 
DELANO MIDDLETON 
HENRY EZEKIAL SMITH 
NAT TURNER 
ANDY BURKE 
JOHNSON WHITTAKER 
ROBERT ROBINSON 
FRED ALEXANDER 
ROBERT TRUETT
L.B. REED 
CLINTON BRIGGS 
FRANK LIVINGSTON 
BUD JOHNSON 
LUCIUS MCCARTY 
POWELL GREEN 
EARNEST MCGOWAN 
LEE SNELL 
PRECIOUS GRANT 
HUGH JOHNSON
J.C. FARMER 
EUGENE BELLS 
TIMOTHY HOOD 
JAMES STEPHENSON 
ALONZA BROOKS 
DEJUAN GUILLORY 
ROCRAST MACK JR. 
JUSTIN THOMPSON 
DARRIUS STEWART

Weapons of Math Destruction –– Talking Tech w/ Cathy O’Neil

CDT's Tech Talk

Related tracks