Organizations rely on AI to approve a loan or shape a defendant’s sentence. But the foundations upon which these intelligent systems are built are susceptible to bias. Bias from the data, from the programmer, and from a powerful company’s bottom line can snowball into unintended consequences. This is the reality AI researcher Timnit Gebru cautioned against at a RE:WIRED talk on Tuesday.
“There were companies purporting [to assess] someone’s likelihood of determining a crime again,” Gebru said. “That was terrifying for me.”
Gebru was a star engineer at Google who specialized in AI ethics. She co-led a team tasked with standing guard against algorithmic racism, sexism, and other bias. Gebru also cofounded the nonprofit Black in AI, which seeks to improve inclusion, visibility, and health of Black people in her field.
Last year, Google forced her out. But she hasn’t given up her fight to prevent unintended damage from machine learning algorithms.
Tuesday, Gebru spoke with WIRED senior writer Tom Simonite about incentives in AI research, the role of worker protections, and the vision for her planned independent institute for AI ethics and accountability. Her central point: AI needs to slow down.
“We haven’t had the time to think about how it should even be built because we’re always just putting out fires,” she said.
As an Ethiopian refugee attending public school in the Boston suburbs, Gebru was quick to pick up on America’s racial dissonance. Lectures referred to racism in the past tense, but that didn’t jibe with what she saw, Gebru told Simonite earlier this year. She has found a similar misalignment repeatedly in her tech career.
Gebru’s professional career began in hardware. But she changed course when she saw barriers to diversity and began to suspect that most AI research had the potential to bring harm to already marginalized groups.
“The confluence of that got me going in a different direction, which is to try to understand and try to limit the negative societal impacts of AI,” she said.
For two years, Gebru co-led Google’s Ethical AI team with computer scientist Margaret Mitchell. The team created tools to protect against AI mishaps for Google’s product teams. Over time, though, Gebru and Mitchell realized they were being left out of meetings and email threads.
In June 2020, the GPT-3 language model was released and displayed an ability to sometimes craft coherent prose. But Gebru's team worried about the excitement around it.
“We haven’t had the time to think about how it should even be built because we’re always just putting out fires.” TIMNIT GEBRU, AI RESEARCHER“Let’s build larger and larger and larger language models,” said Gebru, recalling the popular sentiment. “We had to be like, ‘Let’s please just stop and calm down for a second so that we can think about the pros and cons and maybe alternative ways of doing this.’”
Her team helped write a paper about the ethical implications of language models, called “On the Dangers of Stochastic Parrots: Can Language Models Be Too Big?”
Others at Google were not happy. Gebru was asked to retract the paper or remove Google employees’ names. She countered with an ask for transparency: Who had requested such harsh action and why? Neither side budged. Gebru found out from one of her direct reports that she “had resigned.”
The experience at Google reinforced in her a belief that oversight of AI’s ethics should not be left to a corporation or government.
“The incentive structure is not such that you slow down, first of all, think about how you should approach research, how you should approach AI, when it should be built, when it should not be built,” said Gebru. “I want us to be able to do AI research in a way that we think it should be done—prioritizing the voices that we think are actually being harmed.”
Since leaving Google, Gebru has been developing an independent research institute to show a new model for responsible and ethical AI research. The institute aims to answer similar questions as her Ethical AI team, without fraught incentives of private, federal, or academic research—and without ties to corporations or the Department of Defense.
“Our goal is not to make Google more money; it’s not to help the Defense Department figure out how to kill more people more efficiently,” she said.
At Tuesday’s session, Gebru said the institute will be unveiled on December 2, the anniversary of her ousting from Google. “Maybe I’ll just start celebrating this every year,” she joked.
Slowing the pace of AI might cost companies money, she said. “Either put more resources to prioritize safety or [don’t] deploy things,” she added. “And unless there is regulation that prioritizes that, it’s going to be very difficult to have all these companies, out of their own goodwill, self-regulate.”
Still, Gebru finds room for optimism. “The conversation has really shifted, and some of the people in the Biden administration working on this stuff are the right people,” she said. “I have to be hopeful. I don’t think we have other options.”
