I've been hoping to address that point, actually. Because, I get where you're coming from, and I've felt those frustrations myself. Let's just say I'm probably the last person to hop on the "do what I say or else" bandwagon.

... but like everything else, that turns out not to be an issue here. I would feel fine saying what you said. It's nothing to do with the color of her skin, or the fact that she was working on ethics rather than optimizers. Horrible employees who cause drama wherever they go are a liability and a downer, and I'll happily say that to whoever's listening. Black or white or purple, the goal is to serve the company's business interests.

But, much to my own surprise, as someone who grew up memeing on 4chan and poking fun at leftist dogma, here I am after about a year and a half of ML, wondering "Where's my harassment? I was promised harassment."

Because I feel exactly the opposite. Not only is everyone in the ML twitter scene cool, but they're some of the most open minded people I've ever met. Sure, you get some people showing up sometimes to give you a hard time, e.g. when we train danbooru: https://i.imgur.com/RMZd6mu.png and then say that the dataset is very objectifying, and so on.

But the antidote is to simply be straightforward. Make it clear you actually hear them. "Yes, I am concerned about that, but it's simply the problem domain; people use danbooru as a repository of this content. You're right that we could e.g. make a classifier to pick out the cooler looking gals and focus on those, but the challenge is simply to solve the problem at all. Once we have lovely auto-generated anime, it'll be straightforward to filter it. Come help us get there! It's fun!"

With all the horror stories I've heard, and how afraid everyone is of "the mob," I went into it expecting to "smile, and don't tell them what you're thinking." And I was really surprised to find exactly the opposite atmosphere. Everyone pretty much agrees that yeah, we have a bias problem, and that it's probably good to address that. People also seem to agree that it's ridiculous to take it too far, e.g. when OpenAI enforces a mandatory content filter that you can't turn off and isn't too choosy about what it deems harmful, and prevents you from shipping to production unless you enable it.

So I'm just scratching my head like ... "am I the baddie?" https://www.youtube.com/watch?v=hn1VxaMEjRU&ab_channel=roots...

Because I'd be a part of the problem, if you felt like you can't talk to me freely (at least in private). I don't want to foster that kind of environment. So my reaction of the idea of you "talking down a black female, AI, ethics researcher" is "Y'know, I see where you're coming from, and I was worried about the same thing, but I'm just not getting that sort of vibe at all. I was surprised too; I expected the opposite."

And once you relax about it and look around, the most interesting part to me is that you start to feel like "well... why look at whether she's a black, female, AI, ethics researcher? I should probably read her work and listen to what she's saying, and judge for myself whether it seems crazy." And when you sit down and really listen to people, and put your full mental focus on what they are saying, I find it hard to disagree with a lot of their points, simply from a logical perspective.

So you might argue "Well, you're just part of that culture then. You'd be appalled how I feel, but I'll keep that to myself." Fair. But it's so weird being in a situation where everyone is like "watch out for that mob!" and meanwhile all of the people who actually work in the field seem pretty cool to me.

Everyone feels equally lost, i.e. that we have this magical new power (ML) that we don't really know what to do with. We know it will affect society, but we don't know how it will affect society. We also don't know the best ways to guard against some of the obvious problems on the horizon.

I ran into that problem myself. Since I'm already on a ramble, I may as well lay it all out, because it really is interesting: I was training some FFHQ latent directions, trying to get a skin color working. And I ended up discovering, quite by accident, that my latent vectors were "racist." My model was generating caricatures of black people that I would not be comfortable showing. And I couldn't figure out why. Why black people? I flip the skin color to white, no problem. Flip it to asian, no problem: https://twitter.com/theshawwn/status/1184074334186414080

Flip it to black, and horrible results. (I mean "horrible" as in "this would be an especially bad idea to show anyone," rather than merely "it has some visual defects.")

The answer to this mystery was obvious, but only in hindsight. There are far fewer black people in FFHQ than whites or asians. I found a classifier, got it to identify an order of magnitude more black faces than I had before, retrained my model, and the result was instantly so much better: https://twitter.com/theshawwn/status/1209749009092493312

That experience stayed with me to this day. I think about it a lot, because it would be so easy to overlook that kind of bias when it's numerical data rather than facial data. And as far as I can tell, that's exactly the sort of ethics that Timnit has been arguing for: we need to pay more attention to bias, and unexpected ways that bias can creep in. Which seems reasonable to me.

I don't really know why I'm posting this to you, but, just in case it changes your mind, I leave it to you. I really thought I'd end up feeling every bit as pinched as you expressed, yet it's nothing like that. Felt quite the opposite. I keep re-reading http://paulgraham.com/orth.html wondering if I have orthodox privilege, or if my social group simply doesn't include people who are comfortable enough to express themselves around me, or what. Because you say that there is a massive number of people who feel exactly the opposite, and I can't help feeling curious where they are. Our discord's up to 1200 people, and they don't seem to be there. I talk to dozens of researchers, sometimes on a weekly basis, just to poke my head in and see what they're up to. They don't seem to be there either, even in private. And, picture someone who is the opposite of "leftist" in basically every way. I know a few people like that in the ML scene, and even they don't seem to be saying "whoa, this bias stuff has gone too far, and these ethics concerns are nonsense." It's the opposite. Eleuther has a dedicated ethics channel, people spend a lot of time freely debating about what the "right" ideas might look like, and so on. We also maintain internal research channels with the idea that, if people have concerns of the type you mention, they can freely express themselves there with no fear of any kind of retribution -- that's the whole point of having secret research channels. It's up to ~30 or so active researchers, and no one has brought up concerns like that.

So you see, I end up being dragged to the conclusion that, yes, the politics / activism stuff is a concern, but no, it doesn't seem to be affecting anything. We're not hearing "do this or I'll bite your face off," or something nuts like that. It's more like "Could you please listen to me for a bit? I have this experience I'd like to share." And the experiences tend to be interesting, at least to me.

So that's why I urge you to be super skeptical about the angle you mention. ("You'd be instantly taken to court of public opinion...") Dig into the situation and look for evidence of that yourself. Don't pay attention to newspapers; talk to researchers, and ask them how they feel about it. I just can't find any trace, no matter how hard I look. I feel if you also look, in a scientific way, for evidence to support your concerns, that you might not find it either.

Anyway. I related to what you were saying and just wanted to give perhaps a new way of looking at it, since it's what changed my mind. Feel free to hit me up in twitter DMs if you're looking for someone to chat with about some hard topics, since those are often the interesting ones.

> The answer to this mystery was obvious, but only in hindsight. There are far fewer black people in FFHQ than whites or asians. I found a classifier, got it to identify an order of magnitude more black faces than I had before, retrained my model, and the result was instantly so much better: https://twitter.com/theshawwn/status/1209749009092493312

Ironically, that exact observation about that exact dataset was what got Dr. Gebru so mad at Yann Le Cun in the twitter thread that people brought up. So maybe you're just lucky with who saw your tweets.

You've worked with a visual appearances dataset, lacking sufficient examples from one class of entities, and it failed to perform well for that class. You solved the problem by adding more example of that class. While the malfunction might have hd some, as of yet unquantified, real world impact in some hypothetical police face recognition system, it doesn't follow that:

a. Datasets that are not about visual appearances, are prone to the same problem and to the same degree. Perhaps the house lending datasets / systems have small race (visual appearances) issues, but large class issues. The political debate of how to handle class issues is as old as politics have been around.

b. The extent of real world impact, which depends on the actual system deployed. Perhaps a hypothetical real world system has a 1% failure rate vs .1% failure rate. Should we stop developing useful system just because they are not produce exactly the same results across all visible demographics we can carve ourselves into?

c. Can the impact be mitigated by human post processing. The hypothetical face recognition system is part of the judicial process, there are many checks and balances before one gets to suffer drastic consequences. For example a human actually looking at the picture, or a solid alibi. "Your honor, I was skiing in Canada at the time of the alleged Florida murder".

As others have expressed in this thread, dealing with first order visual issues is easy. everyone can agree at a glance what a correct solution to visual questions is, and bugs are usually straightforward to fix. Language issues on the other hand, are second order, everything is subject to interpretation. Once we open the can of worms of talking 'critically' about language and AI, we are getting uncomfortably close to language police, and via Shapir Whorf, to thought police. The BIG underlying stake of 'AI Ethics', one that possibly neither side has completely articulated just yet:

Should a small group (in the thousands) of hyperachieving hyperpriviledged individuals working in the AI labs at the handful of megacorporations controlling the online flow of human language, get to decide what we can speak, and by extension what we can think?