You put it really succinctly.

These models are trained on images generally depicting a lot of concepts. If I do a simple prompt like monkey I’m probably going to get a lot of images of monkeys in a natural setting, because when trained it’s given images essentially stating that this is an image of “monkey, tree, fruit, foliage, green” and so forth. Over time this pattern repeats, and it will start associating one concept with another. Because a lot of pictures that depict monkey also depict foliage, nature and what have you, it makes sense for it to fill in the rest of monkey with foliage, green, nature and less volcano, playful dog, ice cream since those concepts haven’t been very prominent when presented with monkey.

That is essentially the problem.

Here is the result for “monkey”

And here is monkey, volcano, playful dog, ice cream

The datasets are permeated with these kinds of examples. If you prompt for nurse you’ll get a lot of women in blue/hospitaly clothing, inside hospitals or non-descript backgrounds, and very few men.

Here’s photo of a nurse

The more verbose and specific you get though, the likelier it is that you’ll get the outcome you want. Here for example is male (nurse:1) (wearing white scrubs:1) with pink hair skydiving

This was so outlandish that without the (wearing white scrubs:1) it just put the skydiver in a random pink outfit, even with the added weight on wearing white scrubs it has a tendency to put the subject in something pink. Without the extra weight on (nurse:1) it gave me generic pink (mostly) white men.

If we were to fix the biases present in our society, we’d possibly see less biases in the datasets, and subsequently less bias in the final models. The issue I believe, isn’t so much a technological one, as it is a societal one. The machines are only racist because we are teaching them to be, by example.