It’s also like, I guess I would prefer it to make mistakes like this if it means it is less biased towards whiteness in other, less specific areas?
Like, we know these models are dumb as rocks. We know that they are imperfect and that they mirror the biases of their trainers and training data, and that in American society that means bias towards whiteness. If the trainers are doing what they can to prevent that from happening, whatever, that’s cool… even if the result is some dumb stuff like this sometimes.
I also don’t think it’s a problem for the user to specify race if it matters? Like “a white queen of England” is a fine thing to ask for, and if it isn’t specified, the model will include diverse options even if they aren’t historically accurate. No one gets bent out of shape if the outfits aren’t quite historically accurate, for example
The problem is that these answers are hugely incorrect and if some child learning about history of England would see this, they would create bias that England was always diverse.
The same is true for some recent post, where people knowing nothing about Scotland history could learn from images that half of Scotland population in 18th century was black.
So from my perspective these images are just completely wrong and it should be fixed.
Also if you want diversity, what about handicapped people?
“Current AI is not a knowledge tool. It MUST NOT be used to get information about any topic!”
If your child is learning Scottish history from AI, you failed as a teacher/parent.
This isn’t even about bias, just about what an AI model is. It’s not even supposed to be correct, that’s not what it is for. It is for appearing as correct as the things it has been trained on. And as long as there are two opinions in the training data, the AI will gladly make up a third.
That doesn’t matter though. People will definitely use it to acquire knowledge, they are already doing it now. Which is why it’s so dangerous to let these “moderate” inaccuracies fly.
You even perfectly summed up why that is: LLMs are made to give a possibly correct answer in the most convincing way.
it’s true that this would mislead children, but the model could hallucinate about literally anything. Especially at this stage, no one-- children or adults-- should be uncritically accepting what the model states as fact. That said, I agree LLMs need to improve their factual accuracy
Although it is highly debated, some scholars suggest Queen Charlotte might have had African ancestry, or that she would be considered a POC by today’s standards. Of course, she reigned in the 17-1800s, but it isn’t entirely outlandish to have a “Queen of Color”, if we aren’t requesting a specific queen or a specific race
People of color did live in England in the middle ages? Like not diverse in the way we conceive now, but here are a few papers discussing the racial diversity at the time. It was surely less intermingled than today, but it’s not like these images are impossible
Other things are anachronistic or fantastical about these images, such as clothing. Are we worried about children getting the wrong impression of history in that sense?
Of course increasing visibility and representation of all kinds of marginalized people is important. I, myself, am disabled, so I care about that representation too-- thanks for pointing out how we could improve the model further. I do kinda feel like people would be groaning if the model had produced a Queen with a visible disability, though… I would be delighted to be wrong on this front :)
I know that POC lived in England and it was possible to meet someone like that. But I would prefer if the model gave most possible, most general answers. If I ask for an image of a car I would like to give me four-wheeled red or gray or green car, not three-wheeled pink car just because there exist some car like that.
That’s valid! I agree. I think in this case it would be reasonable for the model to give multiple (or like, at least one, jeez) images with white queens. I don’t disagree with anyone in that sense. I just also don’t think it’s worth pitching a fit when the dumbass model that has been trained to show more racial diversity produces (frankly comical) hallucinations.
The ethos of the trainers is a good one. Attempting to counter the (demonstrated, measurable) bias of many models toward whiteness is a good choice. I prefer that the trainers choose to address the bias even if it (sometimes, in early versions) makes the model make silly mistakes like this. That’s all.
These are not hallucinations. The image generator system prompt has been intensely altered to mix all races and genders. The model is probably not inaccurate up until it being misused. The misuse could be at any level of interaction, so it’s very misleading to base it on such an example
Not sure why you’re getting downvoted. The user essentially asked for the AI to generate some random made up rulers of England. Might as well have asked it for new Game of Thrones characters for all the difference it would have made. These are not real people so it, quite correctly, threw in a whole load of mixed races because why wouldn’t it? No idea why people are getting bent out of shape over someone doing a poor job of assigning prompts.
Yes, because it’s not smart enough to know what a ‘15th century African king’ is, let alone what one should look like. I’m not saying that’s a good thing, but I think people expect too much from these programs. If I wanted a maui chief, I’d start be specifying ‘an African man in 15th century tribal gear’ and take it from there. They mostly seem to work better if you specify the race you want, not just assume it understands enough about the historical period to do that for you.
It’s also like, I guess I would prefer it to make mistakes like this if it means it is less biased towards whiteness in other, less specific areas?
Like, we know these models are dumb as rocks. We know that they are imperfect and that they mirror the biases of their trainers and training data, and that in American society that means bias towards whiteness. If the trainers are doing what they can to prevent that from happening, whatever, that’s cool… even if the result is some dumb stuff like this sometimes.
I also don’t think it’s a problem for the user to specify race if it matters? Like “a white queen of England” is a fine thing to ask for, and if it isn’t specified, the model will include diverse options even if they aren’t historically accurate. No one gets bent out of shape if the outfits aren’t quite historically accurate, for example
The problem is that these answers are hugely incorrect and if some child learning about history of England would see this, they would create bias that England was always diverse.
The same is true for some recent post, where people knowing nothing about Scotland history could learn from images that half of Scotland population in 18th century was black.
So from my perspective these images are just completely wrong and it should be fixed.
Also if you want diversity, what about handicapped people?
Repeat after me:
“Current AI is not a knowledge tool. It MUST NOT be used to get information about any topic!”
If your child is learning Scottish history from AI, you failed as a teacher/parent. This isn’t even about bias, just about what an AI model is. It’s not even supposed to be correct, that’s not what it is for. It is for appearing as correct as the things it has been trained on. And as long as there are two opinions in the training data, the AI will gladly make up a third.
That doesn’t matter though. People will definitely use it to acquire knowledge, they are already doing it now. Which is why it’s so dangerous to let these “moderate” inaccuracies fly.
You even perfectly summed up why that is: LLMs are made to give a possibly correct answer in the most convincing way.
deleted by creator
it’s true that this would mislead children, but the model could hallucinate about literally anything. Especially at this stage, no one-- children or adults-- should be uncritically accepting what the model states as fact. That said, I agree LLMs need to improve their factual accuracy
Although it is highly debated, some scholars suggest Queen Charlotte might have had African ancestry, or that she would be considered a POC by today’s standards. Of course, she reigned in the 17-1800s, but it isn’t entirely outlandish to have a “Queen of Color”, if we aren’t requesting a specific queen or a specific race
People of color did live in England in the middle ages? Like not diverse in the way we conceive now, but here are a few papers discussing the racial diversity at the time. It was surely less intermingled than today, but it’s not like these images are impossible
Other things are anachronistic or fantastical about these images, such as clothing. Are we worried about children getting the wrong impression of history in that sense?
Of course increasing visibility and representation of all kinds of marginalized people is important. I, myself, am disabled, so I care about that representation too-- thanks for pointing out how we could improve the model further. I do kinda feel like people would be groaning if the model had produced a Queen with a visible disability, though… I would be delighted to be wrong on this front :)
I know that POC lived in England and it was possible to meet someone like that. But I would prefer if the model gave most possible, most general answers. If I ask for an image of a car I would like to give me four-wheeled red or gray or green car, not three-wheeled pink car just because there exist some car like that.
That’s valid! I agree. I think in this case it would be reasonable for the model to give multiple (or like, at least one, jeez) images with white queens. I don’t disagree with anyone in that sense. I just also don’t think it’s worth pitching a fit when the dumbass model that has been trained to show more racial diversity produces (frankly comical) hallucinations.
The ethos of the trainers is a good one. Attempting to counter the (demonstrated, measurable) bias of many models toward whiteness is a good choice. I prefer that the trainers choose to address the bias even if it (sometimes, in early versions) makes the model make silly mistakes like this. That’s all.
These are not hallucinations. The image generator system prompt has been intensely altered to mix all races and genders. The model is probably not inaccurate up until it being misused. The misuse could be at any level of interaction, so it’s very misleading to base it on such an example
Not sure why you’re getting downvoted. The user essentially asked for the AI to generate some random made up rulers of England. Might as well have asked it for new Game of Thrones characters for all the difference it would have made. These are not real people so it, quite correctly, threw in a whole load of mixed races because why wouldn’t it? No idea why people are getting bent out of shape over someone doing a poor job of assigning prompts.
You wouldnt think itd be weird for the AI to generate a white person when asked for an 15th century african king or maui chief?
Yes, because it’s not smart enough to know what a ‘15th century African king’ is, let alone what one should look like. I’m not saying that’s a good thing, but I think people expect too much from these programs. If I wanted a maui chief, I’d start be specifying ‘an African man in 15th century tribal gear’ and take it from there. They mostly seem to work better if you specify the race you want, not just assume it understands enough about the historical period to do that for you.
Because how dare those black people