ChatGPT output had to be heavily tweaked behind the scene by an army of Kenyan workers who are paid less than $2 per hour to make it appear coherent.

From The Horrific Content a Kenyan Worker Had to See While Training ChatGPT (cw: descriptions of all sorts of depravity):

The type of work Mathenge performed has been crucial for bots like ChatGPT and Google’s Bard to function and to feel so magical. But the human cost of the effort been widely overlooked. In a process called “Reinforcement Learning from Human Feedback,” or RLHF, bots become smarter as humans label content, teaching them how to optimize based on that feedback. A.I. leaders, including OpenAI’s Sam Altman, have praised the practice’s technical effectiveness, yet they rarely talk about the cost some humans pay to align the A.I. systems with our values. Mathenge and his colleagues were on the business end of that reality.