I was listening to music on youtube and I got one of those low-view video recommendations for some Japanese jazz fusion band. I click it, because the genre is nice and because I like to discover new music, but right off the bat I start to get that uncanny feeling. Just a quick glance at the details in the thumbnail image and I can see the nonsensical asymmetry. The music sounds okay, but I can feel that it’s just… not right. So I go to check the description, and it has a kind of micro-biography for this band, describing their motivations for creating music, the history of their coming together and their first album, the name of the music label that produced it, their inspiration, complete with each individual band member’s name and role.
So I look up the band. Nothing. I look up the members. Nothing. I look up the music label. Nothing.
The band, the members, the music, the album, its cover, are all computer-generated. There is no disclaimer in the video, or its description. It’s the opposite, in fact, it’s all pretending to be real - to be human.
A deception, but also something worse. In this instance I was able to discern that something uncanny was going on, but I know that many people would not, and do not, the same way people are constantly falling for obvious lies in news, social media, etc… So for those people, they’re listening to a Japanese jazz fusion band from the early 90s. They like the music, the sounds are smooth and comforting but groovy, and there’s a false promise that behind the beat there’s a group of musicians from a time before the internet was even known to the vast majority of humanity, expertly working to express the combination of many years of practice, their various inspirations drawn from other humans and the world around them, and their cooperation with one another - their human relationships.
It’s a mockery of art, and of human expression. The presentation isn’t merely a lie; it’s an insult. An assertion that that band, those people, their inspirations, their relationship, doesn’t actually matter. And for every person that clicks the thumbnail, enjoys the music, and then moves on to the next thing without realizing it’s an artifice, the assertion is unchallenged. The insult is justified.
But of course, this isn’t limited to music, and it’s not limited to art. Every single day, imitations replace more and more of what we see, undercutting with each manifestation the value of human interaction.
I could distinguish that this album and this band were counterfeit, but if all I had been presented with was the music - no thumbnail, no description, no fake names - I wouldn’t really have been able to tell that it was pretending to be a product of human expression, I wouldn’t have the comfort of being able to confirm any suspicions - I would only be left with that sense that something was wrong. And what fills me with this creeping sense of dread is that I know how much money and effort is being pumped into this technology to make it more and more convincing, and that every day more of it is generated and dumped into social media, videos, music, chats, image-hosts, even little forums like this, like garbage into the ocean. Meaning that as time passes, from now and onward, I will fall for that sickening lie more and more while becoming increasingly paranoid and distrustful of every conversation I have, every game I play, every video I watch, every piece of music I enjoy.
It’s a wildfire, but no one’s fighting it - and the people with the most power to do so are air-dropping accelerant into the flames.
This is the video that inspired this post.
I’m sure some will try to pick apart the things I’ve said here, but just so they know: I’m not posting this to elicit any debate, I’m only sharing a newly-attained level of awareness of something that truly disgusts and unsettles me, in the way that sci-fi horror does. Invention not to benefit humanity, but to replace a crucial component of what’s important about being human with something artificial.
Footnote: my browser tried to tell me “accelerant” isn’t a word, so I did a search to make sure I wasn’t wrong about the spelling, and underneath the definition confirming I wasn’t wrong, the first link is to some AI-based site called “accelerant ai”.
“You read something which you thought only happened to you, and you discover that it happened 100 years ago to Dostoyevsky. This is a very great liberation for the suffering, struggling person, who always thinks that he is alone. This is why art is important.”
This James Baldwin quote really sums up my views on AI art.
At a certain level, I think most people, including myself, consume this sort of media wanting to know vaguely that it has a history and a progenitor, but uninterested in the specific details, and unwilling to do any sort of investigation.
We’re in an interesting halfway period where the lies are not yet so good we have no way of telling what’s real, but it’s still easy enough to achieve that bubble where the things we interact with are unknowingly fiction.
Eventually we will have to embrace that nothing we see or hear can be known to be real.
I have three concerns:
That IP laws will allow people to have sole access to media
That they will use AI to “fix” said media, either through remasters, or censorship to sanitize unpopular depictions
That their ownership will prevent one from viewing the original to get confirmation of authenticity.
It’s something that can start small or even with good intentions, but slight alterations can go against a creator’s vision and change the meaning or context completely.
It could be used to omit things outright. One of my fears is having a digital copy of a book that could be edited with a less radical message.
I fear that this would cause a level of alienation where we doubt what we saw and what we can expect to see even if it’s in real life.
That alienation and doubt could be good in the long run though where we seek out connections in the real world and ground ourselves with tangible things like books. Having other real people reading and discussing the same book can help us stay vigilant with manipulations.
I know it feels paranoid, but it could be something as subtle as changing the brand of soda from Coke to Pepsi in a Stephen King book because Pepsi bought out a publishing company.
It’s a single word change, but that is enough to erode trust in what you’re perceiving.
It doesn’t help that the people steering the ship into the icebergs have delusions of grandeur. These people believe they are creating the second coming of God. I just went into a rabbit hole on the news mega about one example of these people going off the deep end. Those who make the models don’t think they are doing good things, they think they are bringing about the Rapture and sentencing humanity to eternal damnation. Even when they think they are opening Pandora’s box, their immediate instinct isn’t to stop, but rather to grab a crowbar.
Those who make the models don’t think they are doing good things, they think they are bringing about the Rapture and sentencing humanity to eternal damnation. Even when they think they are opening Pandora’s box, their immediate instinct isn’t to stop, but rather to grab a crowbar.
Need to run LLMs on your own hardware. Most people will not do this.
edit: This doesn’t fix the problem of all the fake stuff, idk
You can mitigate it with RAG on local hardware. I run deepseek-r1:14b via anythingllm on my obsidian notes, and it’s a little better.
Welcome to atonalgroup, a project dedicated to pushing the boundaries of contemporary music.
Our mission is to explore the depths of auditory art, merging the worlds of sound design, composition, and innovation to create unique and immersive musical experiences.
From ambient soundscapes, rare grooves to avant-garde orchestrations, the fictional individuals and releases we feature provides a rich tapestry of auditory experiences designed to inspire and engage listeners from all walks of life.
Join us as we enter a new era in how we humans explore and appreciate music.
atonalgroup
Nominative determinism
when spotify recommended 3 french punk bands to me that i really liked but had never heard of before, i thought this exact scenario was playing out, but it turned out they were real bands. this time.
It is happening with print books on Amazon.
see this is why i literally retvrned to subsistence farming
dirt is still real
there’s birds
there’s birds
Birds aren’t real though
dirt is still real
there’s birds
Well for the time being anyway
dirt will outlast birds by like 50 years. so i think it has at least 62 good years still in it
i just ate some dirt and i can confirm it is real
Japanese knowers: Is the fake band name, 京都溝集団的 (Kyoto Groove Collective), grammatically correct? I was under the impression that 的 signaled an adjective/adverb, but shouldn’t “Collective” here be a noun?
You’re correct. Just 集団 on its own means group/collective (noun) like in company names. no need for 的 . Also the translation of Groove as 沟 (ditch) is also a direct translation it seems. Groove in Japanese would be グルーヴ
The chick’s jacket abruptly turns blue when you get to the right sleeve.
Their other fake artist of some Icelandic pianist gives the game away:
The content, including but not limited to text, images, audio, and video materials, on this platform is protected by copyright laws. Unauthorized reproduction, distribution, modification, or use of any copyrighted material without explicit permission from the copyright owner, atonalgroup, is strictly prohibited. All copyrights and intellectual property rights are retained by atonalgroup.
Why is the copyright owner the Youtube channel instead of the “gifted pianist with a unique cultural heritage?”
I possibly should have posted this in c/doomer or something. Any mod please feel free to move it if you feel it’s appropriate.
I found a YouTube link in your post. Here are links to the same video on alternative frontends that protect your privacy:
