A Telegram user who advertises their services on Twitter will create an AI-generated pornographic image of anyone in the world for as little as $10 if users send them pictures of that person. Like many other Telegram communities and users producing nonconsensual AI-generated sexual images, this user creates fake nude images of celebrities, including images of minors in swimsuits, but is particularly notable because it plainly and openly shows one of the most severe harms of generative AI tools: easily creating nonconsensual pornography of ordinary people.

  • guyrocket@kbin.social
    link
    fedilink
    arrow-up
    105
    arrow-down
    33
    ·
    7 months ago

    This is not new. People have been Photoshopping this kind of thing since before there was Photoshop. Why “AI” being involved matters is beyond me. The result is the same: fake porn/nudes.

    And all the hand wringing in the world about it being non consensual will not stop it. The cat has been out of the bag for a long time.

    I think we all need to shift to not believing what we see. It is counterintuitive, but also the new normal.

    • kent_eh@lemmy.ca
      link
      fedilink
      English
      arrow-up
      91
      arrow-down
      7
      ·
      7 months ago

      People have been Photoshopping this kind of thing since before there was Photoshop. Why “AI” being involved matters is beyond me

      Because now it’s faster, can be generated in bulk and requires no skill from the person doing it.

      • ArmokGoB@lemmy.dbzer0.com
        link
        fedilink
        English
        arrow-up
        44
        arrow-down
        19
        ·
        7 months ago

        I blame electricity. Before computers, people had to learn to paint to do this. We should go back to living like medieval peasants.

      • Vespair@lemm.ee
        link
        fedilink
        English
        arrow-up
        1
        arrow-down
        3
        ·
        7 months ago

        no skill from the person doing it.

        This feels entirely non-sequitur, to the point of damaging any point you’re trying to make. Whether I paint a nude or the modern Leonardi DaVinci paints a nude our rights (and/or the rights of the model, depending on your perspective on this issue) should be no different, despite the enormous chasm that exists between our artistic skill.

      • Bob Robertson IX @discuss.tchncs.de
        link
        fedilink
        English
        arrow-up
        20
        arrow-down
        36
        ·
        7 months ago

        A kid at my high school in the early 90s would use a photocopier and would literally cut and paste yearbook headshots onto porn photos. This could also be done in bulk and doesn’t require any skills that a 1st grader doesn’t have.

        • ChexMax@lemmy.world
          link
          fedilink
          English
          arrow-up
          30
          arrow-down
          9
          ·
          7 months ago

          Those are easily disproven. There’s no way you think that’s the same thing. If you can pull up the source photo and it’s a clear match/copy for the fake it’s easy to disprove. AI can alter the angle, position, and expression on your face in a believable manor making it a lot harder to link the photo to source material

          • Bob Robertson IX @discuss.tchncs.de
            link
            fedilink
            English
            arrow-up
            15
            arrow-down
            5
            ·
            7 months ago

            This was before Google was a thing, much less reverse lookup with Google Images. The point I was making is that this kind of thing happened even before Photoshop. Photoshop made it look even more realistic. AI is the next step. And even the current AI abilities are nothing compared to what they are going to be even 6 months from now. Yes, this is a problem, but it has been a problem for a long time and anyone who has wanted to create fake nudes of someone has had the ability to easily do so for at least a generation now. We might be at the point now where if you want to make sure you don’t have fake nudes created of you, then you don’t have images of yourself published. However now that everyone has high quality cameras in their pockets, this won’t 100% protect you.

      • Dkarma@lemmy.world
        link
        fedilink
        English
        arrow-up
        2
        arrow-down
        18
        ·
        7 months ago

        Not relevant. Using someone’s picture never ever required consent.

    • EatATaco@lemm.ee
      link
      fedilink
      English
      arrow-up
      22
      arrow-down
      1
      ·
      7 months ago

      I suck at Photoshop and Ive tried many times to get good at it over the years. I was able to train a local stable diffusion model on my and my family’s faces and create numerous images of us in all kinds of situations in 2 nights of work. You can get a snap of someone and have nudes of them tomorrow for super cheap.

      I agree there is nothing to be done, but it’s painfully obvious to me that the scale and ease of it that makes it much more concerning.

      • T156@lemmy.world
        link
        fedilink
        English
        arrow-up
        4
        ·
        7 months ago

        Also the potential for automation/mass-production. Photoshop work still requires a person to sit down to do the actual photoshop. You can try to script things out, but it’s hardly an easy affair.

        By comparison, generative models are much more hands-free. Once you get the basics set up, you can just have it go, and churn things at rates well surpassing what a single human could reasonably do (if you have the computing power for it).

    • AstralPath@lemmy.ca
      link
      fedilink
      English
      arrow-up
      20
      arrow-down
      13
      ·
      7 months ago

      This kind of attitude toward non-consensual actions is what perpetuates them. Fuck that shit.

    • dysprosium@lemmy.dbzer0.com
      link
      fedilink
      English
      arrow-up
      8
      arrow-down
      2
      ·
      7 months ago

      Exactly this. And rather believe cryptographically sighed images by comparing hashes with the one supplied by the owner. Then it’s rather a question of trusting a specific source for a specific kind of content. A news photo of the war in Ukraine by the BBC? Check hash on their site. Their reputation is fini if a false image has been found.

      • T156@lemmy.world
        link
        fedilink
        English
        arrow-up
        1
        ·
        7 months ago

        At the same time, that does introduce an additional layer of work. Most people aren’t going to do that just for the extra work that it would involve, in much the same way that people today won’t track down an image back down to the original source, but usually just go by the one that they saw.

        Especially for people who aren’t so cryptographically or technologically inclined that they know what a hash is, where to find one, and how to compare it (without just opening them both and checking personally).

        • dysprosium@lemmy.dbzer0.com
          link
          fedilink
          English
          arrow-up
          1
          ·
          7 months ago

          Sure but that’s no problem if software would do that automatically for users of big (news) sites. Browsers on desktop and apps on phones.

    • HubertManne@kbin.social
      link
      fedilink
      arrow-up
      1
      arrow-down
      10
      ·
      7 months ago

      This is something I can’t quite get through to my wife. She does not like that I dismiss things to some degree when it does not makes sense. We get into these convos where Im like I have serious doubts about this and she is like. Are you saying it did not happen and im like. no. It may have happened but not in quite the way they say or its being portrayed in a certain manner. Im still going to take video and photos for now as being likely true but I generally want to see it from independent sources. like different folks with their phones along with cctv of some kind and such.