• metaStatic@kbin.earth
    link
    fedilink
    arrow-up
    37
    arrow-down
    4
    ·
    3 months ago

    we have to be very careful about what ends up in our training data

    Don’t worry, the big tech companies took a snapshot of the internet before it was poisoned so they can easily profit from LLMs without allowing competitors into the market. That’s who “We” is right?

    • WhatAmLemmy@lemmy.world
      link
      fedilink
      English
      arrow-up
      20
      arrow-down
      1
      ·
      edit-2
      3 months ago

      It’s impossible for any of them to have taken a sufficient snapshot. A snapshot of all unique data on the clearnet would have probably been in the scale of hundreds to thousands of exabytes, which is (apparently) more storage than any single cloud provider.

      That’s forgetting the prohibitively expensive cost to process all that data for any single model.

      The reality is that, like what we’ve done to the natural world, they’re polluting and corrupting the internet without taking a sufficient snapshot — just like the natural world, everything that’s lost is lost FOREVER… all in the name of short term profit!