It’s possible for local AI models to be very economical on energy, if used for the right tasks.
For example I’m running RapidOCR which uses a modern transformer architecture, and absolutely blows away traditional OCR at capturing data from character displays.
Doesn’t even need a GPU and returns results in under a second on a modern CPU. No preprocessing needed, just feed it an image. This little multimodal transformer is just as much “AI” as bloated general purpose GPTs, but it’s cheap, fast and useful.
That’s cool and all, but we’re talking about the AI companies that are trying to get valuated at trillions of dollars and want taxpayers to pay for the upgrades to the grid. The sad part is it’s likely going to work
I’m all for having companies pay for their electricity use and their impact on the grid but that has nothing to do with AI.
Llama took 2 600 mWh to train over 6 months and can run on much less than what’s needed for gaming. ActivisionBlizzard used 86 000 mWh of energy in 2022 for both the datacenters for their games and the development of them. Yet no one in their right mind would suggest to curb stomp gaming to save on energy.
Openai has bigger costs but they run inference, and having them run it actually makes it more efficient, even though I rather open source models you can run on your own machine.
The clear solution is upgrading to a more robust green energy grid, not blocking innovation.
And if we are going to ban things because of their energy use, there are much better candidates than software. A transatlantic flight takes up 500 mWh, so essentially 1000 people flying to Europe and back use up as much energy as the llama model took to train, a model that has been downloaded 3.5 million times in the past month alone on hugging face (only with the official 8b included, and not counting the other sizes or the thousands of finetunes).
Most big companies publish their energy usage like the two examples above. For the plane bit, I just found multiple people calculating it and coming up with the same number online, so that one might be hot air.
Blizzard isn’t asking taxpayers to subsidize them billions “to advance humanity”.
As you say yourself, there are way better models than what is being funded right now, and what is likely to get the monopoly on energy, at our expense.
I’m just stating that “AI” is a broad field. These lightweight and useful transformer models are a direct product of other AI research.
I know what you mean, but simply stating “Don’t use AI” isn’t really valid anymore as soon these ML models will be a common component. There are even libraries and hardware acceleration support for tensor operations on the ESP32-S3.
Right, we need to come up with better terms for talking about “AI”. Personally at the moment I’m considering any transformer-type ML system to be part of the category, as you stated none of them are any more “intelligent” than any others. They’re all just a big stack of tensor operations. So if one is AI, they all are.
Remember long ago when “fuzzy logic” was all the hype and considered to be AI? Just a very early form of classifier network but everyone was super excited at the time.
It’s possible for local AI models to be very economical on energy, if used for the right tasks.
For example I’m running RapidOCR which uses a modern transformer architecture, and absolutely blows away traditional OCR at capturing data from character displays.
Doesn’t even need a GPU and returns results in under a second on a modern CPU. No preprocessing needed, just feed it an image. This little multimodal transformer is just as much “AI” as bloated general purpose GPTs, but it’s cheap, fast and useful.
That’s cool and all, but we’re talking about the AI companies that are trying to get valuated at trillions of dollars and want taxpayers to pay for the upgrades to the grid. The sad part is it’s likely going to work
I’m all for having companies pay for their electricity use and their impact on the grid but that has nothing to do with AI.
Llama took 2 600 mWh to train over 6 months and can run on much less than what’s needed for gaming. ActivisionBlizzard used 86 000 mWh of energy in 2022 for both the datacenters for their games and the development of them. Yet no one in their right mind would suggest to curb stomp gaming to save on energy.
Openai has bigger costs but they run inference, and having them run it actually makes it more efficient, even though I rather open source models you can run on your own machine.
The clear solution is upgrading to a more robust green energy grid, not blocking innovation.
And if we are going to ban things because of their energy use, there are much better candidates than software. A transatlantic flight takes up 500 mWh, so essentially 1000 people flying to Europe and back use up as much energy as the llama model took to train, a model that has been downloaded 3.5 million times in the past month alone on hugging face (only with the official 8b included, and not counting the other sizes or the thousands of finetunes).
Have you got something to read up on regarding comparisons of energy consumption? Sounds really interesting, but I know close to jack shit about this.
Most big companies publish their energy usage like the two examples above. For the plane bit, I just found multiple people calculating it and coming up with the same number online, so that one might be hot air.
That’s completely besides the point.
Blizzard isn’t asking taxpayers to subsidize them billions “to advance humanity”.
As you say yourself, there are way better models than what is being funded right now, and what is likely to get the monopoly on energy, at our expense.
I’m just stating that “AI” is a broad field. These lightweight and useful transformer models are a direct product of other AI research.
I know what you mean, but simply stating “Don’t use AI” isn’t really valid anymore as soon these ML models will be a common component. There are even libraries and hardware acceleration support for tensor operations on the ESP32-S3.
I didn’t say don’t use AI.
As usual with “AI”, there’s no intelligence involved with OCR. It’s just more data processing / classification being lumped into the hype.
Right, we need to come up with better terms for talking about “AI”. Personally at the moment I’m considering any transformer-type ML system to be part of the category, as you stated none of them are any more “intelligent” than any others. They’re all just a big stack of tensor operations. So if one is AI, they all are.
Remember long ago when “fuzzy logic” was all the hype and considered to be AI? Just a very early form of classifier network but everyone was super excited at the time.