I was involved in discussions 20-some years ago when we were first exploring the idea of autonomous and semiautonomous weapons systems. The question that really brought it home to me was “When an autonomous weapon targets a school and kills 50 kids, who gets charged with the war crime? The soldier who sent the weapon in, the commander who was responsible for the op, the company who wrote the software, or the programmer who actually coded it up?” That really felt like a grounding question.
Autonomous cars can also get into basically the trolley problem. If an accident is unavoidable, but the car can swerve and kill its own passenger to avoid killing more people in a larger wreck, should it? And would that end up as more liability for whoever takes the blame?
You could definitely put some blame on the manufacturer, but in legalize you “knew or should have known” that there was a possiblity that your vehicle could hurt or kill someone. You sent it out into the world without a driver in it, not the manufacturer. I wouldn’t be surprised to see warnings and agreements attached to autonomous vehicles telling people that there is risk.
When an autonomous weapon targets a school and kills 50 kids, who gets charged with the war crime?
When a human in a plane drops a bomb on a school full of kids, we don’t charge anyone with a war crime. Why would we start charging people with war crimes when we make the plane pilotless?
The autonomy of these killer toys is always overstated. As front-line trigger pullers, they’re great. But they still need an enormous support staff and deployment team and IT support. If you want to blame someone for releasing a killer robot into a crowd of civilians, its not like you have a shortage of people to indict. No different than trying to figure out who takes the blame for throwing a grenade into a movie theater. Everyone from the mission commander down to the guy who drops a Kill marker on the digital map has the potential for indictment.
But nobody is going to be indicted in a mission where the goal was to blow up a school full of children, because why would you do that? The whole point was to murder those kids.
But in 2021, the Jerusalem Post reported an intelligence official saying Israel had just won its first “AI war” – an earlier conflict with Hamas – using a number of machine learning systems to sift through data and produce targets. In the same year a book called The Human–Machine Team, which outlined a vision of AI-powered warfare, was published under a pseudonym by an author recently revealed to be the head of a key Israeli clandestine intelligence unit.
Last year, another +972 report said Israel also uses an AI system called Habsora to identify potential militant buildings and facilities to bomb. According the report, Habsora generates targets “almost automatically”, and one former intelligence officer described it as “a mass assassination factory”.
The recent +972 report also claims a third system, called Where’s Daddy?, monitors targets identified by Lavender and alerts the military when they return home, often to their family.
Literally the entire point of this system is to kill whole families.
Worrying about who’s gonna get charged with a war crime, during a war, is the opposite of grounded. During a war the only question is “How do we stop the robots from killing the school full of children?”
The issue people are worried about is that no one is making the decision to kill kids, it’s the AI making the call. It’s being given another objective and in the process of carrying that out makes the call to kill kids as part of that objective.
For example, you give an AI drone instructions to fly over an area to identify and drop bombs on military installations, and the AI misidentifies a school as a military base and bombs it. Or you send a dog bot in to patrol an area for intruders, and it misidentifies kids playing out in the streets as armed insurgents.
In a situation where it’s human pilots, soldiers, and analysts and such making the call, we would (or at least should) expect the people involved to face some sort of repercussions- jail time, fines, demotions, etc.
None of which you can really do for a drone.
And that’s of course before you get into the really crazy sci Fi dystopia stuff, where you send a team of robots into a city with general instructions to clear it of insurgents, and the AI comes to the conclusion somehow that the fastest and most efficient way to accomplish that is to just kill every person in the city since it can’t be absolutely sure who is and isn’t a terrorist
I was involved in discussions 20-some years ago when we were first exploring the idea of autonomous and semiautonomous weapons systems. The question that really brought it home to me was “When an autonomous weapon targets a school and kills 50 kids, who gets charged with the war crime? The soldier who sent the weapon in, the commander who was responsible for the op, the company who wrote the software, or the programmer who actually coded it up?” That really felt like a grounding question.
As we now know, the actual answer is “Nobody.”
We don’t even charge people when they blow up schools and hospitals with drone strikes now. Why would this be any different?
To be fair, the answer to the question “when somebody kills a schoolbus of kids, who gets charged with a warcrime?” was always “nobody”
FTFY - it’s the American way.
No they were terrorists the whole time /s
They are now.
That’s also a legal issue with autonomous cars.
Autonomous cars can also get into basically the trolley problem. If an accident is unavoidable, but the car can swerve and kill its own passenger to avoid killing more people in a larger wreck, should it? And would that end up as more liability for whoever takes the blame?
The owner or lesse of the car is responsible. Think of the car as a dog that bit a child.
Are we talking truly autonomous vehicles with no driver, or today’s “self-driving-but-keep-your-hands-on-the-wheel” type cars?
In the case of the former, it should be absolutely the fault of the manufacturer.
You could definitely put some blame on the manufacturer, but in legalize you “knew or should have known” that there was a possiblity that your vehicle could hurt or kill someone. You sent it out into the world without a driver in it, not the manufacturer. I wouldn’t be surprised to see warnings and agreements attached to autonomous vehicles telling people that there is risk.
Say there is a car with no human driver, that is being sold as requiring “no human input other than set destination, stop, and go”.
If that vehicle crashes, you think the person who bought the car (the passenger) has legal liability, and not the manufacturer?
That’s like being a passenger on a bus and getting sued if the bus driver hits a parked car.
The bus company gets sued because they own the bus, not the driver. Same as if you lend your car to someone, you’re at least partially responsible.
When a human in a plane drops a bomb on a school full of kids, we don’t charge anyone with a war crime. Why would we start charging people with war crimes when we make the plane pilotless?
The autonomy of these killer toys is always overstated. As front-line trigger pullers, they’re great. But they still need an enormous support staff and deployment team and IT support. If you want to blame someone for releasing a killer robot into a crowd of civilians, its not like you have a shortage of people to indict. No different than trying to figure out who takes the blame for throwing a grenade into a movie theater. Everyone from the mission commander down to the guy who drops a Kill marker on the digital map has the potential for indictment.
But nobody is going to be indicted in a mission where the goal was to blow up a school full of children, because why would you do that? The whole point was to murder those kids.
Israelis already have an AI-powered target-to-kill system, after all.
Literally the entire point of this system is to kill whole families.
To be fair they are specifically testing AI Aimed and not fired. Firing is still up to and operator
deleted by creator
For now. The goal would obviously be to have a fully autonomous machine.
For now.
We already have heat seeking missiles.
Collateral damage.
Yep. That’s exactly like every AI system employed by the IDF.
Guns don’t kill people. Autonomous robot dogs do.
Do autonomous robot kill dogs fall under the second amendement?
Worrying about who’s gonna get charged with a war crime, during a war, is the opposite of grounded. During a war the only question is “How do we stop the robots from killing the school full of children?”
I feel like the answer would be the person who decided to kill the kids, right? they are the one who made the call to commit the war crime.
The issue people are worried about is that no one is making the decision to kill kids, it’s the AI making the call. It’s being given another objective and in the process of carrying that out makes the call to kill kids as part of that objective.
For example, you give an AI drone instructions to fly over an area to identify and drop bombs on military installations, and the AI misidentifies a school as a military base and bombs it. Or you send a dog bot in to patrol an area for intruders, and it misidentifies kids playing out in the streets as armed insurgents.
In a situation where it’s human pilots, soldiers, and analysts and such making the call, we would (or at least should) expect the people involved to face some sort of repercussions- jail time, fines, demotions, etc.
None of which you can really do for a drone.
And that’s of course before you get into the really crazy sci Fi dystopia stuff, where you send a team of robots into a city with general instructions to clear it of insurgents, and the AI comes to the conclusion somehow that the fastest and most efficient way to accomplish that is to just kill every person in the city since it can’t be absolutely sure who is and isn’t a terrorist
good points made, I agree