Tuesday, July 17, 2012

So when will we see the rise of Skynet?

Al Fin has a very interesting post about autonomous, armed micro drone assassins:
The size of these tiny killer-drones is quickly shrinking, so that it will not be long before lethal drone packages can be delivered under stealth in the form of birds, bats, or insects -- right in the lap of an enemy base or command and control centre.

It has become clear to militaries that remote controlled drones are subject to being hacked, hijacked, or having their communications jammed. Consequently, there is a rush to develop semi-autonomous or fully-autonomous drones capable of carrying out a mission after communications is lost.
This is, of course, a horrifyingly Bad Idea.  After all, what could possibly go wrong?  I have predictions:

1. The people who design this will understand what they're doing less than they think they do.  More specifically, the designers will be very explicit about what they know and what they don't know about the Artificial Intelligence (AI).  These warnings and caveats will be lost as the information
deteriorates
is briefed upwards in the command chain.

2.  The DoD people running this system will think that they are smarter than they are (see #1 above) and so will defer to the judgement of politicians as to when this should be deployed.  The Pols, of course, will see this as the Easy Way Out of making decisions and love the Plausible Deniability of the whole thing - if it goes bad, someone down the chain in the DoD will take the fall.

3. The people down the chain in the DoD will not understand this seductive lure to the politicians.  Of course, ex-politicians employed as lobbyists by the drone manufacturers will get filthy stinking rich.

4. When this goes bad - as everything eventually does go bad - it won't be the ex-politician lobbyists or their employers who take the fall.  Not will it be the current politicians, who of course never had a say in this decision But Who Are Absolutely Horrified And Who Will Not Rest Until The Get To The Bottom Of This.

5. This of course will be deployed domestically, despite any (worthless) assurances by Congress.

6. When this goes bad - as everything eventually does go bad -American citizens will be killed graveyard dead by bad AI.

I'm afraid that I'm pretty cynical about this whole situation.  Which leads to another prediction:

7. Borepatch will not ever get any of that sweet, sweet government funding for his security analysis.

Just sayin'.

12 comments:

Paladin said...

"right in the lap of an enemy base or command and control centre."

And coming soon to a residential neighborhood near you via your friendly Federal/SWAT/Militarized Police Force.

"5. This of course will be deployed domestically, despite any (worthless assurances by Congress)."

Precisely.

Divemedic said...

Of course, ex-politicians employed as lobbyists by the drone manufacturers will get filthy stinking rich.


When buying and selling are controlled by legislation, the first things to be bought and sold are legislators.
P. J. O'Rourke

Farmee said...

In 2012, WOPR became self aware.

Anonymous said...

OK, Armed Intelligentsia (to cop a phrase from another site), what's the best tool to defend your own personal airspace?

Buck-and-ball? Slugs all the way?

D'you think any enterprising manufacturer will start making dronish-like clays for antiaircraft practice?

Anonymous said...

Dr. Strangelove comes to mind.

Dave H said...

D'you think any enterprising manufacturer will start making dronish-like clays for antiaircraft practice?

Already done.

Broken Andy said...

I don't think these things will be given AI in the sense there is a "mind" guiding the actions. It's more like AI in the sense of the autonomic nervous system: this part knows to keep the flight path in the horizontal, that part knows not to detonate until inside a geo-fence, etc...

Borepatch said...

Andy, actually the article talks about how the uplink/downlink is perceived as a serious weakness, and so plans are being made to provide autonomous capability.

Anonymous said...

@ Dave H: Well, well! Gotta love that good ol' American sense of get up and go!

And the First Rule of Drones is: Thou Shalt Establish Air Supremacy — so with all those eleventy-billion firearms out there in the land, that might not be as easy to achieve as the Powers That Be think.

Anonymous said...

Increasing the cost of already-scarce helium, I foresee deployment of small balloons and large fine mesh nets of 2-6 lb test fishing line, followed by pintle-mounted 3 in Schedule 80 potato guns firing similar small mesh nets, radar control optional. It shouldn't be hard to control airspace below 300 feet or so if you don't mind collecting lots of real bugs.

Dave H said...

Gotta love that good ol' American sense of get up and go!

Except they're made in jolly old England. GnatUSA just imports and sells them.

I like the idea of lightweight barrage balloons. Go ahead and charge them with hydrogen. If a bogie gets too close, light it up.

Wraith said...

"Judgement Day is inevitable!"

--CSM101