Feudal Security

Discussion

Cory Doctorow:

“Apple has now arrogated to itself the power to know, with a reasonable degree of granularity, which programs its custom¬ers are using, and to decide whether customers should be permitted to do so. Nothing in this surveillance system prevents it from being used against legitimate software. Nothing prevents it from being used to extract surveil¬lance data about Apple customers – for example, to determine where you are, or whether there is anyone else there with you running a Mac. The only thing that stops Apple from blocking you from running legitimate apps – or from gathering information about your movements and social activities – is its goodwill and good judgment, and therein lies the problem. The security researcher (and Hugo Award-nominee) Bruce Schneier has a name for this arrangement: he calls it feudal security. Here in the 21st century, we are beset by all manner of digital bandits, from identity thieves, to stalkers, to corporate and government spies, to harassers. There is no way for us to defend ourselves: even skilled technologists who administer their own networked services are no match for the bandits. To keep bandits out, you have to be perfect and perfectly vigilant, and never make a single mistake. For the bandits to get you, they need merely find a single mistake that you’ve made.

To be safe, then, you have to ally yourself with a warlord. Apple, Google, Facebook, Microsoft, and a few others have built massive fortresses bristling with defenses, whose parapets are stalked by the most ferocious cybermerce¬naries money can buy, and they will defend you from every attacker – except for their employers. If the warlord turns on you, you’re defenseless.

We see this dynamic playing out with all of our modern warlords. Google is tweaking Chrome, its dominant browser, to block commercial surveillance, but not Google’s own commercial surveillance. Google will do its level best to block scumbag marketers from tracking you on the web, but if a marketer pays Google, and convinces Google’s gatekeepers that it is not a scumbag, Google will allow them to spy on you. If you don’t mind being spied on by Google, and if you trust Google to decide who’s a scumbag and who isn’t, this is great. But if you and Google disagree on what constitutes scumbaggery, you will lose, thanks, in part, to other changes to Chrome that make it much harder to block the ads that Chrome lets through.

Over in Facebook land, this dynamic is a little easier to see. After the Cambridge Analytica scandal, Facebook tightened up who could buy Facebook’s surveillance data about you and what they could do with it. Then, in the runup to the 2020 US elections, Facebook went further, insti¬tuting policies intended to prevent paid political disinformation campaigns at a critical juncture.

But Facebook isn’t doing a very good job of defending its users from the bandits. It’s a bad (or possibly inattentive, or indifferent, or overstretched) warlord, though. We know this thanks to Ad Observer and Ad Observatory, a pair of tools from NYU’s engineering school. Ad Observer is a browser plugin that Facebook users run; whenever they encounter an ad, Ad Observer makes a copy of it and sends it to Ad Observatory, an open repository of Facebook ads. Researchers and accountability journalists use Ad Observatory to document all the ways that Facebook is failing to enforce its own policies.

In October, Facebook sent a legal threat to NYU, demanding that Ad Observer and Ad Observatory shut down. Facebook says that it is doing its duties as an honest warlord here, because Ad Observer could (but doesn’t) violate its users’ privacy. As the local warlord, Facebook has a duty to prevent anyone from supplying the people inside its fortress with tools that could expose those under its protection to risk.

While the risk to Facebook users from Ad Observer is wholly hypotheti¬cal, Ad Observer poses a concrete risk to Facebook itself, by exposing the company’s failings to live up to both its promises and its legal duties stem¬ming from various settlements over past privacy violations – and the one entity Facebook will never, ever protect you from is Facebook. They’ve got lots of resources at their disposal, too: not just cybermercenaries that could tweak Facebook’s systems to try to block Ad Observer, but also a legion of lawyers who can enlist the crown to destroy Ad Observer on its behalf.”

(https://locusmag.com/2021/01/cory-doctorow-neofeudalism-and-the-digital-manor/)