How the way we read is changed by screens, and two books that question who’s doing what with our data.
The Black Box Society: The Secret Algorithms That Control Money and Information
By Frank Pasqual, £25.95, ISBN 9780674368279
Harvard University Press
Like ‘Data and Goliath’ (also reviewed this month), ‘The Black Box Society’ is a stark warning about the potential long-term consequences of failing to keep track of the ingenious uses firms are finding for the data they vacuum from sources as diverse as consumer health websites and magazine subscriptions, not to mention what’s publicly available.
Frank Pasquale, a law professor at the University of Maryland, provides a lawyer’s perspective on the situation in the form of a wake-up call to a society apparently sleepwalking into a situation where the routine surveillance foreseen in dystopian fiction like George Orwell’s ‘Nineteen Eighty-Four’ becomes an acceptable part of everyday life.
A member of the Council for Big Data, Ethics and Society, Pasquale argues that the root of the problem is one of transparency. The black box of his title is the traditional scientific concept of a system where users can see what goes in and what comes out, but have no idea what’s going on inside. This, he says, is what the systems which analyse our personal data have become.
Should we care? It’s tempting to regard the idea of secret algorithms controlling our lives as so much conspiracy theory, but anyone who’s had to deal with a problem with their credit rating will be well aware of the problems that opacity behind decision making can result in. Often it seems that even the ‘customer adviser’ at your bank doesn’t really understand how a decision about you has been reached.
And things could get worse as the ability of autonomous agents to mine for connections among inconceivably huge amounts of personal data becomes ever more sophisticated. Even if you’re not worried about this information being held, says Pasquale, have you considered what could be deduced by someone with your whole life story at their disposal and the persistence to analyse what it means about you?
One metaphor that’s easily grasped is that few of us understand how our car works but we can judge whether it’s doing its job of getting us to our destination safely and comfortably. The software that can have a potentially huge impact on our lives is just as opaque as an engine management system, but at the same time the extent to which it is protected by trade secrecy laws makes it impossible to assess whether it’s doing its job.
A bank or potential employer may have a good reason for turning you down for a loan of declining to give you a job. If that decision is based on a review of your past behaviour which draws on myriad apparently unconnected sources whose results are subjected to a process that’s impossible to interpret, how can you challenge it?
The biggest danger is that this is a self-perpetuating process that means ‘black box insiders’ acquire more and more influence – knowledge being power more than it ever has before – while the outsiders grow more and more ignorant of how key institutions function.
Ulitmately, this is a story about accountability and ‘The Black Box Society’ is a warning of how – if we don’t act soon – we risk losing it forever.
Data and Goliath: The Hidden Battles to Collect Your Data and Control the World
By Bruce Schneier, £17.99, ISBN 9780393244816
“My name is Marlowe, Philip Marlowe. Occupation: private detective. You know, somebody says, ‘Follow that guy,’, so I follow him.”
Traditional private eyes like Raymond Chandler’s quintessential Los Angeles gumshoe Philip Marlowe don’t get much of a look-in these days. We may not like to think about it, but the sort of comprehensive surveillance that until relatively recently would have involved sheer dogged persistence, patience and a few lucky breaks can be carried out in relative comfort by anyone equipped with the right technology.
Tracking where we go, what we read, who we meet. It’s all a case of capturing the right data from the devices we use. And what’s more, most of the time we’re happy to hand over this information in return for small things that make our lives easier in so many little ways.
When surveillance is used as a fictional device, there’s usually a happy coincidence where the person doing the spying stumbles upon just the connection they’re looking for at the right moment in the story. As security technologist and Harvard fellow Bruce Schneier explains in this book, ‘big data’ makes that obsolete. The ability of machines to closely analyse every aspect of our digital footprint at startling speed, and to interpret the connections it finds, means there’s no hiding place.
What Schneier is particularly worried about is the extent to which the other partners in that social contract are abusing the relationship with users. Routine monitoring may keep us safe, for example, but at what point does it become intrusive and infringe our civil liberties, and what can we do about it?
The different ways in which issues are legislated for around the world makes solving the big data problem a complex one, but he suggests a number of measures that need to be adopted urgently. The big problem though, he fears, is the extent to which society has to simply wake up to what’s going on and summon the political will to challenge what law enforcement, government and surveillance industries are doing.
‘Data and Goliath’ likens the situation today to the birth of the industrial revolution, and the smog of information it’s creating to a kind of digital pollution. Just as we wonder how our ancestors ignored the problems they were allowing to build up in their rush for progress, he asks, will our descendants judge us in a similar way when they look at how data collection and use is being allowed to evolve?
This is advertised as a book that everyone with a bank account, smart device or car – let alone an Internet connection in their home – needs to read. The increase in connectivity will quickly expand that audience to anyone with even a passing interest in the implications of living in a digital society.