Big brother: China's data-driven Social Credit system sounds like a sci-fi dystopia

Social media thrives by measuring popularity. As likes, views, retweets, followers and friends increase in number, we’re given a regularly updated measure of our importance, our influence, our self-worth. If we do well, our reputation climbs; we might even become flavour of the month. Make mistakes, however, and our online stock will slump as we’re blocked, unfollowed and rejected. Those who shun social media might see all this as unimportant and frivolous, but people’s lives can be shaped by these numbers: it can affect their job prospects, their social standing and their mental health. If you turbocharged this idea and put a government in control of it, you’d have something resembling the Social Credit system that’s being rolled out in China. A hybrid of credit ratings, behaviour analysis and state blacklists, it was announced by the Chinese Ministry of Public Security in 2015 as a way of assigning every Chinese citizen a score based on their “sincerity”, and to reward high achievers. It’s set to become fully operational in the next two years. “Its objective,” read the Ministry’s announcement, “is to raise the honest mentality and credit levels of the entire society.” It may be a socialist utopian dream, but it’s a nightmare for businesses and citizens who’ve ended up with low scores. As the Social Credit system is shaped via a series of trials, reports have emerged of people being treated unfairly. When a lawyer, Li Xiaolin, was sued for defamation by one of his clients, he found himself prohibited from buying a plane ticket. Journalists deemed untrustworthy by the state have experienced similar difficulties travelling, and additional obstacles might prevent people from renting certain cars or properties. One school in Shandong province refused to enrol students whose parents have low scores. And because an authoritarian state lurks behind the system, the chance of appealing successfully against a score is very low. Life is different for those who score highly. An impressive social rating can allow someone to skip the queue to see a doctor, or rent a car without paying a deposit. Other benefits may include better terms on loans or greater visibility on dating apps. The highest scorers may even qualify for fast-tracked visa applications to travel abroad. As with “likes” on Instagram or retweets on Twitter, people take pride in rising scores and even boast about them. But while the system can open doors, that comes at a price, according to Dr Samantha Hoffman, non-resident Fellow at the Australian Strategic Policy Institute. “In large part,” she says, “the social credit system relies on providing convenience to individuals and entities in order to co-opt their participation in their own management.” The Chinese Supreme Court’s 2013 decision to make public the names of defaulting debtors was perhaps the first seed of the Social Credit system. It’s been reported that 8.7 million flights and 3.4 million train journeys were denied to blacklisted citizens up until the end of 2017. But the integration of smartphones into society, and the widespread use of payment apps such as Alipay and WeChat Pay, has resulted in a colossal amount of behavioural data that can be fed into a calculation of “sincerity”. The companies behind these apps (Ant Financial and TenCent respectively) are two of eight firms authorised by the Chinese government to develop trial credit scoring platforms, and over recent months hints have been dropped as to the kind of measurements being recorded: how often you eat takeaway food, the amount of time you might spend playing online games, what car you drive, your conduct on social media, where you travel... And if your friends score poorly, it can reflect poorly on you, too. The progression from these trial schemes into a fully-fledged Social Credit score relies on data sharing and co-operation between technology firms and the government. “Chinese tech companies operate because the Chinese Communist Party allows them to,” notes Dr Hoffman. “It does not matter if a company, or the individual leadership of a company is indifferent toward the Party – they are still legally responsible to the Party.” Joint projects in fields such as deep learning and quantum computing are already underway, and it seems inevitable that this relationship will grow ever closer, with potentially alarming consequences. In an article for Wired magazine, writer Mara Hvistendahl describes how Ant Financial recently trialled a facial recognition feature at a branch of KFC where you pay for your food by looking into a camera. Initiatives such as this, combined with the Chinese government's plan to develop a facial recognition system for all 1.3 billion of its citizens, edges Social Credit even further into the realm of science fiction. It's easy for anyone living outside China to regard Social Credit as an anomaly peculiar to that country, and not something that could ever affect us. But we own the same smartphones and generate the same data as Chinese citizens. We're already being scored in many ways, both above and below the radar. Credit scoring companies are accumulating ever-greater insight into our spending habits. Our job prospects can be affected by something as simple as our choice of Facebook profile picture. Last month the Washington Post reported on Facebook's move to score the "trustworthiness" of users on a scale of zero to 1, while ratings across services such as AirBnB, Uber and eBay have direct, real-world consequences. It would seem that the key to avoid slipping into a Chinese style system is to stop this data – along with health information, criminal records and much else besides – being pooled together and analysed. But isn’t it the case that smart cities of the future, which are being designed to help governments govern and citizens thrive, relies on just that? “The smart city concept in China is different to the smart city concept in Europe,” says Dr Hoffman, “but that doesn’t mean that smart cities in Europe and elsewhere do not carry risks. They certainly do. There are questions surrounding the suppliers of the technology; the fact that [Chinese firm] Huawei operates in other countries carries risks – not only national security risks but also threats to civil liberties.” In the modern world, convenience is often gained at the expense of privacy. But right now, Chinese citizens look to be paying more than most. _________________________ Read more: Move over video: why podcasts and audio continue to make waves Ahed Tamimi’s face has launched a million clicks: the hefty price of expression in Palestine How Facebook changed Instagram, and then the app changed our world _________________________

Big brother: China's data-driven Social Credit system sounds like a sci-fi dystopia

Whether it’s by Facebook, Uber or Reddit, we’re all being scored without knowing how or why. But what if it was the state using facial-recognition software to rate us, reward us – and punish us? Welcome to the future of China