Last week saw Alan Turing announced as the new face of the £50 note, as well as helping to crack the Enigma code, he is credited with being the father of computer science and artificial intelligence. It is apposite then that on the same day the UK's Financial Conduct Authority announced that it was collaborating with the Alan Turing Institute on the use of AI in the financial sector and later the same week the UK Centre for Data Ethics & Innovation published its interim reports, ahead of its final recommendations in December.
The UK Government, as well as several regulators, is grappling with concepts such as transparency, accountability and explainability against the backdrop of an acceleration in the capabilities of data-driven technologies. CDEI was set up to help navigate these new ethical challenges and inform the Government's thinking on the right governance regime. The interim reports, one on bias in algorithmic decision making and, the other, on online targeting detail the CDEI progress and emerging insights.
Bias in algorithmic decision making
Although this is an issue across all sectors, the CDEI is currently focusing on four including, in the private sector, financial services and recruitment. For financial services, the focus will be on credit and insurance decisions taken about individual customers. In particular using data from non-traditional sources such as social media and emerging machine learning approaches. For recruitment, the focus is on the use of algorithms to automate (or partially automate) recruitment decisions. This could range from bulk screening of CVs and applications, to providing recommendations to human decision-makers, for example, on who to invite to interview, to using artificial intelligence to analyse candidates’ performance in interviews.
The key concern that the CDEI is looking at is the potential, as algorithms become increasingly complex, for existing biases to become entrenched or worsened. Whether through the data input, the design of the algorithm or the way outputs are acted on by humans, it is clear that minimisation of bias in algorithms will require understanding of some difficult concepts and some difficult choices.
Emerging points to watch:
- Although the exact nature of oversight will vary according to the sector, effective human accountability will be key and something boards will want to keep a close eye on.
- There is an understanding that the impression, or existence, of bias is often context specific and that perceptions of fairness are not always consistent. This means there will be significant value judgements to be made by decision makers.
- It is likely algorithms will need to be blind to protected characteristics while also checking for bias against those same characteristics.
- It is acknowledged that it will not be possible to remove all bias from a decision, but where the trade-off between accuracy and fairness will be set may well be one that businesses will have to consider carefully and be prepared to justify.
The online targeting review is broad and encompasses any technology used to analyse information about people and then, automatically, customise their online experience. This includes targeted online advertising, recommendation engines and content ranking systems. The CDEI's focus is on what are perceived to be the high risk sectors including targeted news and information, media, user generated content, advertising, retail and public services.
Whilst the CDEI recognises that the underlying technology in online targeting is enormously beneficial in navigating the internet it is clear there are public concerns with the current model. The key concern is that although people can see the benefit in personalising their online experience the more they understand about it how it works the less likely they are to think the current practices are acceptable. There is a fear that the systems have become so pervasive that they no longer simply predict our existing beliefs and desires, they are starting to shape them. Amongst the greatest areas of concern are the potential to exploit people's vulnerabilities and the impact of targeting on trust in information/markets.
Emerging points to watch:
- Although it does not appear that the CDEI would recommend new regulation without strong evidence and justification, there are clear signals they will be recommending exactly that in December. Likely areas include:
- businesses being more open about the process used to determine the acceptability of targeting algorithms and how the system works.
- regulation of the process of targeting including the inferences that can be made and used in targeting processes, or the types of targeting that can be undertaken.
- stronger obligations on organisations to protect against vulnerability.
- strengthening individual powers eg consent, transparency and facilitating data portability.
- enhancing competition for example through the use of data trusts.
There are some interesting themes emerging from these interim reviews and clear signals of the future of data regulation however these reviews are just part of the regulatory landscape. The Information Commissioner, the Competition and Market's Authority and the Financial Conduct Authority are all in the process of gathering evidence about different aspects of data regulation and the UK Government has proposed a new regulator specifically for Online Harms. It is not clear yet what form regulation will take but in the words of Alan Turing, “We can only see a short distance ahead, but we can see plenty there that needs to be done.”.