The Information Commissioner’s Office (ICO) has published Guidance on how businesses can ensure that their use of big data is compliant with data protection law.

What is big data?

‘Big data’ is a term used to describe the analysis of huge datasets which often bring together data from different sources. Big data is commonly used by businesses looking to personalise their products or services to customers’ individual characteristics, such as insurers using telematics data about driving habits to individualise the premiums they set.

If big data involves analysing personal information, such as data from social media or loyalty cards, the processor of the information will have to comply with the Data Protection Act 1998 (DPA). However, if the data is effectively anonymised, so that it is no longer personal data, the DPA will not apply


The Guidance is grounded in the concept of fairness. This is an important consideration even when data is merely being used to analyse trends but a rigorous consideration of fairness is even more important if the analytics are being used to make decisions affecting individuals. Transparency about how the data will be used is highlighted as an important element in assessing whether big data analytics is fair. The Guidance also states that fairness involves a wide assessment of whether   the processing is within the reasonable expectations of the individuals concerned, taking into account the reason that people are using the service concerned.

The ICO emphasises that while big data analytics is complex, this is not an excuse for non-compliance with the DPA. Organisations must find innovative ways to explain the benefits of the analytics and present users with a meaningful choice – and respect that choice - when they are processing their personal data.

DPA  Conditions  for  Processing

The Guidance highlights the conditions for processing under the DPA which are most likely to be relevant when processing big data. These include:

  • consent;
  • the processing is necessary for the performance of a contract an individual has entered into; and
  • the processing is necessary for the purpose of legimate interests pursued.


If an organisation is relying on people’s consent as the condition for processing their personal data, then that consent must be freely given, specific and informed. This means  people must be able to understand what the organisation is going to do with their data and there must be a clear indication that they consent to it. Big data increasingly uses observed, derived and inferred, rather than provided, data; individuals may be unaware that this data is being collected and  processed.

The Guidance reminds organisations that if they collect data for one purpose and then that data is subsequently used   for another purpose, they will need to notify the individuals concerned and seek consent for the new use. It will also need to assess whether the new processing is incompatible with the original purpose for which the data was collected. Furthermore, the consent would not meet the standard required by the DPA if people do not have a real choice and are not able to withdraw their consent if they wish.

If an organisation buys a large dataset of personal data for analytics purposes, then it becomes a data controller in respect of that data. The organisation needs to be sure that it has met the DPA conditions for the further use of that data. If it is relying on the original consent obtained by the supplier it should ensure that this covers the further processing it plans for the data.

Performance of a contract

Specific consent is not required where the processing is necessary for the performance of a contract to which the data subject is a party. The problem of applying this in a big data context is that the processing must be “necessary”. Big data analytics, by its nature, is likely to represent a level of analysis that goes beyond what is required simply to sell a product or deliver a service.

Legitimate Interests

The processing may be necessary for the legitimate interests  of the organisation collecting the data. An organisation may have a number of legitimate interests that could be relevant including profiling customers in order to target its marketing or preventing fraud of the misuse of its services. However, having established that it has a legitimate interest, the organisation then has to carry out a balancing exercise between those interests and the rights, freedoms and legitimate interests of the individuals concerned.

Data Minimisation

The Guidance highlights that data protection legislation embodies the concept of ‘data minimisation’; in other words, that organisations should minimise the amount of data that they collect and process. In contrast, a key feature of big data is that all available data, rather than a sample, is used. Organisations, therefore, should address from the outset what they expect to learn from their research, and ensure the data used is relevant and not excessive for that aim. If organisations wish to retain data for long periods for reasons of big data analytics they should be able to articulate and foresee the potential uses and benefits to some extent, even if the  specifics are unclear.

An ethical approach to data processing

There is evidence that some companies are looking to place big data in a wider and essentially ethical context,  addressing the question of not only of whether their data processing meets regulatory requirements but also whether that processing is what customers should expect as part of the service they receive. The Guidance further comments that wholesale adoption of such an approach by high-profile companies may well lead to privacy being adopted as a brand value across sectors. Adopting an ethical approach will also  go some way towards ensuring that the analytics complies with data protection principles.

The Guidance points out that an ethical approach by organisations towards data processing would highlight the importance of educating people as citizens and consumers. This means explaining the benefits of the analytics and looking to foster a value exchange, in which people are happy to provide data if they are informed and have trust in how it will be used.

What next?

In its Guidance the ICO has underlined the fact that big data is ‘not a game played by different rules’; it is subject to established data protection legislation. The ICO has   also made clear that many of the challenges of compliance can be overcome by being open and transparent and that organisations need to think of innovative ways to tell customers what they want to do and what they’re hoping to achieve.  Looking for a value exchange may be the way forward.