As we recently reported, the Australian Government has released the Productivity Commission’s final report into Data Availability and Use.

Like other commentators, the Commission concludes that data is too important as a strategic asset, and has too much potential as an asset, to be shackled by a lack of trust by both data custodians and users – that lack of trust is ‘choking the use and value’ of Australian data for governments, industry and consumers.

This message is likely to resonate with the Turnbull Government’s innovation agenda – particularly the Government’s open data policy, which ‘recognises that Australian Government data is a strategic national resource that holds considerable value for growing the economy, improving service delivery and transforming policy outcomes’ and commits to optimising the use and reuse of public data, including releasing non-sensitive data ‘by default’.

The release of the report coincided with Commonwealth Budget announcements of funding (through the Data Integration Partnership of Australia) for transforming the analysis of public data by integration of data across government and providing access via a single entry point to reduce duplication, encourage efficiency and lead to long‑term reform in data collection and use.

A taskforce has been established within the Department of the Prime Minister & Cabinet to formulate the government’s response to the recommendations.

The Government’s response to the report will be a clear sign about how the Government will give practical effect to its policy direction.

So what impacts would the recommendations have?

A new Act

The report concludes that ‘marginal changes to existing structures and legislation will not suffice’ and that Australia should move from a system based on risk aversion and avoidance to one based on transparency and confidence in data processes, treating data as an asset and not a threat.

So instead of piecemeal reform, the report’s recommendations focus on developing a complex and ambitious legislative and administrative package through a comprehensive Commonwealth Act – a Data Sharing and Release Act (DSR Act).

The DSR Act would:

  • create a new Office of the National Data Custodian (NDC) (together with a small advisory board). The NDC would, among other things
    • have a key role in identifying National Interest Datasets (NIDs) (which may include publicly held or privately held data) that must be immediately released – a Commonwealth Parliamentary Committee would be involved in scrutiny of nominations for NID status
    • have an advisory and complaints handling role
    • have the role of accrediting selected public sector and public interest entities as “Accredited Release Authorities” (ARAs). These would decide whether a particular dataset is to be made available for public release or whether release should be limited (on a risk basis) to “trusted users” within the context of published formal risk management processes.
  • create a “Comprehensive Right” for consumers in relation to data held about them (including a right to have the data provided to them in machine-readable form).

Critically, the DSR Act would give explicit authority to data custodians and ARAs to share and release data ‘despite the terms of other laws’ as long as it is done to achieve a public interest purpose, in compliance with DSR Act processes and subject to DSR safeguards.

These changes would have significant implications for the management and use of public sector data. If the Commission’s recommendations are accepted, all “non-sensitive publicly funded datasets — whether held by a government agency or other body receiving public funding for data collection activities” would have to be released, and other datasets will have to be risk assessed for release. A mandatory public consultation process would be involved in identifying datasets for release.

National Interest Datasets

The report proposes a three pronged test for NIDs:

  • the dataset has “spillover benefits”, and its use would be likely to generate broader economic and social benefits beyond those accruing to the initial data holders – how this would be assessed in the absence of a specific proposal for use is something that must be worked out during implementation
  • the dataset can be used as a basis for performance evaluation and comparison between programs and investments by the Commonwealth, States or Territories – for example, data collected by the Australian Government as part of administering the Pharmaceutical Benefits Scheme and the Medicare Benefits Schedule, and data on public hospital performance
  • the dataset has a focus on nationally significant subject matter – such as education.

The decision whether a dataset would be a NID is made by the Minister, after a detailed public scrutiny process.

What charges should be applied for sharing or release is addressed by the Australian Government Charging Framework, which is noted in the report.

Importantly, private sector datasets can be designated if it is determined (through this process) that the public interest in designation outweighs the private interest in protecting commercially sensitive information. The report recognises that the Commonwealth may have to acquire those datasets.


The proposals will involve modifying current privacy protections to ensure that the existing exceptions in privacy legislation allowing the sharing with trusted users of personal information for health and medical research purposes, without obtaining the consent of individuals, are extended to cover public interest research more generally.

The Privacy Act already gives everyone a range of rights to ensure that personal information about them is accurate and up to date (having regard to the purpose for which it was collected). The report recommends 2 extra rights, not now in the Privacy Act – a right to be informed when data is disclosed, and a right to a machine-readable copy of the data.

Confidentiality and secrecy

Apart from privacy laws, dataset owners are, in many cases, subject to secrecy laws and to confidentiality obligations owed to third parties (for example, for commercially sensitive data). The Freedom of Information Act, which already gives everyone a legally enforceable right to access Commonwealth Government documents on request (and there are similar laws in each of the States and Territories), recognises these laws and confidentiality obligations.

The general intention of the report appears to be that all these will, where possible, be overridden by the obligation to release and share datasets in the DSR Act, within the context of detailed arrangements established and administered by the NDC and ARAs (regulators), who would address “risks” and ethical considerations raised by release or sharing.

Bringing in the States and Territories

The report recommends that the new Act “cover the field” with respect to digital data but that the scheme not automatically apply to State and Territory held datasets - instead, the States and Territories would be invited to “opt in” to the scheme. This latter approach, while it will ultimately secure State and Territory commitment to a scheme, is likely to take time and risks “watering down” the scheme.

A big bang

The report goes for the “big bang” approach to reform – a substantial new Act.

Australia-wide, there are hundreds of laws, codes, policies, guidelines and other government rules that regulate data use and access. The report proposes that the new DSR Act sit alongside some of these and replace or modify others. The relationship between the Commonwealth laws and the new Act will need to be worked out on a case-by-case basis to ensure that the right balance is struck.

The report does not go into detail about how the NDC, as “facilitator” of improved data access and best practice, will work with existing Commonwealth bodies such as the Digital Transformation Agency, the Australian Bureau of Statistics and the Information Commissioner (Commonwealth) and relevant State and Territory agencies such as Data Analytics Centre (NSW). For the latter, inter-governmental agreement and coordination arrangements will need to be negotiated.

So what now?

The Commission recommends that action should be taken to implement its recommendations sooner rather than later, and suggests that the implementation could be staggered, with a number of initiatives coming into effect by the end of 2017 and the DSR Act to be introduced and in effect by the end of 2018 (with a suitable transition period).

It will be a big job. All agencies will be affected.

A thorough understanding of the implications of the report will be essential if agencies are to make informed contributions to the Government’s decision-making.

How would the recommendations change what you need to do?