The integrating authority is the single organisation responsible for the sound conduct of the statistical data integration project. It is responsible for the implementation of the data integration project and the management of the integrated datasets throughout their life cycle, ensuring full compliance with commitments made to data custodians, and in line with the high level principles and supporting governance and institutional arrangements. The integrating authority is also responsible for providing researchers with safe and secure access to the integrated data in line with the requirements of data custodians.
To comply with High Level Principle 3, a responsible integrating authority must be nominated by the data custodians for each statistical data integration project involving Commonwealth data.
Integrating authorities must be a secure and trusted institution. In addition, the governance arrangements agreed by the Portfolio Secretaries require integrating authorities undertaking high risk projects (see risk framework) to be accredited. For more information about when these arrangements apply see, Scope of the Commonwealth Arrangements. Accredited Integrating Authorities are assessed by the Oversight Board as having the infrastructure and capability to undertake high risk data integration projects by meeting a set of criteria agreed by the Commonwealth Portfolio Secretaries.
Integrating authorities nominated by data custodians to undertake medium or low risk data integration projects for statistical and research purposes must demonstrate to data custodians that the policy environment in which they operate provides support to prevent disclosure of identifiable information. In particular, employees and officers of the integrating authority must understand and value the importance of protecting identifiable data, and understand the impact any breaches would have on the trust the public places in data custodians of Commonwealth data.
Integrating authorities need to provide assurance to the data custodians that they are able to comply with Principle 6 which states that ‘policies and procedures used in data integration must minimise any potential impact on privacy and confidentiality’ and that they have the capacity to manage risks of both direct and indirect identification, particularly in terms of units with unusual characteristics. This management must take account of the potential increase in identifiability of one set of data when combined with another set.’ Principle 7 concerns the transparency of a data integration project, so the policy environment in which integrating authorities operate should be transparent to the public, this includes registering all data integration projects on the National Statistical Service website.
The establishment of integrating authorities is a key initiative of the Commonwealth arrangements. Under previous approaches researchers were often responsible for data merging and data access. Under the Commonwealth arrangements for data integration, integrating authorities carry out these tasks (see Role 3 below) as part of their responsibility for the end-to-end management of statistical integration projects involving Commonwealth data. Integrating authorities have a different purpose to data linkage units, as the latter have been established for the purpose of creating linkage keys only.
Integrating authorities, in accordance with the requirements of data custodians, are responsible for:
- the management of the data integration project on behalf of data custodians;
- minimising privacy concerns associated with the use of data once it is received from data custodians and after integration; and
- facilitating the use of the integrated data within the constraints of privacy and legislation.
For more information on what an integrating authority should consider when undertaking a data integration project involving Commonwealth data, see A checklist for integrating authorities.
Integrating authorities have four main roles:
Role 1 • Enter into project and data access agreements
The purpose of project agreements is to ensure that datasets are managed in accordance with data custodian requirements, protecting privacy and confidentiality. Data custodians are individually accountable for the security and confidentiality of the source data and must ensure that this accountability is exercised through the conditions specified in the agreement with the integrating authority. Project agreements set out the terms and conditions that accompany the final project approval and are a key mechanism for ensuring that there are adequate control and risk mechanisms in place.
In addition, agreements between integrating authorities and data users (researchers) for access and use of the integrated dataset ensure the datasets are securely accessed, used and disseminated in accordance with data custodian requirements to protect the privacy of data providers.
When the data custodian and the integrating authority are the same entity, appropriate internal governance arrangements, rather than an agreement will need to be in place.
For further information about Drafting Project Agreements see:
- Project agreements - Data custodians and integrating authorities
- Project agreements - Integrating authorities and data users
- Agreement templates
For further information about other responsibilities of the integrating authority during the project application and approval stage see:
- Appoint an integrating authority and assess project feasibility
- Finalise project details
- Final approval
Role 2 • Implement safe and effective arrangements for data integration projects
Integrating authorities are responsible for ensuring that they have safe and effective arrangements in place for undertaking data integration projects that involve Commonwealth data. There are several key areas to consider to achieve this, namely:
- a protective policy environment;
- technical infrastructure and expertise;
- the separation principle; and
- consortium arrangements (see Outsourcing or working in partnership).
Role 3 • Manage datasets for the duration of data integration projects
The integrating authority is responsible for the ongoing management of the integrated data, ensuring that the data is kept secure, confidential and fit for the purposes for which it was approved. If the integrated dataset is retained (for example if the project is ongoing or the integrated dataset is required to support a family of projects [1. Data integration projects using the same source datasets, for similar purposes, with the same integrating authority, are referred to as a family of projects. ] ), the integrating authority is responsible for initiating and managing its regular review, in consultation with source data agencies.
For further information about the role of managing datasets see:
Role 4 • Provide transparency in operation
Transparency ensures that the public is aware of how Australian government data is being used for statistical and research purposes. This is achieved by conducting the statistical data integration project in an open and accountable manner (see Principle 7 High Level Principles for Data Integration Involving Commonwealth Data for Statistical and Research Purposes). Integrating authorities achieve this by registering all Commonwealth data integration projects on the public register, publication of relevant protocols and policies that are in place, conducting regular audits and review processes to evaluate operations, stakeholder communication, and consultation and collaboration.
For further information see:
For some helpful background resources see: Technical resources for integrating authorities
For roles and responsibilities of key players in a data integration project see: