Review of Government Use of Algorithms (NZ0019)

• Open Algorithms: Experiences from France, the Netherlands and New Zealand (June 1, 2020)

• Reflections on the Implementation of Public Sector Algorithmic Policy (December 1, 2020)

Commitment 8: Review of government use of algorithms
Objective:
To increase the transparency and accountability of how government uses
algorithms.
Ambition:
To ensure New Zealanders are informed, and have confidence in how the
government uses algorithms – automatic decision-making processes used by
computer programmes – to identify patterns in data.
Status quo:
Advances in digital and data environments are changing how we live,
work, and interact. These changing environments present considerable
opportunities to improve the lives of citizens, but also present challenges to
ensure that data is used appropriately.
Tools such as data modelling and algorithms have tremendous power to
improve lives. Despite these tools being increasingly used by government
agencies to support decision-making, there is no agreed cross-government
approach to algorithms or the decisions they support.
The Chief Data Steward (Chief Executive of Stats NZ) recently published
the Principles for the Safe and Effective Use of Data and Analytics with the
Privacy Commissioner. Using these, the Chief Data Steward is undertaking
a cross-government review with the Chief Digital Officer (Chief Executive
of the Department of Internal Affairs) to increase the transparency and
accountability of how government uses algorithms – to improve the lives
of New Zealanders. The initial focus will be on operational algorithms that
result in, or inform, decisions directly impacting individuals or groups.
Having guidance, oversight and transparency in place is essential for New
Zealanders to understand how their personal data is used, and for fostering
trust, confidence and integrity around the use of data the Government
holds on their behalf. Lead agency: Stats NZ (working with Department of Internal Affairs
(Government Chief Digital Officer))
Timeline: October 2018 – June 2020
Commitment 8: Review of Government use of Algorithms
OGP Values Transparency,
Accountability
Verifiable and measurable milestones to fulfil
the commitment
Start date End date
Complete an initial review of existing operational
algorithms and their use across a range of
government agencies7 8
1 June 2018 31 October
2018
Consider next steps for all-of-government assurance
related to the use of algorithms in collaboration
with Civil Society representatives
31 October
2018
28 February
2019
Update this commitment (and its milestones) to
reflect progress made in previous milestones
1 March
2019
31 March
2019

8. Review of government use of algorithms [90]

Objective: To increase the transparency and accountability of how government uses algorithms.

Milestones:

1. “Complete an initial review of existing operational algorithms and their use across a range of government agencies”;
2. “Consider next steps for all-of-government assurance related to the use of algorithms in collaboration with Civil Society representatives”;
3. “Update this commitment (and its milestones) to reflect progress made in previous milestones”.

Start Date: June 2018

End Date: June 2020

Context and Objectives

The commitment’s objective and ambition are to increase transparency and accountability around government’s use of algorithms by ensuring the public is informed about and has confidence in their use. It notes that there is no agreed cross-government approach to algorithms or the decisions that they support. This supports views expressed by submitters and participants during the development of the action plan that government’s use of algorithms needed to be more transparent and accountable, [91] with one commenting that “it is disempowering to citizens to be subject to digital processes that are opaque and may contain hidden biases and assumptions”. [92] The feedback reinforces the importance of the topic and the mutual desire to increase transparency and accountability.

This is work that Stats NZ and the Government Chief Digital Officer started in May 2018 following advice to Ministers that many government agencies already use data modelling and algorithms to assist with decision making; there is a need to ensure algorithms are used appropriately, ethically, and free from bias; and there is no centralised record of these tools and the decisions that they support. [93] As this work is a new field its initial scope is limited and needs to be broadened and specified based on the initial review. It meets OGP’s value of civic participation as the commitment will collaborate with civil society representatives to consider the next steps regarding the use of algorithms.

The work involves completing an initial review, [94] working with civil society representatives to consider the next steps for all-of-government assurance of the use of algorithms and subsequent actions arising from progress of Milestones 1 and 2. Following Stats NZ’s advice that they will report publicly on Milestones 2 and 3 and that Milestone 3 is a holding place for adding subsequent activities through to June 2020, [95] the IRM researcher concludes the milestones are specific enough to verify.

External algorithm experts advised the IRM researcher that the initial review’s approach for 14 agencies to self-identify their algorithms was minimal and lacked depth. [96] Another noted the lack of a full list, the omission of some important algorithms, the need for more public debate about the ethics, and requested a specific timeline for reporting to Ministers. [97] Looking ahead to milestones 2 and 3, these stakeholders expect government policy mandating that all algorithms used by government are known, catalogued in a register and transparent; work done to increase officials’ capability to understand what algorithms do and develop skills to update them; and that government has an open first policy with guidelines for open algorithms across government. [98] Others sought assurance that this work would achieve the trust sought by government, and were pleased it would build on the Principles for the safe and effective use of data and analytics, released by the Privacy Commissioner and the Government Chief Data Steward in May 2018. [99] They supported an auditable process assessing risk and other security-related factors. Another noted that the Data Futures Partnership, which gave advice in this area, is no longer operating as it was established for a specific length of time and felt that this commitment as written does not engender trust. [100] Others noted artificial intelligence (AI) often revolves around the use of algorithms and wish to know if this review covers AI. [101] The focus of the commitment is on increasing the transparency and accountability of algorithms however, and the IRM researcher was advised on 12 August 2019 that ‘there is currently no capacity to extend the scope of this work to cover AI.” [102] All supported wider public engagement.

If fully implemented as designed, the commitment would have minor potential impact as the milestones as stated cannot achieve the ambition statement nor deliver the outcomes Ministers agreed to in May 2018. Transformative reform would require a programme to train officials to understand what algorithms do and how to update them, and the wider public engagement and detailed cross-government policy and implementation work identified by stakeholders. Milestone 3 in the action plan enables Stats NZ to consider the next steps in consultation with civil society representatives and to establish further milestones.

Next steps

If this commitment is carried forward to the next action plan or if there are improvements to the implementation of this commitment, the IRM researcher recommends work to:

• document all algorithms used by government and make the register publicly available;
• engage widely with the public during this work; and
• develop and implement policy, guidelines and training requiring open algorithms across government.

8. Review of government use of algorithms

Substantial:

Aim of the commitment

This commitment aimed to increase transparency and accountability around the government's use of algorithms by ensuring the public is informed about and has confidence in their use. Initially, it would review existing operational algorithms and their use across a range of government agencies, consider the next steps for all-of-government assurance in collaboration with civil society representatives, and then update the commitment to reflect progress made. [100]

Did it open government?

Marginal

While this commitment is complete, change in government practice is marginal at this stage, with a focus on gaining agency buy-in. Over 2018 and 2019, Stats NZ collected algorithm details from agencies, released the initial Algorithmic Assessment Report, [101] consulted public service agencies on their response to the report’s recommendations, and released a draft Algorithm Charter for a two-month public consultation phase. [102] In February 2020, Stats NZ released a summary of the public submissions, [103] and in July 2020 the Minister of Statistics released the voluntary Algorithm Charter for Aotearoa New Zealand (the Charter) [104] with 21 government agency signatories. [105] The Charter, published in both English and te Reo Māori, [106] sets standards for safe and ethical use of algorithms by public-sector agencies and guidance for meeting transparency and accountability objectives.

By 28 October 2021, the number of signatories committed to applying the principles of the Charter in their work had significantly grown to 27, with more agencies expressing interest in signing up. The first Charter review was due for completion in November 2021 and its report will be published. Its Terms of Reference state that the Charter is a voluntary, self-regulating agreement that is not legally binding and has no enforcement mechanisms or formal process for monitoring signatory agencies’ compliance. [107] The review’s objective is to learn from the first year of the Charter’s implementation, including the experiences of agencies, any early indications of positive impacts or unintended consequences, the support needs of signatories, embedding te ao Māori perspectives and reflecting the principles of the Treaty of Waitangi / te Tiriti o Waitangi, and the relationship between the Charter and developments in addressing Māori data sovereignty considerations.

Strong concern was expressed in August 2020 that the Charter does not fully address important considerations, such as Māori Data Sovereignty, with comment that the Charter “has the potential to further discriminate against Māori than the current biases of Data, Algorithms and Machine learning already does.” [108] Media commentators also sought consideration of Māori data sovereignty in the Charter, proposed formal oversight, and offered implementation suggestions. [109]

The Charter has seen international interest, with Stats NZ and Transparency International NZ invited to join the pilot cohort of the OGP Leaders Network, in recognition of the Charter, [110] There is online evidence of uptake by the Ministry of Justice, [111] the Accident Compensation Corporation, [112] the Ministry of Social Development, [113] and NZ Police. [114] Stats NZ reports in the Commitment’s end-of-Term report that signatories have been making steady progress in implementing the commitments, although this differs considerably between agencies depending on their data maturity and use of algorithms. It expects the review to provide more information on how much the Charter has improved algorithmic transparency, as well as the successes and challenges agencies have faced in implementing the Charter. [115] At present, change in NZ government practice is at an early stage.

Academic and CSO stakeholders report that the changes to the draft charter after consultation reflected government not community feedback. They were pleased that the final “watered down” version had broad buy-in by agencies and had successfully gained a large number of signatories. They seek more depth in the Charter over time as people become used to it. [116] Civil society stakeholders are pleased that agencies like the NZ Police are now signatories and that the initial work created broad awareness of the Charter in everyday governmental discussions. They also note that the Chief Data Steward has no enforcement power across government, that the Charter is being applied differently across agencies, that none have published a catalogue of the algorithms they are using, and that the Charter’s implementation support document provided to the Minister in 2020 has not been published. [117] The government’s reliance on third-party vendors is seen as a weakness, and training for agencies and vendors would be helpful. [118]

Stakeholders have high expectations that the recommendations from the current review will set the scene for active operationalisation and expansion of the Charter. They seek clarity about cross-government leadership, oversight, monitoring, and appropriate data management. They see these as the key requirements for fully achieving this commitment’s ambition to ensure that New Zealanders are informed and have confidence in how the government uses algorithms to identify patterns in data.