People Focused Refinement of Ontario’s Artificial Intelligence and Algorithms Inventory through User Research (CAON0002)

Commitment ID

CAON0002

Commitment Title

People Focused Refinement of Ontario’s Artificial Intelligence and Algorithms Inventory through User Research

Problem

The Data Catalogue includes AI and algorithmic assets, but mere inclusion is not enough. If the assets are listed, but inadequate or unhelpful metadata is provided, accountability, risk communication and redress cannot be supported. The metadata requirements for ‘Artificial Intelligence and Algorithms’ assets should not be expected to be the same as they are for other data assets, unique metadata needs to be developed.

Status quo

Ontarians are concerned about government AI use and want increased transparency. Our consultations showed an interest in a clear definition of AI assets and increased access to details about government AI systems, increased metadata availability, and even source code.

Currently there are eight ‘Artificial Intelligence and Algorithms’ assets included in the Ontario Open Data Catalogue, as required by Ontario’s Digital and Data Directive and the Simpler, Faster, Better Services Act (2019). These assets are described using existing data focused metadata, plus a template for their descriptions including recommended disclosure elements for AI.

Without consultation and investigation into data catalogue user experience we do not have a clear understanding of how effective this early inclusion of algorithms and early AI assets has been on Ontario’s data catalogue.

To date, no feedback on users’ experience of these ‘Artificial Intelligence and Algorithms’ assets has been received or considered in our data catalogue’s development roadmap. To get a clearer understanding of what the Ontario public would like to see in an Artificial Intelligence and Algorithms Inventory, we need to understand the current catalogue user experience.

Action

The objective is to improve the Artificial Intelligence and Algorithms Inventory user experience on the Ontario Open Data Catalogue. First, we will find out if the government is disclosing what users want to know about AI. This will be accomplished through an initial survey to evaluate how the catalogue is perceived and web analytics to determine current catalogue use. This will be followed by user research that considers adding features, to determine what Ontarians would like to see in the catalogue to meet concerns of transparency, public accountability, plain language, human rights impact and risk assessments, ongoing monitoring, and potential redress mechanisms.

Finally, the results of this review will be published and the Ontario government will commit to making reasonable changes to the data catalogue.

How will the commitment contribute to solving the public problem described above?

The survey and user research will establish current opinion on and desired changes to the AI and Algorithms Inventory and how the Ontario Digital Service can best revise the catalogue standards to serve the public. We will improve the user experience of the Artificial Intelligence and Algorithms Inventory by committing to making changes based on our user research. The baseline data collected will support the assessment of enhancements.

What long-term goal as identified in your Open Government Strategy does this commitment relate to?

Ontarians seeking information on government AI use will be supported by an improved Artificial Intelligence and Algorithms Inventory. We can provide catalogue users with the information they need to help them understand how, when and why the government is using AI. Making key information available and accessible will facilitate increased AI literacy contributing to increased transparency.

Primary Policy Area

Digital Governance

Primary Sector

Other/NA

What OGP value is this commitment relevant to?