Artificial intelligence and automated decision-making – Some guidance for getting it right

The Royal Commission into the Robodebt Scheme handed down its report on 7 July 2023 (Robodebt Report). The Robodebt Report includes 57 recommendations including a number of recommendations relating to ADM. While there has been much discussion recently about the potential opportunities presented by AI, particularly generative AI, the Robodebt Report comes as a stark reminder of the potential risks of AI and the likelihood of regulatory reform as the Australian Government looks to mitigate those potential risks.

The Robodebt Scheme refers to an automated debt assessment and recovery scheme employed by the Australian Government which resulted in hundreds of thousands of welfare recipients being wrongly accused of owing money to Centrelink. The Robodebt Report has made a number of adverse findings in relation to the Robodebt Scheme and recommended the referral of unnamed individuals for civil action or criminal prosecution. The Robodebt Scheme has also separately resulted in the Australian Government settling a class action lawsuit for over AUD 1.8 billion.

Recommendations relating to ADM in the Robodebt Report

As mentioned above, the Robodebt Report includes a number of recommendations relating to ADM. In particular, the Robodebt Report has recommended that the Australian Government should consider legislative reform to introduce a consistent legal framework in which automation in government services can operate.

Where ADM is implemented, the Robodebt Report has recommended that:

• there should be a clear path for those affected by decisions to seek review;
• departmental websites should contain information advising that ADM is used and explaining in plain language how the process works; and
• business rules and algorithms should be made available, to enable independent expert scrutiny.

The Robodebt Report has also recommended that the Australian Government should consider establishing a body, or expanding an existing body, with the power to monitor and audit ADM processes with regard to their technical aspects and their impact in respect of fairness, the avoiding of bias, and client usability.

Broader work to regulate AI and ADM in Australia

The Robodebt Report has been handed down at a time when the Department of Industry, Science and Resources is currently consulting on safe and responsible AI in Australia more broadly and has published a discussion paper on this topic in June 2023 (AI Discussion Paper).

The AI Discussion Paper noted:

• that while the regulation of AI remains in an early state globally, there is a developing international direction towards a risk-based approach for governance of AI, with more advanced developments in the European Union and the United States.
• that Australia’s current approach to managing the potential risks of AI relies on a combination of:
  • a broad set of general regulations that are mainly technology neutral;
  • sector-specific regulation; and
  • voluntary or self-regulation initiatives such as ethical principles for AI.
• before canvassing a broad spectrum of possible responses to the governance of AI, together with a possible risk management approach, that it is proposed that the governance measures adopted by Australia will be guided by the need to:
  • ensure there are appropriate safeguards, especially for high risk applications; and
  • provide greater certainty and make it easier for businesses to confidently invest in AI and ADM and engage in these activities responsibly.

The AI Discussion Paper welcomed feedback on how the Australian Government can mitigate any potential risks of AI and support safe and responsible AI practices. Public consultation on the AI Discussion Paper closed on 4 August 2023 but submissions have not yet been published.

ADM is also a topic that is being addressed as part of the Privacy Act Review.

In particular, the Privacy Act Review Report, published by the Attorney-General’s Department in February 2023, relevantly proposed that:

• privacy policies should set out the types of personal information that will be used in substantially automated decisions which have a legal or similarly significant effect on an individual’s rights (Proposal 19.1);
• high level indicators of the types of decisions with a legal or similarly significant effect on an individual’s rights should be included in the Privacy Act and this should be supplemented by OAIC guidance (Proposal 19.2); and
• a right for individuals to request meaningful information about how substantially automated decisions with legal or similarly significant effect are made should be introduced and entities should be required to include information in privacy policies about the use of personal information to make substantially automated decisions with legal or similarly significant effect (Proposal 19.3).

Additionally, in relation to targeting in an online context, the Privacy Act Review Report relevantly proposed that there be:

• an unqualified right to opt-out of receiving targeted advertising (Proposal 20.3);
• a prohibition on targeting to a child, with an exception for targeting that is in the child’s best interests (Proposal 20.6);
• a requirement that targeting of individuals is fair and reasonable in the circumstances (Proposal 20.8); and
• a requirement to provide individuals with information about targeting, including clear information about the use of algorithms and profiling to recommend content (Proposal 20.9).

Public consultation on the Privacy Act Review Report closed on 31 March 2023. For an overview of the proposals included in that report, please see our previous article here.

How we can help your business

Please get in touch with our team if your business requires:

• assistance in preparing or updating internal policies and guidelines relating to its use of AI and/or ADM, including generative AI tools (for more on this, see an article published by our UK colleagues here);
• assistance in updating its privacy policy and practices in anticipation of the pending reforms; or
• any other advice relating to its use of AI and/or ADM, including generative AI tools.