Can ChatGPT Write Laws? Lawyer Puts the AI Chatbot to the Test

A recent study has focused to an intriguing question: can artificial intelligence (AI) effectively write laws?

This research conducted by Associate Professor Guzyal Hill from Charles Darwin University (CDU) delved into the capabilities of OpenAI's widely used AI tool ChatGPT in analyzing and producing domestic violence legislation, aiming to assess its quality compared to human legal experts, particularly those represented by the Australian Law Council.

Can ChatGPT Write Laws?

The complexity of domestic violence as a deeply human issue, coupled with the increasing prevalence of AI, prompted Hill to embark on this investigation.

With up to 50 women losing their lives to domestic violence annually in Australia, the need for robust legislation to address this societal challenge is paramount. Hill's study involved subjecting ChatGPT to rigorous testing, comparing its output to that of the Australian Law Council.

While ChatGPT demonstrated utility in classifying and identifying patterns of domestic violence, the study revealed that human drafting remains superior in terms of producing high-quality legislation.

ChatGPT's references often drew from US law, highlighting the importance of contextual understanding and the nuances of legal systems across jurisdictions.

AI vs. Human Expertise in Law

As a lawyer and former legislative drafter, Hill emphasized the necessity for further research to explore AI's role in the legal profession. For legal practitioners and students alike, upskilling in AI is imperative.

In a statement, Hill noted that ignoring or misusing AI could lead to unforeseen drawbacks and risks, including the potential disregard for fundamental human rights and principles of the rule of law.

While AI systems like ChatGPT offer augmentation to human analysis and reasoning, they do not replace the need for human expertise in legal drafting.

Instead, Hill noted that AI presents an opportunity for collaboration and innovation in the legal domain, potentially transforming the way law approached globally.

'Not Yet'

"After running several tests and comparing with the definition produced by the Australian Law Council, the answer is 'not yet'-human drafting is still superior. ChatGPT, however, was very useful in classifying and identifying underlying patterns of types of domestic violence," Hill said in the statement.

"For non-lawyers, ChatGPT and similar LLMs should never be used for legal advice. A lot of ChatGPT references include the US law. Law in Australia is simply different, not even talking about differences between, say, Queensland and South Australia. I have noticed ChatGPT now includes a disclosure that it cannot provide legal advice," she added.

The study, titled "AI-Legislation for AI, Legislation with AI: Round No 1 ChatGPT v Australian Law Council", was published in The Loophole.