For lawyers using AI the risks are high and so Australian-based law firm Allens, who have a close relationship with Magic Circle leader Linklaters, have looked at the issue around using generative AI for Australian law. The lessons learned may well translate into multiple jurisdictions when the challenges and opportunities presented by AI.
Allens have been heavily involved in developing legal AI applications, launching ‘Airlie’ as their own iteration of ChatGPT.
The Allens article discusses the recent advancements in generative artificial intelligence (AI) tools, powered by large language models (LLMs), and their potential impact on legal operations.
However, the effectiveness of these AI tools in the legal domain remains largely speculative and anecdotal, say Allens in their report.
While AI’s ability to identify patterns in large volumes of data and generate optimal sentences or phrases seems desirable for lawyers, the limitations of generative AI in providing legal advice are well-known, as it faces challenges in replicating human lawyers’ judgement, which plays a crucial role in legal practice.
To systematically test and track the developments in generative AI’s ability to answer legal questions, Allens, in consultation with Linklaters LLP, has developed the Allens AI Australian Law Benchmark (Allens AI Benchmark).
This benchmark tests the ability of market-leading LLMs, as of February 2024, to answer legal questions under Australian law. The tests approximate how a lay user might try to answer legal questions using AI, rather than a human lawyer.
Allens highlight the need for a repeatable benchmark to evaluate the performance of generative AI tools in the legal domain, specifically in answering legal questions under Australian law, and use the Allens AI Australian Law Benchmark as a solution to this need.
Read More –
