Retrieval vs recall: the line between safe and unsafe legal AI

By Michael Nadalin, Founder, Market Lead · 10 August 2026 · 8 min read

A junior hands you a research memo at 4pm on a Friday. It cites four authorities, two of them Federal Court, one Court of Appeal, one first instance. The pin cites look right. The medium neutral citations are formatted correctly. The propositions attributed to each case are exactly the propositions you were hoping to find. You have a hearing on Tuesday.

Here is the uncomfortable question: did anything in that chain actually open those judgments? Not "did it sound like it", but did a document get fetched, opened and read before that sentence was written? Because if the answer is no, the formatting tells you nothing. A citation that was generated to look correct and a citation that was copied from a real judgment are visually identical on the page.

That is the whole distinction. Every argument about whether legal AI is safe or dangerous collapses into one technical question: is the tool retrieving law, or recalling it? Everything else, the interface, the branding, the confidence of the prose, is downstream of that.

Recall: the model is writing from memory, and memory is lossy

A large language model trained on text does not store documents. It stores statistical relationships between tokens. When you ask a bare model for the leading authority on a point of Australian law, it is not looking anything up. It is producing the sequence of words that is most probable given your question and everything it absorbed during training.

Sometimes that produces a correct citation, because the case was discussed thousands of times in the training data and the pattern is strong. Sometimes it produces a citation that is structurally perfect and factually invented, because "a Federal Court case about unconscionable conduct in 2019" is a shape the model knows how to make, whether or not that specific case exists. The model has no internal flag that distinguishes the two. It is doing the identical operation in both instances.

This is why "the AI lied" is the wrong frame, and why telling it to be careful does not fix anything. It is not lying. It is doing exactly what it was built to do, which is produce plausible text. Plausibility and accuracy happen to overlap most of the time, which is precisely what makes the failures so hard to catch.

Retrieval: the model reads a document you can also open

Retrieval flips the order of operations. Before the model writes a word of analysis, a search runs against an actual corpus of legal documents: judgments, legislation, regulations, practice notes. The relevant passages are pulled out and placed into the model's context, and the model is then asked to answer using that text and nothing else.

The model still does the writing. What changes is that the raw material is now real text sitting in front of it rather than a statistical impression of text it saw during training. The task shifts from "recall what the law says" to "read these six passages and tell me what they say", which is a far more constrained job and a far easier one to get right.

It also changes what you can do as the reader. When the source passage travels with the answer, verification stops being an act of faith and becomes a click. You open the judgment, you read the paragraph, you form your own view about whether the proposition holds. That is normal practice. It is what you would do with a memo from a junior anyway.

Why a fabricated citation is so much more dangerous than a wrong one

Lawyers are well practised at handling a case that is distinguishable, superseded or simply weaker than hoped. That is ordinary professional risk and the profession has habits for managing it. Fabrication is a different category, because the error is invisible at exactly the point where you would normally catch it.

Consider the way most people actually check a memo. You scan the citations for familiarity. You notice if a court or a year looks off. You pull the two authorities the argument really turns on. A fabricated citation survives every one of those checks, because it was generated to look exactly like something that would pass them. The one thing that catches it is opening the document, which is the step everyone skips when time is short.

Australian courts have been direct about where responsibility sits. Several have issued practice notes and guidance on the use of generative AI in the preparation of court documents, and the consistent theme is that the practitioner is accountable for what is filed, regardless of what produced the first draft. The tool is not a party to the proceeding. You are.

The two minute test you can run on any tool

You do not need to understand the architecture to work out which kind of tool you are holding. You need one question you already know the answer to, and one question about something obscure enough that the answer is unlikely to be well represented anywhere.

Ask the obscure one first. A retrieval system will either return the relevant provision with a link, or tell you it found nothing on point. A recall system will produce something confident, fluent and specific. Then try to open the source. Not a link to a general homepage, not a case name you have to go and search yourself, the actual paragraph. If the source cannot be opened in one click, the tool is asking you to take its word for it, and there is no reason to.

What to ask a vendor before you sign anything

Most legal AI marketing sits on a careful ambiguity. "Trained on Australian law" sounds like retrieval and usually means recall. "Grounded in authoritative sources" can mean anything. The questions below are the ones that force a specific answer, and the quality of the answer tells you as much as the content of it.

Pay particular attention to what happens when the corpus does not contain an answer. A vendor who has thought hard about this will describe the refusal behaviour without prompting, because building a system that says "nothing on point" is harder than building one that always produces something. A vendor who has not will change the subject to the size of their database.

Retrieval lowers the cost of checking, it does not remove the duty

The honest claim for retrieval is narrower than the marketing usually makes it. It does not make a tool correct. It does not remove the need for professional judgment about whether an authority is apposite, whether it is still good law, or whether the argument holds. A retrieval system can still surface a real case and characterise it badly, and a corpus with gaps will confidently answer within its gaps.

What retrieval does is change the economics of verification. When the source paragraph sits next to the proposition, checking takes seconds rather than the twenty minutes it takes to reconstruct someone else's research from scratch. Verification that is cheap actually gets done. Verification that is expensive gets skipped at 4pm on a Friday, which is exactly when it matters most.

That is the design principle Legal Brain is built on: search real Australian legislation and judgments first, answer from what was found, and put the source paragraph in front of you so the check is a click rather than a project. The judgment about whether the law helps your client stays where it has always been, with the practitioner whose name goes on the document.

Frequently asked questions

What is the difference between retrieval and recall in AI?

Recall means the model produces an answer from patterns absorbed during training, with no lookup at any point. Retrieval means the system searches a real document corpus first, then writes the answer from the passages it found. In legal work the practical difference is that retrieval gives you a source paragraph you can open and check, while recall gives you text that is only as reliable as the model's memory.

Why do AI tools invent case citations?

Because generating text that looks like a citation is the same operation as generating a real one. The model learned the shape of Australian medium neutral citations from millions of examples, so it can produce a well formed citation for a case that does not exist without any internal signal that something has gone wrong. Better prompting does not fix it, since no instruction can conjure a source that was never fetched.

Is it safe for Australian solicitors to use AI for legal research?

It depends entirely on whether the tool retrieves law or recalls it, and on whether you verify the sources before anything is filed or sent. Several Australian courts have issued practice notes and guidance on generative AI in court documents, and the consistent position is that the practitioner remains accountable for what they file. Use tools that link to the primary source, and check that source before you rely on it.

How can I tell if a legal AI tool is using retrieval?

Ask it something genuinely obscure and see whether it declines or invents an answer, then try to open the underlying judgment or section directly from the response. A retrieval system links to the specific paragraph it used and will say when it found nothing on point. If you have to go and search for the cited source yourself, you are being asked to trust the output rather than check it.