Practice

Discovery with AI: 40,000 documents and the duty you still owe

By Michael Nadalin, Founder, Market Lead · 28 August 2026 · 8 min read

Forty thousand documents arrive from four custodians. Discovery orders are made, the timetable assumes you will manage, and the client has already asked what the review will cost before you have opened a single file.

Run the arithmetic. If a junior solicitor sustains fifty documents an hour, forty thousand documents is eight hundred hours of reading. That is five months of one person doing nothing else, or a team of four for most of a quarter, before anyone drafts a chronology. On most commercial matters that number does not fit the budget, the timetable or the amount in dispute.

So the question is not whether to put technology across the set. Firms have been doing that for years. The question is narrower and harder: which decisions do you hand to a machine, which ones stay with a solicitor, and what record do you keep so that you can stand up and describe the process if the other side or the court asks.

Run the arithmetic before you run the review

Most of the value in a large discovery exercise comes before anything clever happens. Exact duplicates, near duplicates, email threading down to inclusive messages, date range filters, custodian filters and system file exclusions will strip out a large share of a raw collection without any judgment call being made at all. Those steps are deterministic and reproducible, which is exactly why they are easy to defend.

The mistake is to skip that work and point a model at the raw set. You then pay for the model to read the same attachment forty times, and every validation number you produce is polluted by duplication. Do the mechanical reduction first, record what each step removed, and only then decide what needs a review decision.

The figures below are an illustration of the shape of a cull, not a benchmark. What matters is that you can produce the equivalent numbers for your own matter and explain each drop.

Where a large collection actually goes
Collected from custodians
40,000
After exact and near duplicate removal
26,000
After email threading to inclusive messages
18,000
After date, custodian and domain filters
11,000
Ranked as likely relevant, sent to human review
4,000
An illustrative cull on a hypothetical 40,000 document set. Your ratios will differ by matter and data source. Record what each step removed and why.

What the model decides and what you decide

The useful distinction is not between tasks a model can perform and tasks it cannot. It performs all of them, with varying reliability. The distinction is between decisions that are reversible and testable, and decisions that produce an irreversible consequence such as a waiver of privilege or a certificate signed by a solicitor.

Rank the work by how much of it you can genuinely hand over, and be honest about the bottom of the list. A model that produces a fluent summary of an email chain is not the same thing as a model that has understood the commercial significance of the chain, and it will not tell you the difference.

How much you can hand over, task by task
Deduplication, threading and metadata filtering
Deterministic and reproducible. You can test the rules on a sample and get the same answer twice. Nothing here turns on judgment.
Hand it over
Ranking documents by likely relevance
The model orders the queue. A solicitor still reads what it puts at the top and samples what it buries at the bottom.
Hand over, then test
First pass issue coding and summarising
Good for building a working chronology quickly. Unreliable as a final answer. Every summary you actually rely on gets checked against the source document.
Draft only
Flagging candidate privileged material
Spotting the firm domain, counsel names and the usual phrases is a net, not a decision. It will miss legal advice repeated inside a spreadsheet or an attachment.
Net, not decision
The final privilege call and the discovery certificate
You sign it. No vendor, model or protocol carries that for you, and no court will accept that it did.
Never
Read this from the top down. The further you go, the more supervision the task needs, and the last row never moves.

The obligations that actually bite

Nothing about technology assisted review changes the duties. It changes how easy they are to breach quietly. The failures that cause trouble are rarely dramatic: a criteria change nobody wrote down, a cloud service nobody read the terms for, a methodology disclosed for the first time in reply submissions.

The Australian courts are not hostile to this. Federal Court practice material and the Supreme Court of Victoria's guidance on technology in civil litigation both contemplate technology assisted review, and judges are generally more comfortable with a method that was put on the record early than one revealed under pressure. The gap is almost always in the record keeping, not in the permission.

Comfortable assumption, and what the duty actually requires
The vendor ran the review, so the methodology is theirs to explain.
You certify discovery. You have to be able to describe the collection, the cull and the validation in your own words, without the vendor in the room.
Proportionate means we did the best we could inside the budget.
Proportionate means a reasonable search you can justify against the issues actually in dispute, decided and recorded at the time, not reconstructed afterwards.
Uploading the set to a cloud tool is an IT question.
It is a confidentiality question and a Privacy Act question. Where does the data sit, who at the vendor can read it, do inputs train the model, and what does APP 8 require of you for any offshore disclosure.
We will explain the method if the other side asks.
Raise it early, ideally at the first case management hearing. A protocol agreed in advance costs a fraction of one argued about after production.
Privilege can be sampled like any other category.
An inadvertent production is not a statistical problem, it is a waiver problem. Privilege gets a wider net, a lower threshold and a second set of eyes.
The juniors are running the platform, so supervision is handled.
Supervision under the conduct rules means the responsible solicitor understands what the tool is doing well enough to sign off on it. Delegating the tool does not delegate the duty.
Each row is a real failure mode. The left column is what practitioners tend to assume until an interlocutory application forces the point.

Validating the cull is the whole job

Here is the part firms skip. Any ranking system can put plausible documents at the top of a queue. What you have to be able to say is something about the documents it did not put there, because those are the ones you are about to not produce.

That means sampling the discard pile, not just the review pile. Draw a random sample from everything the process is about to exclude, have a solicitor code it against the same criteria, and count how much relevant material was sitting in there. If the answer is uncomfortable, you retrain or you widen the net, and you do it before production rather than after a complaint.

Stop on a measured basis. Stopping because the deadline arrived is a defensible commercial decision and an indefensible discovery one, and the difference shows up in cross examination on the affidavit.

A validation sequence you can describe under questioning
1
Fix the relevance criteria in writing first
One document, agreed with the partner and ideally consistent with the pleadings and any discovery order. Date it. This is the yardstick every later number is measured against.
2
Draw a control sample before training
A random sample coded by a solicitor against the fixed criteria, held back from training. This gives you a stable measure to compare results to, rather than measuring the model against itself.
3
Train, rank, and review from the top
Senior review of the highest ranked material first, feeding decisions back. Track disagreement between reviewers early, because inconsistent coders produce an inconsistent model.
4
Sample what you are about to discard
Random sample of the excluded set, coded blind. This is the number that answers the question the other side will ask, which is what did you miss.
5
Stop on a measured basis
Set the stopping point by reference to what the discard sample shows, and record the decision and the reasoning. Not because the calendar ran out.
6
Keep the defensibility file
Criteria document, every version of it with dates, seed decisions, sample results, platform audit logs, and a short plain English narrative of the whole process written while it is fresh.
Every step produces an artefact. If a step leaves no artefact, it did not happen as far as the court is concerned.

Agree the protocol before you press go

Most disputes about technology assisted review are not really about the technology. They are about a party being told after the fact that thirty thousand documents were excluded by a method they had no chance to comment on. That is an easy application for an opponent to run and an expensive one to defend.

Sequence the conversation properly and it largely disappears. Scope first, protocol second, method third, and all of it before processing costs are sunk.

When to raise what
Before collection
Agree custodians, date range and data sources. Include the awkward sources: messaging apps, shared drives, personal devices used for work, and anything with no text layer.
First case management hearing
Put the proposed use of technology assisted review on the record. Disclosed early it is a case management efficiency. Disclosed late it looks like something you were hoping not to mention.
Before the model touches anything
Settle the document management protocol: fields, formats, dedup rules, family handling, near duplicate treatment and how redactions will be applied and exchanged.
During review
Log every change to the relevance criteria and the date. A definition that shifts mid review without a record is the most common reason a validation result cannot be relied on later.
At production
Serve the list with a short, accurate description of the search actually performed. Vague wording is what invites the interlocutory application you were trying to avoid.
After production
Retain sample results, seed decisions and audit logs for the life of the matter and any appeal. Reconstructing them eighteen months later is not possible.
The cost of each of these conversations rises sharply the later it happens. Scope disagreements after processing mean paying to process twice.

What to do on the next matter that lands

Start with the mechanical reduction and record it. Fix the relevance criteria in writing before anyone codes a document. Decide, explicitly, which of the tasks on that ladder you are handing over and which you are keeping, and put the privilege review in the keeping column. Sample the discard pile. Write the narrative of the process while you can still remember it.

Do the vendor diligence once, properly, and reuse it. Where the documents sit, who can access them, whether your inputs are used to improve anyone's model, what happens on termination, and what your cross border position is. That review takes an afternoon and it covers every matter you run on that platform for the next two years.

Separate the two kinds of AI work in your head, because they fail differently. A review platform ranks documents you already hold, and its failure mode is a document you never see. A research tool like Legal Brain answers questions about Australian law and rules, and its failure mode is a confident answer that is wrong, which is why the source is always the point rather than the summary. Neither one signs the certificate. You do, and everything above exists so that when you sign it you can say precisely what you did and why it was reasonable.

Frequently asked questions

Can you use AI to review documents for discovery in Australian courts?

Yes. Australian courts have practice material that contemplates technology assisted review, including in the Federal Court and the Supreme Court of Victoria, and it is well established on large commercial matters. The expectation is that you can describe and justify the method, not that you avoid it. Raise it early rather than disclosing it after production.

Do I have to tell the other side that AI was used in the document review?

The safer position is yes, and early. Parties are generally expected to agree a document management protocol and to be transparent about the search performed, and a methodology raised at the first case management hearing rarely becomes a fight. A methodology revealed only after production is where interlocutory applications come from.

Is it a breach of confidentiality to upload client documents to an AI tool?

It depends entirely on the arrangement. Check whether inputs are used for training, where the data is hosted, who at the provider can access it, and what your obligations are under APP 8 if anything is disclosed offshore. Where the material is sensitive or the risk is material, get the client's informed instructions before uploading rather than after.

Can AI handle the privilege review on a large discovery set?

It can build the net. Flagging documents involving legal advisers, counsel and the usual privilege language will surface most candidates quickly. It cannot make the call, because an inadvertent production is a waiver problem rather than a statistical one, and privileged advice repeated inside an attachment or a spreadsheet is exactly what these systems miss.

Check your understanding

Two quick questions

No score is stored. Pick an answer to see why it is right.

  1. 1You have run technology assisted review across 40,000 documents. Which step is most likely to be the one you cannot defend if it is challenged?

  2. 2Which of these can a language model reasonably be trusted to finalise without a solicitor reading the underlying documents?

Research Australian law without handing over client data

Legal Brain searches Australian legislation and case law, shows you the source behind every answer, and anonymises client-identifying detail before anything reaches a model.

Request early access