AI vs. Junior Associates: Compare the Task and the Review

A fast answer and a completed assignment are different endpoints

An AI system produces a polished research note. A junior associate takes longer and returns a rougher document with an unresolved jurisdiction question. Which performed better? Without the assignment, sources and review standard, the comparison tells us very little. The polished note may have answered the wrong question. The rough note may have identified the fact that prevents a reliable answer. Either may also simply be poor work.

The useful comparison starts with a defined task and ends with an accepted handoff. That includes finding omissions, checking support and deciding what remains uncertain. A model demonstration that ends at generation and a human assignment that ends after verification do not measure the same work.

This is a method for examining a workflow, not a benchmark showing that either participant wins. I would use it before making a staffing argument: first establish what changed in the work, then ask what that change could mean for capacity and learning.

Fix the question before measuring the answer

Write down the recipient, materials, jurisdiction, date boundary and deliverable. Specify whether the assignment is a summary of supplied documents, research beyond them or a recommendation requiring professional judgment. These are different tests. A system confined to a supplied file should not be penalized for failing to find an outside authority it was never asked or enabled to research. Nor should a fluent summary count as a complete research assignment.

Give the comparison an explicit acceptance standard. For a source-based note, that might mean every material proposition is traceable to the supplied source, conflicting evidence is visible and missing inputs are identified. These are proposed criteria; the supervising lawyer must adapt them to the assignment.

Record differences in access and assistance. If the associate has a research database and the model has only excerpts, the result describes those configurations. If a lawyer repeatedly repairs the prompt or output, that work belongs in the record. Hiding it makes the apparent comparison cleaner and the operational conclusion weaker.

Inspect the failures that change the next decision

A synthetic comparison makes the endpoint visible

Consider an invented document-summary exercise. The file contains a draft agreement referring to Schedule B, but Schedule B is absent. The assignment is to summarize obligations supported by the file and identify what cannot yet be determined. This is a teaching example, not a client matter or an observed test result.

Output A confidently describes the missing schedule. Output B summarizes the supplied clauses and flags the missing attachment. B has handled this particular boundary better, whether a person or a tool produced it. But B is not therefore a complete legal analysis: the reviewer still needs to check the supplied clauses and the significance of the missing document.

Now imagine A arrives first and B arrives later. Generation time favors A; correction and follow-up may favor B. We cannot supply a numerical tradeoff without measuring the actual work. The example shows why the endpoint matters, not how much time a firm will save.

The lawyer retains the professional decision

ABA Formal Opinion 512, issued July 29, 2024, addresses generative AI through duties including competence, confidentiality and supervision. It explains that appropriate verification depends on the task and tool and that AI output does not replace professional judgment. It interprets the ABA Model Rules; applicable jurisdictional rules still need attention.

That matters to the comparison because producing text does not transfer responsibility for using it. A workflow can make drafting easier while leaving a difficult acceptance decision with the lawyer. The reviewer needs enough understanding and access to check the material proposition, not just an assurance that another stage was automated.

Count reviewer interventions by type: correcting a factual error, locating support, resolving a scope question or rewriting for the recipient. Separate necessary substantive review from preventable repair. Those categories reveal whether the proposed workflow helps the team or merely moves effort to a scarcer reviewer.

A task result does not settle the associate question

Even a well-run comparison establishes something narrower than a staffing decision. It concerns an assignment, tool configuration, participant experience and review process. Repeated drafting performance does not alone tell a firm who will develop client knowledge, learn to recognize unusual facts or become able to supervise the next generation of work.

There is a real tradeoff: preserving every old exercise can waste time, while removing every first attempt can remove occasions to learn. The response should be deliberate assignment design. Protect the reasoning step a junior needs to practice, then use assistance where the learning objective permits it. The junior training guide develops that supervisor decision.

For a practical next step, take one synthetic exercise from the review lab and write an acceptance standard before inspecting an answer. Record what required correction and what remained unobserved. If your purpose is candidate assessment rather than workflow comparison, use the separate work-sample scorecard, with its limits on interpretation. Keep the conclusion as narrow as the evidence you actually collected.