The ABA's own technology committee called it out: too many legal tech vendors are putting 'marketing dressing on a pumped-up chatbot' and calling it AI. At least 40% of legal tech products marketed as 'AI-powered' in 2025 are using the term to describe basic automation, keyword search, or simple rule-based logic that predates large language models by a decade.

AI washing isn't just annoying -- it's expensive. Firms that buy 'AI' products that don't actually use AI waste $20,000-100,000 on tools that deliver marginal improvements while missing the genuine efficiency gains of real AI. Here's how to spot the difference.

AI washing is the practice of marketing products as 'AI-powered' when they rely on technology that isn't meaningfully intelligent. Common patterns in legal tech:

Pattern 1: Rebrandled keyword search. A legal research tool that matches keywords and Boolean operators gets rebranded as 'AI-powered legal research.' The technology is the same search algorithm that existed in 2010 -- it just has a chatbot interface pasted on top.

Pattern 2: Rule-based automation called 'AI.' Contract review tools that flag clauses based on if/then rules (if the indemnification cap is below $X, flag it) are marketed as 'AI contract analysis.' This is automation, not intelligence. It works for standardized documents but can't handle anything the rules don't anticipate.

Pattern 3: Template generation marketed as 'AI drafting.' Tools that fill in template fields based on user inputs and call it 'AI-generated' documents. This is mail merge with better UX.

Pattern 4: Thin API wrappers. Companies that pipe your query to ChatGPT or Claude's API, maybe add a system prompt with legal context, and charge $300/month for access you could get for $20/month directly. The 'proprietary AI' is a prompt template.

Pattern 5: 'AI-enhanced' analytics. Practice management tools that add basic charts and call it 'AI-powered analytics.' Counting your billable hours by practice area isn't AI. It's a spreadsheet pivot table.

Use these five tests during any vendor evaluation:

Test 1: The Novel Input Test. Give the tool a query or document it hasn't seen before -- something unusual, complex, or outside standard patterns. Real AI handles novel inputs reasonably. Rule-based systems fail on anything outside their rules. If the 'AI' contract review tool can't handle a non-standard indemnification clause, it's just pattern matching.

Test 2: The Explanation Test. Ask the vendor to explain their AI architecture. What model(s) do they use? Is it proprietary or built on foundation models? What training data was used? If they can't or won't answer specifically, treat their AI claims skeptically. 'Proprietary AI technology' with no specifics is a red flag.

Test 3: The 'Turn Off AI' Test. Ask the vendor what their product does without the AI component. If the answer is 'everything it does now, just slightly slower,' the AI is cosmetic. Real AI tools have capabilities that don't exist without the AI layer.

Test 4: The Benchmark Test. Ask for accuracy data from independent testing (not vendor-conducted). Real AI tools have benchmark results they can share. Tools that rely on 'customer testimonials' instead of accuracy data are hiding something.

Test 5: The Pricing Sanity Test. If the tool costs $300/month and its core function is available through ChatGPT Enterprise ($20/user/month) or Claude Pro ($20/month), ask what the extra $280 buys. If the answer is 'our proprietary prompt library,' you're paying $3,360/year for prompts you could write yourself.

Real AI vs. Automation: The Technical Distinction

Not everything called AI is bad -- some of it is genuinely useful automation that just got mislabeled. Here's the hierarchy:

Level 1: Rules-Based Automation (not AI) If/then logic. Predefined rules. Template filling. Works well for standardized processes but can't handle exceptions or novel situations. Examples: auto-populating contract templates, flagging documents containing specific keywords, routing emails based on subject line.

Level 2: Machine Learning (basic AI) Statistical models trained on data to recognize patterns. Better than rules at handling variation but limited to patterns in the training data. Examples: TAR 1.0 (predictive coding in e-discovery), sentiment analysis in client reviews, document classification.

Level 3: Large Language Models (current AI) Transformer-based models (GPT-4, Claude, Gemini) that understand language semantically and generate coherent output. Can handle novel inputs, reason across contexts, and produce original analysis. Examples: Lexis+ AI, Harvey, CoCounsel, AI-powered brief drafting.

Level 4: AI Agents (emerging) AI systems that can plan multi-step tasks, use tools, and execute complex workflows with minimal human direction. Still early in legal tech. Examples: Harvey's workflow automation, emerging doc review agents.

What to demand from vendors: Know which level their product operates at. Level 1 tools labeled as 'AI' are AI washing. Level 2 tools labeled as 'AI' is a stretch but not fraud. Levels 3 and 4 are genuine AI. Price and expectations should match the level.

The Cost of Getting Fooled

AI washing costs law firms real money:

Direct costs: A firm that buys a $50,000/year 'AI contract review' tool that's actually rule-based automation could get 80% of the same functionality from a $5,000/year automation tool. The AI premium was $45,000 spent on marketing, not technology.

Opportunity costs: Worse than overpaying is buying the wrong tool and concluding 'AI doesn't work for our practice.' A firm that tries a rebranded keyword search tool, sees no improvement, and decides AI research isn't ready -- that firm just delayed real AI adoption by 12-18 months while competitors pull ahead.

Vendor lock-in costs: If you build workflows around a tool that's fundamentally automation, switching to real AI later means rebuilding workflows, retraining staff, and potentially migrating data. The longer you use the wrong tool, the more expensive the switch.

Reputation costs: Telling clients you use 'AI-powered' tools when you're using glorified templates erodes trust when clients eventually discover the reality. Better to honestly say 'we use automation for X and AI for Y' than to overclaim.

Industry estimate: law firms collectively waste $200-400 million annually on legal tech products marketed as AI that don't meaningfully use AI. That's money that could fund genuine AI adoption.

The Vendor Evaluation Checklist

Before buying any 'AI-powered' legal tech product, require answers to these questions:

Architecture: - [ ] What AI model(s) does the product use? (Foundation models, fine-tuned models, proprietary models?) - [ ] Is the AI a core capability or a feature added to an existing product? - [ ] What training data was used, and how is the model updated?

Performance: - [ ] What independent accuracy benchmarks exist? - [ ] What is the hallucination/error rate on legal tasks? - [ ] Can you provide a side-by-side comparison with a non-AI approach to the same task?

Differentiation: - [ ] What can this product do that a direct ChatGPT/Claude API connection cannot? - [ ] What happens if I 'turn off' the AI component -- what functionality remains? - [ ] How does this differ from your product 2 years ago before the AI features?

Security: - [ ] SOC 2 Type II certification? - [ ] DPA with no-training, no-retention commitments? - [ ] Where is data processed and stored?

Economics: - [ ] What is the price premium for AI features vs. the non-AI version? - [ ] What measurable ROI have existing customers achieved? - [ ] Can we run a 30-day pilot before committing?

If a vendor can't or won't answer these questions specifically, their product is likely AI-washed. Real AI vendors welcome scrutiny because their technology stands up to it.

The Bottom Line: At least 40% of legal tech products marketed as 'AI-powered' are using the term to describe basic automation or keyword search. The ABA called it 'marketing dressing on a pumped-up chatbot.' Test every vendor with novel inputs, ask for independent accuracy data, and demand to know what their product does without the AI layer. Real AI handles novel situations, provides measurable accuracy improvements, and can explain its architecture. Everything else is AI washing.

AI-Assisted Research. This piece was researched and written with AI assistance, reviewed and edited by Manu Ayala. For deeper takes and the perspective behind the research, follow me on LinkedIn or email me directly.