I Fed the Same 40-Page Contract to ChatGPT, Claude, and Gemini

Only one of the three caught the liability clause that could've cost my client six figures. I ran the exact same 40-page vendor agreement through ChatGPT-4, Claude 3.5 Sonnet, and Gemini 1.5 Pro using identical prompts, and the results weren't even close. If you're using AI to review contracts, leases, or any legal document, you need to know which model actually reads the fine print — because two of them are giving you false confidence.

The Clause All Three Models Almost Missed

Page 27, section 14.3. Buried in a paragraph about "service continuity," there was a clause that shifted unlimited liability onto my client for third-party data breaches — even ones the vendor caused.

I asked all three the same question: "Summarize the liability and indemnification clauses in this contract, and flag anything unusual or high-risk for the party signing as the client."

ChatGPT-4 gave me a clean, well-organized summary. It listed the standard indemnification language and called it "fairly standard for vendor agreements." It missed the shift in liability entirely.

Gemini 1.5 Pro did something similar — it summarized the section accurately in terms of what it said, but didn't flag why it mattered. It treated the clause as informational, not a risk.

Claude 3.5 Sonnet was the only one that stopped and said: "This clause is unusual — it extends the client's liability to breaches caused by the vendor's own systems, which contradicts the risk allocation described in section 3.1." That's not summarizing. That's actually reasoning about the document.

Why This Happens (And It's Not About "Smarter" AI)

Here's the mental model that changed how I use these tools: summarization and risk analysis are two completely different tasks, and most people ask for the first when they need the second.

When you say "summarize this contract," you're asking the model to compress information. When you say "flag risks," you're asking it to compare clauses against each other and hold the entire document in tension at once. That second task requires much stronger reasoning over long context — and that's exactly where the models split.

Claude's advantage here isn't magic. It's built with a stronger emphasis on long-context reasoning — meaning it doesn't just retrieve chunks of the document, it tracks relationships between clauses that are pages apart. Section 3.1 and section 14.3 were 24 pages apart. Claude connected them. The other two treated each section as its own island.

This matters beyond legal documents. Any time you're asking AI to review something long — a financial report, a research paper, a technical spec — you have to know whether you're asking it to describe or to compare. Most people never make that distinction, and it's why they walk away with a false sense of security.

How to Actually Contract-Review With AI Starting Today

Don't just ask for a summary. Ask for contradictions and inconsistencies — that's the prompt that forces real reasoning instead of surface-level compression.

Here's the exact workflow I now use for any contract over 10 pages:

Step 1: Upload the full document to Claude (not ChatGPT — save that for later). Use this prompt: "Read this entire contract. Identify any clauses that contradict each other, shift risk unexpectedly, or create obligations not mentioned elsewhere in the document. Quote the exact section numbers."

Step 2: Take whatever Claude flags and cross-check it in ChatGPT-4 with: "Explain in plain English what this clause means for [your role — buyer, vendor, tenant, etc.] and what could go wrong if I sign it as-is." ChatGPT is genuinely better at plain-English explanations once you already know where to look.

Step 3: Run the same flagged sections through Gemini with: "Compare this clause to standard industry language for [type of contract]. Is this typical or aggressive?" Gemini's tied to Google's data and is often stronger at benchmarking against "what's normal."

Ten minutes. Three tools. Each one doing what it's actually good at, instead of asking one model to do everything.

The Part Most People Get Wrong

Most people pick one AI tool, get comfortable with it, and assume it's equally good at everything. That's wrong, and it's the single biggest mistake I see in how people use AI for serious work.

ChatGPT being excellent at writing emails doesn't mean it's excellent at legal reasoning. Gemini being deeply integrated with your Google Docs doesn't mean it catches contradictions across 40 pages. Loyalty to one tool is costing you accuracy — and in something like contract review, accuracy is the entire point.

The fix isn't finding "the best AI." It's matching the task to the tool's actual strength, every single time. That takes ten extra minutes. Missing a liability clause takes six figures.

Key Takeaways

• Summarizing ≠ risk analysis: Asking AI to "summarize" a contract will not catch hidden risks — you have to explicitly ask for contradictions and unusual clauses.
• Claude wins on long-context reasoning: It's the strongest at connecting clauses that are pages apart in long documents.
• ChatGPT wins on plain-English clarity: Use it after risks are flagged to understand what a clause actually means for you.
• Gemini wins on benchmarking: It's best at telling you whether language is standard or unusually aggressive for your industry.
• Never rely on one AI for high-stakes documents: Run important contracts through at least two models with different prompts, not the same prompt twice.

What to Do Right Now

Open Claude right now and upload any contract, lease, or agreement you've been putting off reading closely. Paste this exact prompt: "Read this entire contract. Identify any clauses that contradict each other, shift risk unexpectedly, or create obligations not mentioned elsewhere. Quote the exact section numbers." You'll have your first real risk flag in under two minutes.