Choosing and Running Models

The families, and their personalities

Vendors ship models in roughly three tiers: a top "frontier" tier, a mid "workhorse," and a cheap tier. The personalities stay surprisingly stable across versions even as benchmark scores churn.

Family	Known for	Typical use
Anthropic (Claude)	Reliable agents & coding, disciplined instruction-following, long-run coherence	Coding agents, enterprise automation, regulated industries
OpenAI (GPT)	Biggest ecosystem, broad multimodal (voice, images)	Consumer chat products, Microsoft-stack shops
Google (Gemini)	Price-performance at the cheap tier, huge context, video/audio input	High-volume cheap inference, massive-document work
Open-weight(Llama, Qwen, Mistral, DeepSeek)	Full data control, no per-token cost at scale, ~6–12 months behind frontier	On-prem, air-gapped, data-residency, custom tuning

Current flagships, as of writing (check before you trust this)›

As of mid-2026: Anthropic's Claude Opus 4.8 (plus Sonnet 4.6 and Haiku 4.5 for cheaper tiers), OpenAI's GPT-5.5 family, and Google's Gemini 3.1 Pro (plus 3.5 Flash and 3.1 Flash-Lite) are the three frontier lines, separated by single-digit percentage points on most benchmarks. On the harder SWE-bench Pro coding benchmark, Opus 4.8 leads at roughly 69% against GPT-5.5's ~59% and Gemini 3.1 Pro's ~54%, illustrative of the pattern (Claude ahead on hard coding, Gemini cheapest at volume, GPT-5.5 pushing hardest on agentic tool use), not a number worth memorizing. This entire paragraph will be stale within a couple of quarters, that's the point made two sections up. Check a live leaderboard before you plan around any of it.

Try it yourself: Feel the tier difference directly›

Send the exact same non-trivial question: 'explain the CAP theorem and why it matters for distributed databases' works well: to a frontier model and to that same vendor's cheapest/smallest model. Compare depth, nuance, and whether either one hedges or gets something subtly wrong. The gap is the 'personality' difference the article means.

Why the leaderboards lie

Public benchmark scores are mostly marketing. Test questions leak into training data (so scores are inflated), and "which answer do people prefer" rewards confident, verbose, nicely-formatted answers over correct ones.The only number that predicts your result is your own eval: 20–50 real examples from your actual task, run against a few candidate models, graded with a rubric. Half a day of work, and it beats every public chart. Re-run it on every upgrade, models sometimes getworseat your specific task even as the public scores rise.

Try it yourself: Build a five-example eval by hand›

Pick 5 real examples from something you actually do (grading an email's urgency, summarizing a paragraph, whatever's at hand). Run all 5 through two different models. Grade each answer yourself with a simple rubric (correct / partially correct / wrong). That's a miniature version of the eval process described above, and it'll probably disagree with at least one public leaderboard ranking.

Spending less (in order of impact)

Route by difficulty. Send easy requests to the cheap model, hard ones to the expensive one. A typical mix is ~70% cheap / 25% mid / 5% frontier, cutting cost several-fold.
Cache stable prompt prefixes(put unchanging content first). For agents this is the biggest lever, because an agent re-sends its whole transcript every step.
Batch the non-urgent work: about half price if you can wait. Nightly enrichment, bulk analysis.
Keep output tight. Output is ~5× the cost of input, so ask for concise, schema-shaped answers.

Try it yourself: Estimate the routing savings›

Take a hypothetical workload of 1,000 requests/day. Estimate the cost if 100% go to a frontier model, versus the ~70% cheap / 25% mid / 5% frontier split described above (use any provider's real per-token pricing). The multiplier is usually larger than people expect before they do the arithmetic.

Where the model runs (decided before quality)

For sensitive data (and identity data absolutely counts) the first question isn't "which model is best," it's "where does my data go?"

Option	Where your data goes	Trade-off
Vendor API directly	To the vendor (under contract, no-training options)	Newest features first; least control
Your cloud tenancy(Bedrock, Vertex, Azure)	Stays inside your cloud's security boundary	The enterprise default; features lag by months
Self-host open-weight	Never leaves your network	Total control; you own the serving & ops headache

Try it yourself: Find the actual data policy›

Pick a model provider you or your company uses and find their data-processing terms: specifically, whether prompts sent through the API are used for training by default (usually not, for paid API tiers: often yes, for free consumer chat apps unless you opt out). This single fact is the real gate for anything touching sensitive data, more than any benchmark score.