Choosing a model

Choose a model by matching evidence to a defined workload. Use this framework when several candidates appear capable but differ in quality signals, cost, latency, context, provider status, or operational risk. Ethen supplies catalog and research fields for comparison; it does not provide one universally best model or a verified automatic router for every product.

Start with workload

Write a workload brief before opening a leaderboard or sorting the catalog. Include:

• the input and expected output;
• required capability or modality;
• typical and maximum context needs;
• acceptable response time;
• expected request frequency or throughput needs;
• quality criteria;
• budget sensitivity;
• provider or local constraints;
• privacy, approval, and review requirements;
• consequences of a poor answer or failed run.

This brief prevents a high score from replacing the actual task. A coding model selected for an image workflow is still the wrong choice, and a large-context model may add no value to a short classification task.

Build a shortlist from Model Library capability and status, then use Model Intelligence to investigate research evidence where profiles exist.

Begin with the work the model must perform: input type, output type, reasoning depth, response speed, context size, and whether the path needs to be runnable through the current environment.

Use capability family as the first filter, then compare evidence. A strong benchmark on an unrelated task is not a reason to select a model.

Define unacceptable outcomes

A workload brief should identify failure conditions as well as success criteria. Examples include unsupported modality, omission of required source material, output that cannot be reviewed, latency that disrupts an interactive task, or provider status that prevents execution. These are workload-specific examples, not platform guarantees.

Unacceptable outcomes help eliminate candidates early and define the representative test. They also clarify when a cheaper or higher-scoring model is still unsuitable because the cost of failure is too high.

Quality

Quality is workload-specific. Model Intelligence profiles can include benchmarks, composite metrics, comparisons, charts, summary cards, FAQs, and derived verdicts. These are useful signals, but each has limits.

Use at least three evidence types:

1. Task-relevant benchmark or profile data. Prefer evidence that resembles the intended work.
2. Data quality indicators. Check source coverage, missing fields, extraction confidence, notes, and quality flags.
3. Representative testing. Run a small set of real tasks through a supported route and review the outputs.

A composite score can help rank candidates but can hide tradeoffs among included metrics. A derived verdict is presentation logic. Neither should be treated as guaranteed truth.

Document the quality criteria before testing. Examples include factual accuracy against supplied material, code correctness under review, adherence to format, completeness, or consistency across repeated prompts.

Model Intelligence can present benchmark scores, summary cards, comparisons, chart sections, FAQs, and quality flags.

Use several signals rather than one composite score. Check whether data is present, how the chart was adapted, and whether the recommendation is derived presentation logic.

Cost

Catalog and profile data can represent input price, output price, cache read and write pricing, and web-search pricing. Values may be missing or change over time.

Cost comparison should account for more than one field:

Do not treat a missing price as free. Do not freeze a current value in a long-lived recommendation. Record the provider and date when cost affects a decision.

A more expensive model can be economical if it reduces retries or review, while a lower-priced model can be better for high-volume, low-risk tasks. The workload brief should decide which tradeoff matters.

The catalog can expose input, output, cache read, cache write, and web-search price fields.

Missing price data is not free usage. Current prices can change, and a listed unit may need provider-specific verification. Estimate the workload’s input and output shape before comparing cost.

Latency

Latency is the time associated with receiving a response, while throughput describes the rate of processing. The catalog can represent both, but fields may be absent and measurement conditions are not fully defined by this batch.

Consider the user experience:

• interactive work may prioritize response time;
• batch or background work may prioritize throughput;
• tool-assisted tasks may be dominated by integration time rather than model time;
• long outputs may behave differently from short prompts;
• provider and route conditions can change observed performance.

Use profile charts or catalog fields as directional evidence, then test with representative prompts through the intended provider path. Avoid claiming a permanent fastest model.

When latency is missing, keep it unknown. A model should not move up or down the shortlist because an absent field was converted to zero.

Latency and throughput answer different questions. Latency reflects waiting time before or during a response, while throughput reflects generation speed once output is moving.

The best choice depends on interaction style. An interactive product may value predictable latency; a batch task may value total throughput or cost. Avoid a universal fastest-model claim.

Context

Context has two relevant meanings. Workspace context is the material associated with the task. The context window is a model field representing request capacity when available.

Estimate actual need rather than maximizing the field. Ask:

• How much source material must be included at once?
• Can the work be split into smaller, reviewable stages?
• Will earlier outputs be reused?
• Is the model classified as long-context, and is the exact context field present?
• What maximum output is required?
• Does the task remain reliable when the input is large?

A larger context window does not guarantee better retrieval, reasoning, or adherence. It can also increase cost. Test the real document shape and preserve evidence about what was supplied.

Maximum output is separate from context capacity. A model can accept a large input while still having a smaller represented output limit.

Context window and maximum output tokens are separate fields. The first concerns how much input/context the model can accept; the second constrains generated output where supplied.

Larger is not automatically better. Long inputs can affect cost and latency, and the current catalog may omit one or both fields.

Risk and availability

Operational status can eliminate otherwise attractive candidates. Use the catalog’s status categories as action signals:

Risk includes more than failure to run. Consider provider dependence, missing evidence, dynamic pricing, incomplete benchmark coverage, context sensitivity, output-review burden, and the effect of fallback.

A local-oriented lane may change operational control, but hardware and runtime specifics are not covered here. An open-weight label may change portability or licensing considerations, but individual license terms must be verified separately.

A model must fit the environment as well as the workload. Check status, provider configuration, supported modality, source evidence, and the consequence of failure.

Catalog-only and missing-key models are not ready for the same use path as runnable records. Do not rely on automatic fallback unless it is explicitly configured and documented.

Decision checklist

Use a scored matrix only after defining non-negotiable requirements.

Record why a candidate was rejected as well as why the winner was selected. Then test the chosen model with representative work and preserve the result. A selection is provisional when fields are missing, product maturity is under review, or the provider state can change.

A defensible selection records the task, required capability, evidence reviewed, expected context and output, cost sensitivity, latency need, provider status, and fallback or recovery plan.

Run a small representative test, review the result, and preserve the exact model ID and current status. Revisit the decision when pricing, availability, or model evidence changes.

Weighting the decision

Weights should follow the workload rather than a permanent platform formula. An interactive assistant may give latency and answer quality more weight. A document-analysis task may prioritize context, evidence, and cost. A state-changing agent may prioritize reliability, provider status, and verification over small benchmark differences.

Document the weights before scoring candidates. Otherwise, a reviewer can unintentionally change the importance of a criterion to favor a preferred model. If evidence is missing, mark the criterion unknown rather than assigning a neutral score.

Elimination before ranking

Remove candidates that fail non-negotiable requirements before calculating a total. Unsupported modality, inadequate context, absent provider configuration, or an unacceptable policy constraint should not be offset by a high quality score.

After elimination, rank only the viable candidates. This keeps the matrix from recommending a model that cannot perform the task in the current environment.

Representative test set

Use several examples that reflect normal, difficult, and failure-prone cases. Keep instructions and review criteria consistent across candidates. Record model ID, provider, date, visible status, and any route or quality flags.

Review the outputs without relying only on style. Check correctness against supplied sources, completeness, adherence to constraints, format, consistency, and how the model handles uncertainty. For code, add the appropriate review and tests; model confidence is not execution evidence.

Revisit conditions

A model decision should be revisited when provider status changes, prices or limits are updated, a new version appears, normalized evidence improves, or the workload changes. Preserve the earlier record rather than rewriting history.

A current selection can remain valid even when another model ranks higher on a new leaderboard, provided the chosen model still meets the actual task requirements. The purpose of review is to detect material change, not to chase every dynamic ranking.