Do we need a custom model or an API?

Start with the task boundary

The choice is not a contest between generic and bespoke technology. It follows from a defined task, acceptable failure modes and operating constraints. Classification, extraction, semantic search and open-ended generation have different evidence needs. Establish representative inputs, expected outputs and review criteria before selecting a model. Include ambiguous, malformed and adversarial cases rather than a polished demonstration set. If people cannot agree what a correct result contains, custom training will encode that disagreement at greater expense rather than resolve it.

A hosted API is usually the fastest baseline because infrastructure, serving and model updates are provided. It also introduces provider terms, regional availability, rate limits, model-version changes and variable consumption costs. An open-weight model offers more control over deployment and versioning but transfers responsibility for serving, patching, scaling and safety. Training a model adds data engineering, experimentation and continuous evaluation. These are operating models, not one-off procurement choices, and should be compared over the expected workload and governance requirements.

Use the least invasive adaptation first

Begin with clear instructions and deterministic application logic around the model. Retrieval supplies current or private knowledge; tools let the application perform authoritative actions; constrained outputs make responses parseable. Fine-tuning is useful when the task requires stable behaviour, terminology or output structure demonstrated by enough consistent examples. It is not a dependable way to inject frequently changing facts. Full custom training is a substantially different programme, requiring a defensible corpus, specialist infrastructure and repeated validation, and is rarely the first rational step for an application feature.

• Build a versioned evaluation set from representative, permitted examples.
• Compare quality, latency and cost at the complete workflow level.
• Test provider limits, data handling and model retirement procedures.
• Measure the effect of retrieval and tools before changing model weights.
• Retain a fallback path for unavailable or unsuitable model responses.

Data determines which options are real

Customisation requires data that is lawful to use, sufficiently representative and consistently labelled. Large archives are not automatically training sets; duplicates, stale decisions and outcome leakage can make evaluation misleading. Sensitive records may require anonymisation, access controls and documented retention. Synthetic examples can cover formats and edge cases but should not replace real distribution checks. The evaluation set must remain separate from training material. Otherwise apparent improvement may be memorisation, and the deployed model can fail on the first genuinely unseen input.

Control includes change management

A model endpoint is a dependency whose behaviour can change. Hosted providers may introduce new versions or retire old ones, while self-hosted models change when weights, quantisation, inference libraries or prompts change. Pin versions where possible, record every component used for a decision and run regression evaluation before promotion. Routing between models can reduce cost or provide resilience, but only if each route meets the same contract. A nominally interchangeable API can differ in tool calling, refusal behaviour, tokenisation and structured-output reliability.

Security and deployment constraints can decide early

Data classification may rule out some providers or require contractual controls, regional processing and restricted retention. Self-hosting can keep inference within a controlled environment, but it does not automatically make the system secure. Model files, prompts, retrieved documents, logs and vector indexes all need access control and lifecycle management. Prompt injection and unsafe tool calls are application concerns regardless of model ownership. Separate untrusted content from instructions, authorise tools outside the model and avoid placing secrets in a context window.

Make the decision reversible

Place model access behind an application-owned interface that captures inputs, outputs, versions, latency and usage without coupling domain logic to one provider’s response shape. Keep prompts, schemas and evaluation cases versioned. A short comparative spike can then test credible candidates against the same workload. The resulting decision record should state why the chosen option meets present constraints and what evidence would trigger reconsideration. This supports changing providers or deployment models without pretending that models are commodities or designing an abstraction too weak for their real differences. Exit testing should confirm that evaluation traces and application data remain exportable without a provider-specific console.