AI Integration and Deployment
A model is a dependency that is slower, more expensive and less reliable than the rest of your stack. It has to be integrated like one.
- Typical duration
- Two to six months
- Team shape
- Two to four senior engineers
- Starts with
- A failure-mode review
Design for the day the model is unavailable
Model providers have outages, rate limits and latency that varies by an order of magnitude between calls. Integrated carelessly, that becomes your application's outage, your application's timeout, and a bill nobody forecast.
So the first conversation is about failure rather than capability: what the product does when the model is slow, when it is unavailable, and when it returns something unusable. Every one of those needs a designed answer before the feature ships, because the alternative is discovering all three in the same week, in production, with no fallback to switch to.
What usually goes wrong, and what we do instead
AI integration failures are ordinary integration failures with a larger bill attached.
Where AI integrations go wrong
- A synchronous call in a request path, so provider latency becomes yours
- No fallback, so a provider outage is a product outage
- Costs discovered at the end of the month rather than budgeted per call
- Prompts changed in production with no way to compare before and after
- Sensitive data sent to a provider because nobody checked what was in it
How we build it instead
- Model calls off the critical path wherever the product allows it
- A designed fallback for slow, unavailable and unusable responses
- Cost budgeted and metered per feature, with alerting on spend
- Prompts versioned and evaluated, so a change is a measured decision
- Data classified before it leaves, with redaction where it must not
What we build
The parts of an AI integration that decide whether it survives a bad day at the provider.
Model gateway
One internal interface in front of every provider, so a switch is configuration rather than a rewrite.
Fallbacks and degradation
A designed answer for slow, unavailable and unusable, rather than an error page and an apology.
Cost control
Budgets and metering per feature, with caching for the repeated requests that quietly dominate spend.
Data boundaries
Classification and redaction before anything leaves, with self-hosted models where it must not.
Prompt and model versioning
Changes evaluated against a scored set before release, so a tweak is not a gamble.
Observability
Latency, error rate, spend and output quality per feature, alerting before users notice.
How the pieces fit together
The model sits behind your own service rather than being called from your application directly, which is what makes every other decision here possible.
What each part does
- Your application: calls one internal interface and never learns which provider is behind it or that it changed.
- The gateway: routing, retries, timeouts and rate limiting in one place, so behaviour is consistent across features.
- Redaction: data classified and stripped before it leaves your boundary, according to a policy rather than a habit.
- Providers: more than one where the workload allows, so an outage is a routing change rather than an incident.
- Cache and budget: repeated requests served without a call, and per-feature spend metered with alerting.
- The fallback: what the product does when the model is slow, down or unusable — designed, not discovered.
How the work runs
The same five stages as every engagement, applied to a dependency you do not control.
Diagnose
Where the model would sit, what data would leave, and what the product should do when it fails.
Decide
The gateway, the fallback behaviour and the data policy, written down before the first feature is wired.
Prove
One feature in production behind the gateway, with its fallback exercised deliberately rather than assumed.
Deliver
Features integrated in slices, each metered and monitored before the next is scoped.
Hand over
Your team changes a provider and a prompt with us watching, then without us.
What we build it with
Chosen for what has to be operated afterwards, and by whom.
Models
- Anthropic
- OpenAI
- Gemini
- PyTorch
Services
- Python
- TypeScript
- Node.js
- REST
- Temporal
Data
- PostgreSQL
- pgvector
- Redis
- Kafka
Platform
- AWS
- Azure
- Kubernetes
- Datadog
- Sentry
What you get, and when
Handed over as it is produced, not assembled at the end.
- The failure modes and the designed answer to each
- A data classification and redaction policy, agreed
- A forecast running cost per feature, before anything is wired
- Source in repositories you own
- A gateway fronting every provider behind one interface
- Prompts versioned, with an evaluation run on every change
- Latency, spend and quality dashboards per feature
- Budget alerts, so a runaway loop is caught in hours
- A runbook for a provider outage your team has rehearsed
The rest of our AI work
Most AI engagements need one of these underneath them, and some need two.
Related work
Engagements where an unreliable dependency had to sit inside a reliable product.
Questions about AI integration
The ones we are asked most often before a first conversation.
What happens if the provider has an outage?
How do we stop the bill running away?
Can we run models in our own environment?
How do we know what data is being sent?
Model calls are slow. How do we handle that?
How hard is it to change model later?
Adding a model to a system that has to stay up?
Tell us where it would sit and what data would leave. We will tell you what has to happen when it fails.