Local AI models
Local AI models (bring your own LLM) run your product's AI requests on a model account each customer already controls: their Amazon Bedrock, Vertex AI, Azure AI Foundry, OpenAI, or Anthropic account. Usage counts against their commitment, not your AI bill, and only their approved models are used.
Your backend keeps one OpenAI-shaped client pointed at ai.alien.dev. Each request carries a customer ID, and Alien AI Gateway routes it to the AI provider account that customer connected — AWS Bedrock, Vertex AI, OpenAI, and others.
Why customers ask for it
- Committed AI spend. Large companies have commitments with OpenAI, Anthropic, AWS, Google Cloud, or Microsoft, and want your product's usage to count toward them.
- AI governance. Company policy may allow only some providers, clouds, regions, or models.
- Their data terms. Their own agreement with the provider, such as zero data retention, covers your requests.
Your application never receives their provider credentials.
Pick a path
| Where the code that calls the model runs | Use | Do prompts leave the customer's cloud? |
|---|---|---|
| Your hosted product | The AI Gateway | Yes, they pass through Alien on the way to the customer's provider |
| A deployment in the customer's cloud | The AI resource | No |
From your hosted product, your backend keeps one OpenAI-compatible integration. The AI Gateway, an API Alien hosts at ai.alien.dev, handles each customer's credentials, IAM roles, regions, and routing.
Add it to an OpenAI client
Point an existing OpenAI client at the gateway, send the customer's external ID (your own stable ID for that customer), and use a model ID the gateway lists for them:
import OpenAI from "openai"
const ai = new OpenAI({
baseURL: "https://ai.alien.dev/v1",
apiKey: process.env.ALIEN_AI_API_KEY,
defaultHeaders: {
"X-Alien-External-ID": customer.id,
},
})
const response = await ai.chat.completions.create({
model: "byo/claude-opus-5",
messages: [{ role: "user", content: "Hello" }],
})The gateway accepts OpenAI Chat Completions, OpenAI Responses, and Anthropic Messages requests, plus model discovery. Python, Rust, and the Anthropic SDK are on Add to your product.
Routing by customer
Each request carries three values:
- The AI Gateway API key selects your Alien project.
X-Alien-External-IDselects which customer's provider connection to use.modelselects a model available through that connection.
Most AI gateways and model routers pick a provider from connections you configured. The AI Gateway picks the connection the customer configured. You can use both: they solve different problems.
| Provider connection used | Model used | |
|---|---|---|
| Model router | One of the connections you configured | Chosen by your routing rules |
| AI Gateway | The connection that customer configured | Requested by your application, from that connection's models |
How customers connect a provider
Each customer's admin connects their provider once, from a settings page in your product. It opens the deployment portal, the branded setup page Alien hosts.
| Provider | What the customer grants |
|---|---|
| OpenAI, Anthropic, Databricks | An API key from their account |
| Amazon Bedrock | An IAM role in their AWS account, created by CloudFormation or Terraform |
| Vertex AI, Azure AI Foundry | An identity in their Google Cloud project or Azure tenant |
Each cloud provider needs its own trust setup, done with the customer's own cloud access. That's why connecting is a guided flow, not a text field on your settings page.
What Alien records
The gateway forwards each request, so prompts and responses pass through Alien on the way to the customer's provider.
- Logs record routing and outcome metadata: model, provider, status, latency, and token counts.
- Logs never record prompt or response bodies.
- Usage reports aggregate the same metadata.
To keep prompts inside the customer's cloud, run the code that calls the model there.
Inside a deployment
When the code that calls the model already runs in a deployment in the customer's cloud, declare an AI resource and link it:
import * as alien from "@alienplatform/core"
const llm = new alien.AI("llm").build()
const agent = new alien.Worker("agent")
.code({ type: "source", src: "./", toolchain: { type: "typescript" } })
.link(llm)
.build()Your code calls the provider in the same account with the workload's own cloud identity: no API keys, and prompts never leave the customer's cloud. The provider is Amazon Bedrock on AWS, Vertex AI on Google Cloud, and Azure AI Foundry on Azure. See Local AI models in a deployment.
Open-weight models
- Managed open-weight models. Bedrock, Vertex AI, and Azure AI Foundry host families such as Llama, Mistral, Qwen, and GPT-OSS. They work through either path, with no GPUs to manage.
- Your own GPUs. Run a model server, such as vLLM, in a
Containerwith GPUs in the customer's cloud. The weights, the inference, and the data all stay in their account. See Serve open-weight models on GPUs.