Docs

Run model calls on each customer's AWS or Google Cloud account

This example builds "bring your own models" for a hosted product: each customer connects their own model account, and your backend's model calls run there. A call for Customer 1 runs on Amazon Bedrock in their AWS account or Vertex AI in their Google Cloud project, on their bill and under their governance.

Architecture diagram. Left: Your cloud containing Your backend, OpenAI or Anthropic SDK; AI Gateway, ai.alien.dev. Right: Customer 1's cloud account containing models (AI) containing Bedrock / Vertex AI.

Your backend keeps one model client. The gateway calls the customer's provider with short-lived credentials for their account.

What you build:

  • An eight-line stack that the customer's admin sets up once (the initial setup), granting model access to a role in their account. No compute, network, or stored key is created there.
  • Backend calls through the Alien AI Gateway with your usual OpenAI or Anthropic SDK and the customer's ID.
  • One code path for every customer, whether they connected Bedrock or Vertex AI.

Source: examples/customer-models-ts.

Buyers ask for this because they already have committed spend with their cloud provider, an approved list of models, and a security team that reviewed that provider.

The common alternative is collecting credentials: each customer pastes an IAM user or service account key into your settings page. You then hold long-lived credentials for every customer, write a client per provider, and debug each customer's IAM policy. Many security teams refuse to hand over the key at all.

The AI resource

alien.ts
import * as alien from "@alienplatform/core"

const models = new alien.AI("models").build()

export default new alien.Stack("customer-models")
  .platforms(["aws", "gcp"])
  .add(models, "frozen", { remoteAccess: true })
  .build()
  • alien.AI("models") is a portable handle to the managed model service on each cloud. It has no options: your application picks the model on each request.
  • "frozen" means the initial setup owns the resource. The customer's admin creates it with their own credentials, and a release from you can't replace or delete it. See Frozen and live.
  • { remoteAccess: true } publishes the resource to code outside the customer's cloud. The initial setup creates a separate cloud identity for remote bindings, trusted by Alien, with only the permissions this resource needs. Without this flag, nothing outside the deployment can use the resource.

A remotely published AI resource must be the only remoteAccess resource in its stack. All published resources share one role, so resolving one binding would hand out the authority of all of them. Alien refuses to deploy a stack that pairs it with a remote Storage, Key, or other published resource.

.platforms(["aws", "gcp"]) limits the release to AWS and Google Cloud. The AI resource also supports Azure; see per-cloud behavior for what changes if you add it.

What the initial setup creates

The resource creates almost nothing. Each cloud's controller grants model access and starts watching which models the account can use.

CloudServiceWhat the initial setup doesWhat the remote role can do
AWSAmazon Bedrock in the deployment's regionCreates no resource; applies permissions onlybedrock:InvokeModel, bedrock:InvokeModelWithResponseStream
Google CloudVertex AI on the global locationEnables the Vertex AI API in the projectaiplatform.endpoints.predict, aiplatform.endpoints.explain

On Google Cloud the location is always global, not the deployment's region. Google serves many models only on the global endpoint, and regional endpoints add a price premium. A customer that needs single-region inference can't get it from this resource today.

After the initial setup, Alien checks every 15 minutes which catalog models the account can use. It reads the provider's control plane and never invokes a model, so it consumes no quota:

  • Bedrock: the agreement, entitlement, authorization, and region status of each model.
  • Vertex AI: which publisher models exist in Model Garden.

The AI Gateway uses this observation to decide which models to list for the customer.

The customer can revoke access at any time by deleting the setup stack.

Release the stack

Scaffold the example and publish a release:

alien init customer-models-ts customer-models
cd customer-models
pnpm install
alien release

The release contains only the stack, so it builds and publishes in seconds.

You can't run this stack with alien dev. Remote bindings exist only on AWS, Google Cloud, and Azure deployments, and the gateway needs a real Bedrock or Vertex AI account. Use a test AWS account or Google Cloud project as your first customer.

Onboard a customer

Create a setup link for the customer: a URL that opens the deployment portal, the setup page Alien hosts for their admin. The external ID is the stable ID your own application already uses for this customer, such as their organization ID:

alien onboard "Customer 1" --allow admin@customer-1.com --external-id customer-1

Send the link to the customer's cloud admin. They choose AWS or Google Cloud and run the initial setup with their own credentials, using alien-deploy, Terraform, or on AWS a CloudFormation stack. See Follow a deployment.

Model access may need one more step from the admin:

  • AWS: pick a region where Bedrock offers the models you need, and accept any model agreements in the Bedrock console.
  • Google Cloud: enable Anthropic models in Vertex AI Model Garden.

The initial setup can finish before every model is usable. List the customer's models to tell.

Create a gateway key

Your backend authenticates to the gateway with a project API key that can only call the AI Gateway:

alien api-keys create --for ai-gateway --description production-backend
export ALIEN_AI_API_KEY=<the secret printed once>

Keep it in your backend's secret manager. Never ship it to a browser.

List the customer's models

curl https://ai.alien.dev/v1/models \
  -H "Authorization: Bearer $ALIEN_AI_API_KEY" \
  -H "X-Alien-External-ID: customer-1"

The API key selects your project and X-Alien-External-ID selects the customer. The gateway finds that customer's running deployment with one frozen, remotely published AI resource and lists the models the latest observation marked available:

{
  "object": "list",
  "data": [
    {
      "id": "byo/claude-opus-5",
      "object": "model",
      "provider": "anthropic",
      "displayName": "Claude Opus 5"
    }
  ]
}

Model IDs use the byo/ namespace: the model runs on the customer's own account. The gateway strips the prefix before it calls the provider and also accepts the bare ID.

An empty list means the initial setup worked but no model is usable yet. On Bedrock, the usual causes are an unaccepted agreement, a missing entitlement, or a region that doesn't serve the model.

The list shows what the provider's control plane reports, so treat a real request as the final proof.

Send a request

The CLI prints a ready-to-run request for the active environment:

alien examples ai-gateway --protocol openai-chat --external-id customer-1
curl "https://ai.alien.dev/v1/chat/completions" \
  -H "Authorization: Bearer $ALIEN_AI_API_KEY" \
  -H "X-Alien-External-ID: customer-1" \
  -H "Content-Type: application/json" \
  -d '{
  "messages": [
    {
      "content": "Hello",
      "role": "user"
    }
  ],
  "model": "byo/claude-opus-5"
}'

Pass --model to pick another model, and --protocol openai-responses or --protocol anthropic-messages for the other client APIs. Without --external-id, the command uses $CUSTOMER_ID.

The gateway accepts these routes:

Client APIRoute
OpenAI Chat CompletionsPOST /v1/chat/completions
OpenAI ResponsesPOST /v1/responses
Anthropic MessagesPOST /v1/messages
Model discoveryGET /v1/models

In your backend, point the SDK you already use at the gateway and set the customer header per request:

lib/ai.ts
import OpenAI from "openai"

export function aiFor(customerId: string) {
  return new OpenAI({
    baseURL: "https://ai.alien.dev/v1",
    apiKey: process.env.ALIEN_AI_API_KEY,
    defaultHeaders: { "X-Alien-External-ID": customerId },
  })
}

const res = await aiFor("customer-1").chat.completions.create({
  model: "byo/claude-opus-5",
  messages: [{ role: "user", content: "Summarize this document" }],
})

The same request runs on Bedrock for a customer who connected AWS and on Vertex AI for one who connected Google Cloud. Your code doesn't change.

How a request finds the customer's account

For each request, the gateway:

  1. Authenticates the API key and resolves your project.
  2. Finds the customer's deployments by external ID. You can send X-Alien-Deployment-ID instead to select one deployment directly.
  3. Looks for exactly one deployment whose current release has one frozen AI resource with remoteAccess, in the running state, whose binding matches the cloud: Bedrock on AWS, Vertex AI on Google Cloud.
  4. Checks the requested model against the latest availability observation.
  5. Gets short-lived credentials for the remote bindings role and calls the provider.

If no deployment matches, or more than one does, the gateway returns an error instead of guessing. That happens before the initial setup finishes, after the customer deletes the stack, and when the same customer has two deployments with an AI resource.

Per-cloud behavior

AWSGoogle CloudAzure (not in this example)
ProviderAmazon BedrockVertex AIAzure AI Foundry
LocationDeployment regionglobalDeployment region
Created by the initial setupNothing; permissions onlyVertex AI API enabledAn AI Services account with a predefined set of model deployments

To offer Azure, add "azure" to .platforms(...) and release again. Unlike AWS and Google Cloud, the Azure initial setup creates real resources: an AI Services account and model deployments with provisioned capacity. The customer's subscription must have quota for them.

Without the gateway

The gateway is one way to use the resource. The @alienplatform/bindings package can also resolve the binding directly from your backend, with an API key created with alien api-keys create --for remote-bindings:

import { Bindings } from "@alienplatform/bindings"

const bindings = await Bindings.forRemoteCustomer({
  project: "my-project",
  externalId: "customer-1",
  token: process.env.ALIEN_REMOTE_BINDINGS_KEY!,
})

const lease = await bindings.ai()
// lease.binding: { service: "bedrock", region } or { service: "vertex", project, location }
// lease.clientConfig: short-lived cloud credentials for that binding
// lease.expiresAt: when the credentials expire

Use this when you need a provider feature the gateway doesn't translate, and you are willing to write a client per provider. Prompts then go straight from your backend to the customer's provider.

Before production

  • Resolve the external ID on the server. Derive X-Alien-External-ID from the signed-in account. A caller who can choose the header freely can use another customer's account.
  • Check /v1/models before you show a model as available. A connected account doesn't mean every model is usable: agreements, regions, and quotas differ per account. Availability doesn't promise unused quota at the moment of a request either.
  • Plan a fallback. Decide what your product does for customers who haven't connected an account: use your own provider, or disable the feature.
  • Watch provider errors. alien logs --source ai-gateway --status provider-error --model byo/claude-opus-5 shows failed requests with routing metadata only.
  • Keep the stack isolated. Don't add other resources with remoteAccess to this stack. Put them in their own stack.

If you don't need your own alien.ts, the models setup item deploys an equivalent stack: one frozen AI resource opted into remote access, on AWS, Google Cloud, and Azure. It also lets customers connect providers that don't run in their cloud, such as Anthropic, OpenAI, and Databricks. See Local AI models.

Adapt it

Prompt
Use the alien customer-models-ts example (github.com/alienplatform/alien/tree/main/examples/customer-models-ts) as a model. Add an alien.ts with one frozen AI resource added with { remoteAccess: true }, and no other remoteAccess resources. In my hosted app, route LLM calls for customers who connected their own account to https://ai.alien.dev/v1 with my Alien AI Gateway key and the header X-Alien-External-ID set to the customer's ID, derived on the server. Check GET /v1/models before offering a model, and keep my current provider for everyone else. Read https://alien.dev/docs/guide/local-ai-models.md and https://alien.dev/docs/guide/local-ai-models/in-a-deployment.md first.

Next

On this page