Process events in the customer's cloud
This example builds an event pipeline that runs in the customer's cloud, where the events are produced, with no credentials for their queues or buckets in your product. Only the results you ask for leave the account.
Architecture diagram. Left: Your cloud, Your backend. Right: Customer cloud containing inbox (Queue); data (Storage); hourly; processor (Worker); events (Kv).
You build one Worker, serverless code Alien runs in the customer's account, that:
- Handles each message on the
inboxqueue. - Reacts to new files in the
databucket. - Runs every hour on a schedule.
- Records each event in a KV store, which your backend reads with commands (named calls into the deployment).
Source: examples/event-pipeline-ts.
Stack
import * as alien from "@alienplatform/core"
const inbox = new alien.Queue("inbox").build()
const data = new alien.Storage("data").build()
const events = new alien.Kv("events").build()
const processor = new alien.Worker("processor")
.code({ type: "source", src: "./", toolchain: { type: "typescript" } })
.commandsEnabled(true)
.publicEndpoint("api")
.link(inbox)
.link(data)
.link(events)
.trigger({ type: "queue", queue: inbox.ref() })
.trigger({ type: "storage", storage: data.ref(), events: ["created"] })
.trigger({ type: "schedule", cron: "0 * * * *" })
.permissions("execution")
.build()
export default new alien.Stack("event-pipeline")
.platforms(["aws", "gcp", "azure"])
.add(inbox, "frozen")
.add(data, "frozen")
.add(events, "frozen")
.add(processor, "live")
.permissions({
profiles: {
execution: {
"*": [
"queue/data-read",
"queue/data-write",
"storage/data-read",
"storage/data-write",
"kv/data-read",
"kv/data-write",
],
},
},
})
.build().link() and .trigger() do different things:
.link(inbox)lets code callqueue("inbox")..trigger({ type: "queue", queue: inbox.ref() })makes a message oninboxinvoke the Worker.
Here the Worker both sends to the queue (from a command and an HTTP route) and consumes it.
Each trigger fires on a different event:
queuefires once per message oninbox. Addbatch: { maxMessages, maxWindowSeconds }to receive messages in batches. A Worker can have one queue trigger.storagefires when an object is created indata. Use"deleted"too if you need it.schedulefires on a standard 5-field cron expression, here every hour on the hour.
The queue, bucket, and KV are frozen: the customer's admin creates them once, during initial setup (the one-time step with their own credentials), and releases can't delete them or their data. The Worker is live and updates on every release. See Frozen and live.
The stack is limited to AWS, Google Cloud, and Azure because Kubernetes doesn't support Worker triggers.
Handlers
// Process messages from the inbox queue
onQueueMessage("*", async message => {
const ev = kv("events")
const payload =
typeof message.payload === "string"
? message.payload
: new TextDecoder().decode(message.payload as Uint8Array)
await ev.setJson(`queue:${message.id}`, {
type: "queue",
source: message.source,
payload,
processedAt: new Date().toISOString(),
})
})
// React to new files in storage
onStorageEvent("*", async event => {
const ev = kv("events")
const key = event.objectKey.replace(/\//g, "_")
await ev.setJson(`storage:${key}`, {
type: "storage",
bucket: event.bucketName,
objectKey: event.objectKey,
eventType: event.eventType,
size: event.size,
processedAt: new Date().toISOString(),
})
})
// Run on a schedule (every hour)
onCronEvent("*", async event => {
const ev = kv("events")
await ev.setJson(`cron:${Date.now()}`, {
type: "cron",
schedule: event.scheduleName,
scheduledTime: event.timestamp,
processedAt: new Date().toISOString(),
})
})"*" handles every queue, bucket, or schedule wired to this Worker. Pass a name, such as onQueueMessage("inbox", ...), when one Worker has several sources that need different code.
A handler that returns acknowledges the event. A handler that throws gets the message redelivered.
Queue and storage triggers deliver at least once, so a handler can run twice for one event. These handlers are idempotent: the queue record is keyed by message.id and the storage record by the object key, so a redelivery overwrites the same KV entry.
Results for your backend
command(
"get-events",
z.object({ type: z.string().optional(), limit: z.number().optional() }),
async ({ type, limit }) => {
const ev = kv("events")
const prefix = type ? `${type}:` : ""
const results: { key: string; value: unknown }[] = []
for await (const entry of scanAll(ev, prefix)) {
results.push({
key: entry.key,
value: JSON.parse(new TextDecoder().decode(entry.value)),
})
if (limit && results.length >= limit) break
}
return { events: results, count: results.length }
},
)
command("get-stats", async () => {
const ev = kv("events")
let queueCount = 0
let storageCount = 0
let cronCount = 0
for await (const _ of scanAll(ev, "queue:")) queueCount++
for await (const _ of scanAll(ev, "storage:")) storageCount++
for await (const _ of scanAll(ev, "cron:")) cronCount++
return { queue: queueCount, storage: storageCount, cron: cronCount }
})
command("send-test-message", z.object({ message: z.string() }), async ({ message }) => {
const q = queue("inbox")
await q.send(message)
return { sent: true }
})scanAll follows the KV scan cursor across pages, as in the webhook example. send-test-message lets you drive the pipeline during development.
Work over HTTP
The same Worker also serves HTTP. POST /ingest puts a message on the queue and returns right away:
// Enqueue a message over HTTP. The response returns as soon as the message is
// queued; the audit-log write is handed to `waitUntil`, so it runs in the
// background after the response is sent instead of blocking the caller.
app.post("/ingest", async c => {
const { message } = await c.req.json<{ message: string }>()
const q = queue("inbox")
await q.send(message)
waitUntil(
kv("events").setJson(`audit:${Date.now()}`, {
type: "http-ingest",
message,
at: new Date().toISOString(),
}),
)
return c.json({ queued: true })
})waitUntil keeps the instance alive until the promise settles, even after the response is sent. Use it for work the caller shouldn't wait for; its errors are logged, not returned.
GET /stats returns the same counts as get-stats.
Run it locally
alien init event-pipeline-ts event-pipeline
cd event-pipeline
alien devalien dev runs the queue, bucket, KV, and all three triggers locally. Send a message, from a second terminal:
alien dev commands invoke --deployment local-dev/default \
--command send-test-message --params '{"message": "hello from queue"}'{ "sent": true }The local trigger service polls the queue and calls the handler, so wait a few seconds, then read the result:
alien dev commands invoke --deployment local-dev/default \
--command get-events --params '{"type": "queue"}'{
"events": [
{
"key": "queue:<message-id>",
"value": {
"type": "queue",
"source": "inbox",
"payload": "hello from queue",
"processedAt": "2026-10-24T12:00:03.000Z"
}
}
],
"count": 1
}alien dev commands invoke --deployment local-dev/default --command get-stats{ "queue": 1, "storage": 0, "cron": 0 }The HTTP path works the same way:
curl -X POST <url>/ingest -H "Content-Type: application/json" -d '{"message": "hello over http"}'{ "queued": true }Test it
tests/pipeline.test.ts sends a message, polls get-events every two seconds for up to 30 seconds, and checks that the payload arrived as hello from queue and that get-stats counts it:
pnpm testDeploy it
alien login
alien link
alien release
alien onboard customer-1 --allow admin@customer-1.comalien onboard prints a setup link, a URL that opens the setup page Alien hosts for the customer's admin. The admin opens it and runs the initial setup once. After that:
- Upload a file to the
databucket in the customer's account, and astorage:record appears. - Within the hour, the first
cron:record appears. - Later releases replace the Worker without the admin.
How it works on each platform
| Platform | processor | inbox and its trigger | data and its trigger | Schedule | events |
|---|---|---|---|---|---|
| AWS | Lambda | SQS queue, event source mapping | S3 bucket, S3 notifications | EventBridge | DynamoDB table |
| Google Cloud | Cloud Run | Pub/Sub topic and subscription, push | Cloud Storage bucket, notifications | Cloud Scheduler | Firestore collection |
| Azure | Container Apps | Service Bus queue, KEDA | Storage account, Blob storage binding | Cron binding | Table Storage table |
alien dev | Native process | Local queue | Directory on disk | Local scheduler | SQLite file |
On AWS, the SQS visibility timeout follows the Worker's timeout: max(30s, min(12h, timeout × 6)).
Before production
- Keep messages small. A queue message is at most 64 KiB. Put large payloads in Storage and send the key.
- Make KV keys valid. KV keys allow only
a-z A-Z 0-9 - _ : .and at most 512 bytes. The storage handler replaces/but not spaces or other characters. Hash or encode the object key first. - Use a dead-letter path. A message whose handler always throws is redelivered. Catch errors you can't recover from, record them, and return.
- Remove the test surface. Drop
send-test-message, and authenticate or remove/ingestand the public endpoint if nothing outside your backend sends work.
Next
- Receive webhooks in the customer's cloud
- Worker reference: triggers, handlers, and limits
- Queue reference
- Compute
Webhooks and an API
Give each customer an HTTPS endpoint that stores webhooks in their own account, query the events from your backend with commands, and configure each deployment with stack inputs.
Route commands
Call the same command name on two resources in one customer deployment and pick the one that answers with a target.