Docs

Process events in the customer's cloud

This example builds an event pipeline that runs in the customer's cloud, where the events are produced, with no credentials for their queues or buckets in your product. Only the results you ask for leave the account.

Architecture diagram. Left: Your cloud, Your backend. Right: Customer cloud containing inbox (Queue); data (Storage); hourly; processor (Worker); events (Kv).

Queue messages, new files, and an hourly schedule trigger one Worker. Your backend reads the results with commands.

You build one Worker, serverless code Alien runs in the customer's account, that:

  • Handles each message on the inbox queue.
  • Reacts to new files in the data bucket.
  • Runs every hour on a schedule.
  • Records each event in a KV store, which your backend reads with commands (named calls into the deployment).

Source: examples/event-pipeline-ts.

Stack

alien.ts
import * as alien from "@alienplatform/core"

const inbox = new alien.Queue("inbox").build()
const data = new alien.Storage("data").build()
const events = new alien.Kv("events").build()

const processor = new alien.Worker("processor")
  .code({ type: "source", src: "./", toolchain: { type: "typescript" } })
  .commandsEnabled(true)
  .publicEndpoint("api")
  .link(inbox)
  .link(data)
  .link(events)
  .trigger({ type: "queue", queue: inbox.ref() })
  .trigger({ type: "storage", storage: data.ref(), events: ["created"] })
  .trigger({ type: "schedule", cron: "0 * * * *" })
  .permissions("execution")
  .build()

export default new alien.Stack("event-pipeline")
  .platforms(["aws", "gcp", "azure"])
  .add(inbox, "frozen")
  .add(data, "frozen")
  .add(events, "frozen")
  .add(processor, "live")
  .permissions({
    profiles: {
      execution: {
        "*": [
          "queue/data-read",
          "queue/data-write",
          "storage/data-read",
          "storage/data-write",
          "kv/data-read",
          "kv/data-write",
        ],
      },
    },
  })
  .build()

.link() and .trigger() do different things:

  • .link(inbox) lets code call queue("inbox").
  • .trigger({ type: "queue", queue: inbox.ref() }) makes a message on inbox invoke the Worker.

Here the Worker both sends to the queue (from a command and an HTTP route) and consumes it.

Each trigger fires on a different event:

  • queue fires once per message on inbox. Add batch: { maxMessages, maxWindowSeconds } to receive messages in batches. A Worker can have one queue trigger.
  • storage fires when an object is created in data. Use "deleted" too if you need it.
  • schedule fires on a standard 5-field cron expression, here every hour on the hour.

The queue, bucket, and KV are frozen: the customer's admin creates them once, during initial setup (the one-time step with their own credentials), and releases can't delete them or their data. The Worker is live and updates on every release. See Frozen and live.

The stack is limited to AWS, Google Cloud, and Azure because Kubernetes doesn't support Worker triggers.

Handlers

src/index.ts
// Process messages from the inbox queue
onQueueMessage("*", async message => {
  const ev = kv("events")
  const payload =
    typeof message.payload === "string"
      ? message.payload
      : new TextDecoder().decode(message.payload as Uint8Array)

  await ev.setJson(`queue:${message.id}`, {
    type: "queue",
    source: message.source,
    payload,
    processedAt: new Date().toISOString(),
  })
})

// React to new files in storage
onStorageEvent("*", async event => {
  const ev = kv("events")
  const key = event.objectKey.replace(/\//g, "_")

  await ev.setJson(`storage:${key}`, {
    type: "storage",
    bucket: event.bucketName,
    objectKey: event.objectKey,
    eventType: event.eventType,
    size: event.size,
    processedAt: new Date().toISOString(),
  })
})

// Run on a schedule (every hour)
onCronEvent("*", async event => {
  const ev = kv("events")

  await ev.setJson(`cron:${Date.now()}`, {
    type: "cron",
    schedule: event.scheduleName,
    scheduledTime: event.timestamp,
    processedAt: new Date().toISOString(),
  })
})

"*" handles every queue, bucket, or schedule wired to this Worker. Pass a name, such as onQueueMessage("inbox", ...), when one Worker has several sources that need different code.

A handler that returns acknowledges the event. A handler that throws gets the message redelivered.

Queue and storage triggers deliver at least once, so a handler can run twice for one event. These handlers are idempotent: the queue record is keyed by message.id and the storage record by the object key, so a redelivery overwrites the same KV entry.

Results for your backend

src/index.ts
command(
  "get-events",
  z.object({ type: z.string().optional(), limit: z.number().optional() }),
  async ({ type, limit }) => {
    const ev = kv("events")
    const prefix = type ? `${type}:` : ""
    const results: { key: string; value: unknown }[] = []

    for await (const entry of scanAll(ev, prefix)) {
      results.push({
        key: entry.key,
        value: JSON.parse(new TextDecoder().decode(entry.value)),
      })
      if (limit && results.length >= limit) break
    }

    return { events: results, count: results.length }
  },
)

command("get-stats", async () => {
  const ev = kv("events")
  let queueCount = 0
  let storageCount = 0
  let cronCount = 0

  for await (const _ of scanAll(ev, "queue:")) queueCount++
  for await (const _ of scanAll(ev, "storage:")) storageCount++
  for await (const _ of scanAll(ev, "cron:")) cronCount++

  return { queue: queueCount, storage: storageCount, cron: cronCount }
})

command("send-test-message", z.object({ message: z.string() }), async ({ message }) => {
  const q = queue("inbox")
  await q.send(message)
  return { sent: true }
})

scanAll follows the KV scan cursor across pages, as in the webhook example. send-test-message lets you drive the pipeline during development.

Work over HTTP

The same Worker also serves HTTP. POST /ingest puts a message on the queue and returns right away:

src/index.ts
// Enqueue a message over HTTP. The response returns as soon as the message is
// queued; the audit-log write is handed to `waitUntil`, so it runs in the
// background after the response is sent instead of blocking the caller.
app.post("/ingest", async c => {
  const { message } = await c.req.json<{ message: string }>()
  const q = queue("inbox")
  await q.send(message)

  waitUntil(
    kv("events").setJson(`audit:${Date.now()}`, {
      type: "http-ingest",
      message,
      at: new Date().toISOString(),
    }),
  )

  return c.json({ queued: true })
})

waitUntil keeps the instance alive until the promise settles, even after the response is sent. Use it for work the caller shouldn't wait for; its errors are logged, not returned.

GET /stats returns the same counts as get-stats.

Run it locally

alien init event-pipeline-ts event-pipeline
cd event-pipeline
alien dev

alien dev runs the queue, bucket, KV, and all three triggers locally. Send a message, from a second terminal:

alien dev commands invoke --deployment local-dev/default \
  --command send-test-message --params '{"message": "hello from queue"}'
{ "sent": true }

The local trigger service polls the queue and calls the handler, so wait a few seconds, then read the result:

alien dev commands invoke --deployment local-dev/default \
  --command get-events --params '{"type": "queue"}'
{
  "events": [
    {
      "key": "queue:<message-id>",
      "value": {
        "type": "queue",
        "source": "inbox",
        "payload": "hello from queue",
        "processedAt": "2026-10-24T12:00:03.000Z"
      }
    }
  ],
  "count": 1
}
alien dev commands invoke --deployment local-dev/default --command get-stats
{ "queue": 1, "storage": 0, "cron": 0 }

The HTTP path works the same way:

curl -X POST <url>/ingest -H "Content-Type: application/json" -d '{"message": "hello over http"}'
{ "queued": true }

Test it

tests/pipeline.test.ts sends a message, polls get-events every two seconds for up to 30 seconds, and checks that the payload arrived as hello from queue and that get-stats counts it:

pnpm test

Deploy it

alien login
alien link
alien release
alien onboard customer-1 --allow admin@customer-1.com

alien onboard prints a setup link, a URL that opens the setup page Alien hosts for the customer's admin. The admin opens it and runs the initial setup once. After that:

  • Upload a file to the data bucket in the customer's account, and a storage: record appears.
  • Within the hour, the first cron: record appears.
  • Later releases replace the Worker without the admin.

How it works on each platform

Platformprocessorinbox and its triggerdata and its triggerScheduleevents
AWSLambdaSQS queue, event source mappingS3 bucket, S3 notificationsEventBridgeDynamoDB table
Google CloudCloud RunPub/Sub topic and subscription, pushCloud Storage bucket, notificationsCloud SchedulerFirestore collection
AzureContainer AppsService Bus queue, KEDAStorage account, Blob storage bindingCron bindingTable Storage table
alien devNative processLocal queueDirectory on diskLocal schedulerSQLite file

On AWS, the SQS visibility timeout follows the Worker's timeout: max(30s, min(12h, timeout × 6)).

Before production

  • Keep messages small. A queue message is at most 64 KiB. Put large payloads in Storage and send the key.
  • Make KV keys valid. KV keys allow only a-z A-Z 0-9 - _ : . and at most 512 bytes. The storage handler replaces / but not spaces or other characters. Hash or encode the object key first.
  • Use a dead-letter path. A message whose handler always throws is redelivered. Catch errors you can't recover from, record them, and return.
  • Remove the test surface. Drop send-test-message, and authenticate or remove /ingest and the public endpoint if nothing outside your backend sends work.

Next

On this page