Docs

Query a private database

This example builds a connector that queries a customer's database your product can't reach, without a VPN, peering, or their password in your systems. The connector runs beside the database and returns only the rows you allow.

Architecture diagram. Left: Your cloud, Your backend. Right: Customer network containing connector (Worker); credentials (Vault); cache (Kv); customer's database.

The connector reads credentials from the Vault and queries the database inside the customer's network. Only the rows you allow come back.

You build the connector as a Worker, serverless code Alien runs in the customer's account, that:

  • Reads the connection config from a vault in the customer's account.
  • Exposes named operations (test-connection, query, list-tables) as commands your backend calls.
  • Caches query results in a KV store.

The same handlers could also run from a schedule or a queue inside the customer's account.

Source: examples/data-connector-ts.

Why not connect from your backend

Querying from your cloud needs two things the customer doesn't want to give you:

  • A network path into their VPC: a VPN, peering, or a public IP on the database.
  • A database password stored in your systems.

A connector removes both. The database stays private, and the password stays in the customer's own secret store, read only by the Worker at runtime. Your backend gets named operations and their results, never a connection.

Queries with several round trips also get faster, because the round trips happen inside the customer's network.

The connector

alien.ts
import * as alien from "@alienplatform/core"

const credentials = new alien.Vault("credentials").build()
const cache = new alien.Kv("cache").build()

const connector = new alien.Worker("connector")
  .code({ type: "source", src: "./", toolchain: { type: "typescript" } })
  .commandsEnabled(true)
  .link(credentials)
  .link(cache)
  .permissions("execution")
  .build()

export default new alien.Stack("data-connector")
  .add(credentials, "frozen")
  .add(cache, "frozen")
  .add(connector, "live")
  .permissions({
    profiles: {
      execution: {
        "*": ["vault/data-read", "kv/data-read", "kv/data-write"],
      },
    },
  })
  .build()
  • credentials is a Vault: secrets in the cloud's native secret service. The connector can read it (vault/data-read) but not write it.
  • cache is a KV store for query results, so repeated questions don't hit the database.
  • Both are frozen: the customer's admin creates them once, during the initial setup (the one-time step with their own credentials), and releases can't replace them. The connector is live and updates on every release. See Frozen and live.
  • There's no public endpoint. Commands are the only way in.

Reading the connection

src/index.ts
async function getConnectionConfig() {
  const v = await vault("credentials")
  const raw = await v.get("database")
  return JSON.parse(raw)
}

The secret named database holds a JSON connection config. get() throws if the secret is missing, so a connector that hasn't been configured fails loudly on the first command.

Operations your product can call

src/index.ts
command("test-connection", async () => {
  const config = await getConnectionConfig()
  return {
    connected: true,
    database: config.database,
    host: config.host,
    // password is never returned: it stays in the customer's cloud
  }
})

command(
  "query",
  z.object({ sql: z.string(), useCache: z.boolean().optional() }),
  async ({ sql, useCache }) => {
    const c = await kv("cache")
    const cacheKey = `query:${createHash("sha256").update(sql).digest("hex").slice(0, 16)}`

    if (useCache) {
      const cached = await c.get(cacheKey)
      if (cached) {
        return { ...JSON.parse(new TextDecoder().decode(cached.value)), cached: true }
      }
    }

    // In production, use the connection config to connect to the actual database
    await getConnectionConfig()
    const result = simulateQuery(sql)

    if (useCache) {
      await c.set(cacheKey, JSON.stringify(result))
    }

    return { ...result, cached: false }
  },
)

command("list-tables", async () => {
  return { tables: Object.keys(SAMPLE_DATA) }
})
  • test-connection proves the secret is readable and returns only non-sensitive fields.
  • query hashes the SQL into a cache key, returns a cached result when asked, and otherwise runs the query. Every response says whether it came from the cache.
  • list-tables returns the table names.

The template answers from in-memory sample data (users and orders) through simulateQuery, so it runs without a database. Replace simulateQuery with a real client such as pg or mysql2, built from the config in the vault.

Run it locally

alien init data-connector-ts private-database
cd private-database
alien dev

The vault starts empty. In a second terminal, play the customer's admin and store a connection config as the database secret:

cat > db.json <<'EOF'
{"host": "db.customer.internal", "port": 5432, "database": "warehouse", "user": "readonly", "password": "local-only"}
EOF
alien dev vault set credentials database --file db.json

Locally the vault is a file under .alien/. Now check the connection:

alien dev commands invoke --deployment local-dev/default --command test-connection
{ "connected": true, "database": "warehouse", "host": "db.customer.internal" }

The password isn't in the response. Run a query:

alien dev commands invoke --deployment local-dev/default \
  --command query --params '{"sql": "SELECT * FROM users"}'
{
  "rows": [
    { "id": 1, "name": "Alice", "email": "alice@example.com", "role": "admin" },
    { "id": 2, "name": "Bob", "email": "bob@example.com", "role": "user" },
    { "id": 3, "name": "Charlie", "email": "charlie@example.com", "role": "user" }
  ],
  "rowCount": 3,
  "cached": false
}

Run the same orders query twice with "useCache": true. The first response has "cached": false and fills the cache. The second has "cached": true and never touches the database:

alien dev commands invoke --deployment local-dev/default \
  --command query --params '{"sql": "SELECT * FROM orders", "useCache": true}'

list-tables returns { "tables": ["users", "orders"] }.

Test it

The test plays both roles. It deploys locally and writes the secret the way a customer would. Then it checks that:

  • test-connection never returns password.
  • A query returns rows.
  • The second cached query reports cached: true.
tests/connector.test.ts
beforeAll(async () => {
  deployment = await deploy({ app: ".", platform: "local" })

  await deployment.setExternalSecret(
    "credentials",
    "database",
    JSON.stringify({
      host: "db.customer.internal",
      port: 5432,
      database: "warehouse",
      user: "readonly",
      password: "customer-secret-password",
    }),
  )
}, 300_000)
pnpm test

Deploy it next to the database

Release and onboard as usual:

alien release
alien onboard customer-1 --allow admin@customer-1.com

alien onboard creates a setup link, a URL that opens the setup page Alien hosts for the customer's admin, who runs the initial setup once.

The connector has to run where it can reach the database. On AWS, a Worker runs in the deployment's private subnets when the deployment has a network.

The admin can point the deployment at their existing VPC during initial setup with --network byo. See Networking and endpoints.

Then call the connector from your backend:

backend/warehouse.ts
import { CommandsClient } from "@alienplatform/commands"

const commands = await CommandsClient.forDeployment({
  deploymentId: customer.deploymentId,
  apiKey: process.env.ALIEN_API_KEY!,
})

const { rows } = await commands
  .target("connector")
  .invoke<{ rows: unknown[]; rowCount: number }>("query", {
    sql: "SELECT * FROM orders",
    useCache: true,
  })

How it works on each platform

Platformconnectorcredentialscache
AWSLambdaSSM Parameter Store SecureString parametersDynamoDB table
Google CloudCloud RunSecret Manager secretsFirestore collection
AzureContainer AppsKey VaultTable Storage table
alien devNative processJSON file on diskSQLite file

Alien's management identity (the identity initial setup creates for ongoing management) can't read the vault by default. Only the connector's identity, with vault/data-read, can.

Before production

  • Replace free-form SQL with named queries. query accepts any SQL string, which hands your backend, and anything that can reach it, the full power of the database user. Expose operations such as orders-by-status with bound parameters instead, as Chat with private Postgres does.
  • Collect the secret during initial setup. For a value the admin types once, declare a deployer secret input. Initial setup then shows the admin where to write it, and the deployment waits until it exists.
  • Bound the cache. Pass { ttl } to c.set() so stale results expire, and keep cached values under the 24 KiB KV value limit.
  • Use a read-only database user in the connection config.

Next

On this page