Query a private database
This example builds a connector that queries a customer's database your product can't reach, without a VPN, peering, or their password in your systems. The connector runs beside the database and returns only the rows you allow.
Architecture diagram. Left: Your cloud, Your backend. Right: Customer network containing connector (Worker); credentials (Vault); cache (Kv); customer's database.
You build the connector as a Worker, serverless code Alien runs in the customer's account, that:
- Reads the connection config from a vault in the customer's account.
- Exposes named operations (
test-connection,query,list-tables) as commands your backend calls. - Caches query results in a KV store.
The same handlers could also run from a schedule or a queue inside the customer's account.
Source: examples/data-connector-ts.
Why not connect from your backend
Querying from your cloud needs two things the customer doesn't want to give you:
- A network path into their VPC: a VPN, peering, or a public IP on the database.
- A database password stored in your systems.
A connector removes both. The database stays private, and the password stays in the customer's own secret store, read only by the Worker at runtime. Your backend gets named operations and their results, never a connection.
Queries with several round trips also get faster, because the round trips happen inside the customer's network.
The connector
import * as alien from "@alienplatform/core"
const credentials = new alien.Vault("credentials").build()
const cache = new alien.Kv("cache").build()
const connector = new alien.Worker("connector")
.code({ type: "source", src: "./", toolchain: { type: "typescript" } })
.commandsEnabled(true)
.link(credentials)
.link(cache)
.permissions("execution")
.build()
export default new alien.Stack("data-connector")
.add(credentials, "frozen")
.add(cache, "frozen")
.add(connector, "live")
.permissions({
profiles: {
execution: {
"*": ["vault/data-read", "kv/data-read", "kv/data-write"],
},
},
})
.build()credentialsis a Vault: secrets in the cloud's native secret service. The connector can read it (vault/data-read) but not write it.cacheis a KV store for query results, so repeated questions don't hit the database.- Both are frozen: the customer's admin creates them once, during the initial setup (the one-time step with their own credentials), and releases can't replace them. The connector is live and updates on every release. See Frozen and live.
- There's no public endpoint. Commands are the only way in.
Reading the connection
async function getConnectionConfig() {
const v = await vault("credentials")
const raw = await v.get("database")
return JSON.parse(raw)
}The secret named database holds a JSON connection config. get() throws if the secret is missing, so a connector that hasn't been configured fails loudly on the first command.
Operations your product can call
command("test-connection", async () => {
const config = await getConnectionConfig()
return {
connected: true,
database: config.database,
host: config.host,
// password is never returned: it stays in the customer's cloud
}
})
command(
"query",
z.object({ sql: z.string(), useCache: z.boolean().optional() }),
async ({ sql, useCache }) => {
const c = await kv("cache")
const cacheKey = `query:${createHash("sha256").update(sql).digest("hex").slice(0, 16)}`
if (useCache) {
const cached = await c.get(cacheKey)
if (cached) {
return { ...JSON.parse(new TextDecoder().decode(cached.value)), cached: true }
}
}
// In production, use the connection config to connect to the actual database
await getConnectionConfig()
const result = simulateQuery(sql)
if (useCache) {
await c.set(cacheKey, JSON.stringify(result))
}
return { ...result, cached: false }
},
)
command("list-tables", async () => {
return { tables: Object.keys(SAMPLE_DATA) }
})test-connectionproves the secret is readable and returns only non-sensitive fields.queryhashes the SQL into a cache key, returns a cached result when asked, and otherwise runs the query. Every response says whether it came from the cache.list-tablesreturns the table names.
The template answers from in-memory sample data (users and orders) through simulateQuery, so it runs without a database. Replace simulateQuery with a real client such as pg or mysql2, built from the config in the vault.
Run it locally
alien init data-connector-ts private-database
cd private-database
alien devThe vault starts empty. In a second terminal, play the customer's admin and store a connection config as the database secret:
cat > db.json <<'EOF'
{"host": "db.customer.internal", "port": 5432, "database": "warehouse", "user": "readonly", "password": "local-only"}
EOF
alien dev vault set credentials database --file db.jsonLocally the vault is a file under .alien/. Now check the connection:
alien dev commands invoke --deployment local-dev/default --command test-connection{ "connected": true, "database": "warehouse", "host": "db.customer.internal" }The password isn't in the response. Run a query:
alien dev commands invoke --deployment local-dev/default \
--command query --params '{"sql": "SELECT * FROM users"}'{
"rows": [
{ "id": 1, "name": "Alice", "email": "alice@example.com", "role": "admin" },
{ "id": 2, "name": "Bob", "email": "bob@example.com", "role": "user" },
{ "id": 3, "name": "Charlie", "email": "charlie@example.com", "role": "user" }
],
"rowCount": 3,
"cached": false
}Run the same orders query twice with "useCache": true. The first response has "cached": false and fills the cache. The second has "cached": true and never touches the database:
alien dev commands invoke --deployment local-dev/default \
--command query --params '{"sql": "SELECT * FROM orders", "useCache": true}'list-tables returns { "tables": ["users", "orders"] }.
Test it
The test plays both roles. It deploys locally and writes the secret the way a customer would. Then it checks that:
test-connectionnever returnspassword.- A query returns rows.
- The second cached query reports
cached: true.
beforeAll(async () => {
deployment = await deploy({ app: ".", platform: "local" })
await deployment.setExternalSecret(
"credentials",
"database",
JSON.stringify({
host: "db.customer.internal",
port: 5432,
database: "warehouse",
user: "readonly",
password: "customer-secret-password",
}),
)
}, 300_000)pnpm testDeploy it next to the database
Release and onboard as usual:
alien release
alien onboard customer-1 --allow admin@customer-1.comalien onboard creates a setup link, a URL that opens the setup page Alien hosts for the customer's admin, who runs the initial setup once.
The connector has to run where it can reach the database. On AWS, a Worker runs in the deployment's private subnets when the deployment has a network.
The admin can point the deployment at their existing VPC during initial setup with --network byo. See Networking and endpoints.
Then call the connector from your backend:
import { CommandsClient } from "@alienplatform/commands"
const commands = await CommandsClient.forDeployment({
deploymentId: customer.deploymentId,
apiKey: process.env.ALIEN_API_KEY!,
})
const { rows } = await commands
.target("connector")
.invoke<{ rows: unknown[]; rowCount: number }>("query", {
sql: "SELECT * FROM orders",
useCache: true,
})How it works on each platform
| Platform | connector | credentials | cache |
|---|---|---|---|
| AWS | Lambda | SSM Parameter Store SecureString parameters | DynamoDB table |
| Google Cloud | Cloud Run | Secret Manager secrets | Firestore collection |
| Azure | Container Apps | Key Vault | Table Storage table |
alien dev | Native process | JSON file on disk | SQLite file |
Alien's management identity (the identity initial setup creates for ongoing management) can't read the vault by default. Only the connector's identity, with vault/data-read, can.
Before production
- Replace free-form SQL with named queries.
queryaccepts any SQL string, which hands your backend, and anything that can reach it, the full power of the database user. Expose operations such asorders-by-statuswith bound parameters instead, as Chat with private Postgres does. - Collect the secret during initial setup. For a value the admin types once, declare a deployer secret input. Initial setup then shows the admin where to write it, and the deployment waits until it exists.
- Bound the cache. Pass
{ ttl }toc.set()so stale results expire, and keep cached values under the 24 KiB KV value limit. - Use a read-only database user in the connection config.
Next
Kubernetes agent
A TypeScript agent in each customer's Kubernetes cluster that reports pods, deployments, and nodes to your API and answers commands, with read-only access, a generated enrollment token, and updates on every release.
Agent on employee devices
Run a Rust Daemon on each device that watches files, flags sensitive data, keeps events in an encrypted local database, and answers your backend only through commands.