Docs

Releases and operations

After a machine joins, your team runs it from Alien: releases roll out on their own, and you watch health and logs without logging in to the machine. This page covers that ongoing management for bare metal and on-prem machines.

alien release
alien deployments machines customer-1/on-prem

Ship a new version

Each alien release moves the production channel, and every deployment that follows it updates. The machine agent polls Alien every 5 seconds over outbound HTTPS and applies the new release.

  • Channels. Point a deployment at staging with alien deployments set-channel <deployment-id> staging, then release there first with alien release --channel staging.
  • Pinning. alien deployments pin <deployment-id> <release-id> keeps one deployment on a release while the channel moves on.
  • Rollback. Promote the earlier release again with alien releases promote <release-id> --channel production. A machine whose new version never became healthy gets the earlier release at once: its process has at most 30 seconds to stop. Machines still on the earlier release keep running as they are.

A release must include the machines platform. See Releases.

How a new version replaces the old one

A release replaces your workloads one at a time:

  1. For a daemon with networkMode: "host", the machine pulls the new image before it stops the old process. A failed pull keeps the old one serving.
  2. It sends the old process SIGTERM and waits up to stopGracePeriod for it to exit. The default is 30 seconds, the maximum 24 hours.
  3. It starts the new version and waits until it passes its health check, or until it runs when the daemon has none. The wait is bounded by the startup timeout: 10 minutes by default, counted from when the new process starts running. Set startupTimeoutSeconds on .healthCheck() to change it (up to 24 hours).
  4. Then it moves to the next machine.

If the new version never becomes healthy, the update fails with an error that names the machine and the timeout. The rollout stops on that machine, and the machines after it keep running the previous release.

The machine that updated no longer runs the previous release. To get it serving again, promote the previous release or ship a fix. You can also retry or delete the failed deployment.

Write the daemon to stop taking new work on SIGTERM, then finish or hand off what's running and exit. The grace period is a ceiling, not a wait: a process that exits early lets the rollout continue at once.

Containers roll the same way: one replica at a time, and the next only after the replacement is healthy.

How the machine agent updates itself

The machine agent updates itself, separately from your releases:

  • Each new version is checked against its SHA-256 and started once before it replaces the old one.
  • Machines update one at a time. Each must stay healthy for 5 minutes before the next one starts.
  • A machine agent that fails to start three times reverts to the previous version.
  • Your workloads and public endpoints keep running during the update.

Manage machines marks a machine agent that's behind as outdated.

Check machine health

alien deployments machines customer-1/on-prem

alien deployments machines lists each machine with its status, last heartbeat, network, mesh peers, and machine agent version. Add --json for the full inventory. In the dashboard, open the deployment, then Manage machines.

StatusMeaning
ReadyConnected and running its workloads
UnhealthyConnected, but the machine reports a problem
OfflineNo heartbeat for 5 minutes
DrainingMoving its work off before it leaves
DrainedEmpty and ready to remove

The deployment itself shows Waiting for machines until the first machine joins, then Running.

Read logs and run commands

alien logs --deployment customer-1/on-prem --follow
alien logs --deployment customer-1/on-prem --system --level error --since 2h

The first command streams what your daemons and containers print. --system adds the platform's own logs, including the machine agent. See Logs and metrics.

To run a command on a machine:

alien debug exec customer-1/on-prem --machine <machine-id> -- journalctl -u alien-machine --since "10 min ago"

--machine takes an ID from alien deployments machines. See Debug a deployment.

When a machine goes offline

A machine that can't reach Alien keeps running the last release it accepted:

  • Running daemons and containers continue.
  • A process that exits is restarted on the machine.
  • After a reboot, the machine starts its last release again.

Releases and commands wait until it reconnects, then it catches up. The accepted release is bound to the machine that received it, so a cloned disk refuses to start it. Replacing the machine's primary network adapter has the same effect. In both cases, run the join command again.

Recover a machine

A machine renews its credentials on its own while it's online. One that stays offline for more than about 8 days can't renew them when it comes back. Recover it instead of removing and rejoining it:

  1. In Manage machines, choose Recover on the machine, then Create recovery command. Only workspace admins can.
  2. Run the command on the machine within 1 hour. It works once.

The machine comes back with the same machine ID and workloads. A machine that is online, or still has valid credentials, needs no recovery, and Recover says so. From the API, POST /v1/machines/deployments/{id}/machines/{machineId}/recovery-command does the same with a workspace or project admin API key.

Drain a machine

Drain moves work off a machine before maintenance or removal. The machine stops taking new work, and each replica stops within its stopGracePeriod.

alien deployments drain-machine customer-1/on-prem <machine-id> --deadline 6h
alien deployments cancel-machine-drain customer-1/on-prem <machine-id>

In Manage machines, open the machine's menu and choose Drain machine. The dialog sets Deadline, hours, 6 by default; past it, the drain shows as overdue. Force drain, or drain-machine --force, cuts each grace period to 5 seconds.

Remove a machine

From the machine itself, leave drains it, waits for its work to stop, and uninstalls the machine agent:

sudo acmectl leave --purge

--purge also deletes the machine's identity and state. Without it, the identity stays on disk.

For a machine you can't reach, remove it from Alien: Remove machine in Manage machines (Force remove if it isn't drained), or:

alien deployments remove-machine customer-1/on-prem <machine-id>

A removed machine can't rejoin with its old identity. Run leave --purge on it, then the join command again.

Manage join tokens

A join token works for up to 100 machines for 7 days, or until you revoke it. Treat a join command as a secret, and revoke its token when the machines you meant to add have joined.

alien deployments join-tokens list customer-1/on-prem
alien deployments join-tokens create customer-1/on-prem
alien deployments join-tokens revoke customer-1/on-prem <token-id>
alien deployments join-tokens rotate customer-1/on-prem
  • create prints a new join command. Existing tokens keep working.
  • revoke stops one token. Machines that already joined keep running.
  • rotate revokes every token and prints a command with a new one. Running alien deploy --machines again rotates too.

In the dashboard, Manage join tokens lists them, and Rotate all join tokens replaces them. A token is shown only once.

Delete the deployment

alien deployments delete customer-1/on-prem

Then run sudo acmectl leave --purge on each machine to remove the machine agent.

Next

On this page