Docs

Add to an existing product

Remote Operator, the outbound-only container your team uses to diagnose and fix customer environments, can join a product you already ship with your own Helm chart or Amazon ECS service. How you release doesn't change.

Each installation is one Remote Operator container in one customer environment. The dashboard generates the chart or template, and the install ends with one command:

helm install operator-<installation-id> ./remote-operator \
  --namespace checkout --kube-context customer-1-staging \
  --values operator-values.yaml --replace --atomic --wait --timeout 5m
  1. 1

    Open setupyou

    Set up Remote operator, then choose Existing Kubernetes or Amazon ECS Fargate.

  2. 2

    Review the templateyou

    Read the image, the RBAC or IAM it grants, and the log scope.

  3. 3

    Create an installationyou

    Generate one-time values for one environment.

  4. 4

    Installcustomer admin

    Run the generated Helm or AWS command in the target namespace or account.

  5. 5

    Verifyyou

    Run a read-only diagnostic through the connected installation.

When Alien already deploys your product, Remote Operator is built in instead. See Built in or added on its own.

Open setup

  1. Create a project with Add Alien to your Helm chart, or open an existing project.
  2. Select Set up Remote operator. After the first installation exists, the button reads Manage installations.
  3. The page opens with a prompt for a coding agent. To follow the steps yourself, select Set up manually instead.
  4. Under Installation platform, choose Existing Kubernetes or Amazon ECS Fargate.
Prompt
Add Alien Remote Operator to our Helm chart without changing how the application is released. Read https://alien.dev/docs/production/remote-operator/add-to-existing-product.md. Install it as a dedicated chart next to our application release, keep per-installation secrets out of git, install into a test namespace, and verify a read-only diagnostic succeeds. Ask me before touching a customer environment or a shared CRD.

Existing Kubernetes

You need Bash, jq, yq, OpenSSL, Helm 3.13 or later (or Helm 4), an existing namespace, a default StorageClass, and outbound HTTPS from the namespace.

Review the Remote operator template. Enter a Pod label key and value, for example app and checkout-api, and select Apply log scope. The Operator collects logs only from Pods with that label. Alien builds an image for your project, and the header shows Ready to package for your existing namespace below. when it's done. Read the ServiceAccount, Roles, and image.

Create a test installation. Enter an environment name such as customer-1-staging and select Generate test values. Save the two files the page shows once, outside version control: operator-credentials.yaml (a Secret) and operator-values.yaml. The registration token in the Secret expires at the time shown.

Install and verify. Under Owning Helm release, choose Dedicated operator release: a separate chart named remote-operator, installed next to your application's release. Enter the Kubernetes context and namespace, download each chart file into remote-operator/, confirm the review checkbox, and run the generated install command.

The install command:

  1. Checks that the namespace exists.
  2. Stops if this installation already completed once. A completed installation can't be installed again; register a new one.
  3. Creates the Secret, only if it's absent or identical to the file. Credentials never appear in Helm values or release history.
  4. Runs a server-side dry run, then the helm install shown at the top of this page.
  5. Returns when the Operator pod is ready. If it fails, Helm rolls the release back.

The chart includes a shared access-request CustomResourceDefinition. Helm installs it only if it's absent and never upgrades or deletes it. Have a cluster administrator review it, and don't replace or delete an existing one.

Use the per-file chart downloads. The single Download file in the review step also contains the CRD, and a release that owns the CRD deletes it on uninstall.

Product release

Use Product release only for a chart that Alien builds from your alien.ts. That chart already contains the Operator, disabled by default. Setup offers it only when the newest ready package embeds this installation's Operator image.

The chart refuses to enable the Operator on the first helm install, so setup gives two commands: install the product release with the Operator off, then enable it. See Upgrade a product release.

Per-installation values

In file and resource names, <installation-id> is the installation's ID with underscores replaced by hyphens.

  • operator-credentials.yaml is the Secret operator-<installation-id>-credentials with three keys: sync-token (the one-time registration token), encryption-key, and collector-token.
  • operator-values.yaml points the chart at that Secret and records the SHA-256 of the encryption key. The chart refuses a Secret whose encryption key doesn't match.

After the first install, the encryption key never changes. Only sync-token changes, through key rotation.

If the registration token expires before you install, open the same installation in setup and select Generate replacement values, then replace both files. See Before the first registration if an earlier attempt already created the Secret.

The setup page URL keeps the installation, namespace, context, and log scope, never secret values.

In a project that uses only Remote Operator, the project overview lists installations that haven't registered under Unfinished setups, with Resume setup. An entry shows values expired when its token has expired.

Kubernetes permissions

The Operator's Role lets it list Deployments, StatefulSets, DaemonSets, Pods, events, and Pod metrics in its namespace, and manage its access-request resources. Nothing more, unless an enabled operation declares it: it can't get or watch a Deployment, or read ConfigMaps, Services, or Jobs.

Each operation you enable adds only the rules it declares:

  • The only writes an operation can declare are delete on Pods and patch on the scale of a Deployment, StatefulSet, or ReplicaSet.
  • Alien rejects any operation rule on Secrets.
  • In the template, a comment above each rule names the operation that needs it.

Other Roles exist only when their feature is on:

  • Pod logs. get on pods/log, bound when the Operator collects logs through the Kubernetes API (logCollector.mode: podApi). A dedicated release uses this mode. In nodeAgent mode, a Fluent Bit DaemonSet with its own ServiceAccount reads node log files instead.
  • Dynamic containers. Only in a product chart Alien builds: get, list, create, update, and delete on Deployments, Services, and Secrets in the release namespace, for containers requested through the API. A dedicated release never has this Role.

When the enabled operations change, setup shows Permissions need an update. Its Kubernetes requirements entry lists the rules added and removed.

Check it after install:

kubectl auth can-i list pods --namespace checkout \
  --as system:serviceaccount:checkout:<operator-service-account>

The answer is yes. Repeat it for write actions: every yes should match an operation you enabled. For a dedicated release, get secrets and update deployments answer no.

Cloud permissions

Cloud operations need a workload identity for the Operator. When a selected operation needs one, a Connect cloud APIs step generates the Terraform or Pulumi configuration for it.

  • AWS. Enter Exact S3 bucket ARNs and Exact SQS queue ARNs when an enabled S3 or SQS operation reads a bucket, object, or queue. The policy limits those actions to the listed ARNs.
  • Google Cloud. Enter Cloud Storage bucket names for Cloud Storage operations. The Operator gets storage.objects.list and storage.buckets.get on those buckets only, and they must be in the Operator service account's project. Other Google Cloud operations, such as Compute Engine and Pub/Sub, are granted on the whole project.

If a required bucket or queue is missing, setup shows an error such as Google Cloud permission '<permission>' requires at least one installer-provided Cloud Storage bucket name instead of the configuration. Preview a plugin's grants offline with alien operations permissions.

Amazon ECS Fargate

The Operator runs as one Fargate service in an ECS cluster and network the customer already has. You need the AWS CLI, jq, OpenSSL, and curl, with credentials for the target account, plus subnets and security groups that allow outbound HTTPS and NFS.

Review the exact ECS target and CloudFormation. Enter the environment name, AWS account ID, Region, ECS cluster ARN, subnet IDs, and security group IDs. Select Generate CloudFormation, then Download JSON, and review it.

Register, deploy and verify. Select Create one-time registration and copy the token. Run Bootstrap and deploy from the folder with the JSON file and paste the token at the prompt.

The command stores the token in AWS Secrets Manager in the customer's account and runs aws cloudformation deploy. The template contains no secret values. The stack creates:

  • the ECS service (0.25 vCPU, 512 MiB)
  • a task role
  • a CloudWatch Logs group (30-day retention)
  • encrypted EFS storage for the Operator's identity

Before it writes anything, the command checks that your AWS credentials belong to the account you entered. It then sends the token with the exact account, Region, cluster, and stack name to Alien. Alien accepts the token only for the target recorded at setup and only while no Operator has registered from that stack; otherwise the command stops and changes nothing.

The first run creates the secret <stack>-registration with the token and a locally generated encryption key. Reruns keep that encryption key. The secret must use the default aws/secretsmanager KMS key.

Network and storage requirements:

  • Private subnets need outbound HTTPS to Alien through NAT or another public egress path. VPC endpoints don't reach Alien's endpoint.
  • When the stack creates EFS, choose at most one subnet per Availability Zone, and keep the same subnets in the same order on every update. The stack adds an inbound NFS rule to the first security group.
  • To use existing EFS instead, enter both an EFS filesystem ID and an access point ID. The access point must use POSIX UID and GID 1000, with a mount target in each selected Availability Zone whose security group allows TCP 2049 from the task's security group.

The EFS filesystem and access point are kept when the stack is deleted, so the Operator identity survives. The cluster, subnets, and security groups are never owned by the stack.

Verify the installation

When the Operator sends its first heartbeat, the setup status changes to Connected and asks you to verify a read-only diagnostic. Select Run read-only pod diagnostic (Kubernetes) or Run read-only CloudWatch diagnostic (ECS), or run one yourself:

alien operations invoke --deployment customer-1/staging --operation kubernetes/get-pods

The status changes to Test installation verified only after a read-only operation succeeds on that installation. A ready pod alone is not proof. Select Open installation to see it under Deployments.

Each environment needs its own values. Never copy a Secret, token, or encryption key to another cluster or customer.

Declare operations in Helm values

An Operator installed on its own declares its operations in the Helm value remoteOperator.operations. Settings written as { env: NAME } read from operationsEnv, so secrets can come from Kubernetes Secrets:

operator-values.yaml
remoteOperator:
  operations:
    plugins:
      kubernetes:
        approval: { "*": auto, restart-pod: manual, scale: manual }
      clickhouse:
        settings:
          url: https://clickhouse.checkout.svc:8443
          password: { env: CLICKHOUSE_PASSWORD }
        approval: { "*": auto }
  operationsEnv:
    - name: CLICKHOUSE_PASSWORD
      valueFrom:
        secretKeyRef: { name: clickhouse, key: password }

See Operations for the catalog and approval rules.

Versionless releases

When you add the Operator to a product you already ship, Alien didn't build the application, so it has no release to compare against. The Operator reports what it sees:

  • the Helm chart that installed the workloads
  • their images
  • the version from labels such as app.kubernetes.io/version, when all workloads agree on one

To record the version you shipped, create a versionless release. It records a version and git metadata without building or pushing anything:

alien release --no-stack --version 1.4.0

Run it from your existing release pipeline after you publish the chart.

Upgrade, rotate, and remove

Every later change starts from the same installation on the setup page: upgrade the image or permissions, rotate the connection key, or uninstall. Upgrades keep the installation's registration, encryption key, and identity storage.

When enabled operations need new permissions, setup shows Permissions need an update until you apply the new chart or template.

To remove it, uninstall the dedicated release or delete the stack with the command setup gives you, then retire the registration in Alien. See Upgrade, rotate, and remove for each procedure.

Next

On this page