Overview
ServOS (Lightweight Agent Cloud Terminal) is designed with a “security-first” architecture. Since ServOS manages SSH keys and connects to your critical infrastructure, it employs industry-standard encryption and operational security practices to ensure your secrets remain safe.
Master Key (SERVOS_MASTER_KEY)
The cornerstone of ServOS’s security is the Master Key. This is a 32-byte random key used to encrypt all sensitive data stored in the database.
- Generation: The key is generated during the initial setup via
openssl rand -base64 32(or internal helper). - Storage: The key is passed to the ServOS API process via the
SERVOS_MASTER_KEYenvironment variable. It is never stored in the database. - Persistence: You (the admin) are responsible for backing up this key. If you lose the Master Key, all encrypted data (SSH keys, env vars) is irretrievably lost.
- Verification: On startup, ServOS verifies the key is valid Base64 and decodes to exactly 32 bytes.
- The “Bus Factor” Risk: If you lose this key, there is no recovery. The database is effectively a brick. Ensure this key is stored in a secure, shared password manager (e.g., 1Password, Bitwarden) accessible to authorized team members, so that access is not lost if a single administrator leaves.
Data Encryption
ServOS uses AES-256-GCM (Galois/Counter Mode) for authenticated encryption.
Algorithm Details
- Cipher: AES-256-GCM
- Library: Rust
aes-gcmcrate (audited and standard). - Nonce: A unique 12-byte random Nonce (Initialization Vector) is generated for every encryption operation.
- Format: The stored format in the database is:
Base64(Nonce[12] || Ciphertext).
Encrypted Fields
The following fields are always encrypted at rest:
- SSH Private Keys: Both Server keys (for connecting to your nodes) and Builder keys.
- SSH Passphrases: If your keys are password-protected.
- Environment Variables: All service environment variables (
env_vars). - Git Tokens: Personal Access Tokens for private repository cloning.
- Registry Passwords: Credentials for pulling images from private Docker registries.
Operational Security
Setup Token Location
During the initial installation, ServOS generates a Setup Token used to create the first admin account.
- Standard Install: The token is printed to the terminal logs.
- File Location: If you missed the logs, you can find the token on the server at:
/var/lib/servos/setup_token - Docker: If running via Docker, check the container logs (
docker logs servos-api) or the mounted volume.
Ephemeral Secrets Injection
ServOS avoids passing sensitive environment variables directly to processes where they might appear in ps aux or inspection tools.
How it works:
- When deploying a service, ServOS creates a temporary, secure file on the target server containing the localized environment variables.
- The file is created at
/var/lib/servos/secrets/{uuid}.envand permission-locked (readable only by root/owner). - Docker receives the variables via the
--env-fileargument with the temporary file mounted into the container. - The temporary file is immediately deleted after the container starts.
SSH & Trust On First Use (TOFU)
ServOS acts as an SSH client.
- Key Exchange: Uses standard Ed25519 or RSA keys.
- Host Verification: ServOS implements TOFU. When you first commission a server, ServOS records its SSH Host Fingerprint. Subsequent connections verify the fingerprint matches. If it changes (MITM attack or server rebuild), connection fails until you explicitly reset the fingerprint.
Webhooks
Incoming webhooks (e.g., from GitHub/GitLab) are verified using HMAC-SHA256 signatures to ensure authenticity.
Agent Security (Hardening)
The servos-agent follows a strict least-privilege model. Although it must start as root to initialize eBPF programs and handle networking, it immediately drops privileges.
- User Switch: Drops to the
servosuser and group. - Capability Bounding: Retains only the minimum required Linux capabilities:
CAP_NET_ADMIN: To manage WireGuard interfaces.CAP_BPF: To load and manage the mesh eBPF classifier.CAP_IPC_LOCK: To satisfy BPF map memory locking on kernels that require it.
This ensures that even if the agent is compromised, it cannot execute arbitrary root commands or access the host filesystem outside its scope.
Recommended Practices
- Secret Rotation: Rotate your
SERVOS_MASTER_KEYperiodically. Note: Key rotation requires a database migration script to re-encrypt data (currently manual). - Database Security: While data is encrypted, access to the database allows attackers to corrupt or delete data. Ensure your PostgreSQL instance is firewalled.
- HTTPS: Always run ServOS behind a reverse proxy (like built-in Traefik or Cloudflare) with HTTPS enabled.