Troubleshooting

Common errors and how to resolve them

Troubleshooting

This guide covers common issues you might encounter while using ServOS.

Deployment Errors

“Host key verification failed”

Symptom: Deployment fails during the “Booting” or “Pulling” stage with an SSH error.

Cause: The server’s SSH host key has changed (e.g., you reinstalled the OS) and ServOS’s stored fingerprint does not match.

Resolution:

  1. Go to the Servers tab.
  2. Select the problematic server.
  3. Click “Edit”.
  4. Depending on availability, click “Reset Fingerprint” or simply delete and re-add the server if it’s a fresh install.

“Traefik 404 Not Found”

Symptom: You can access the server IP, but your domain returns a “404 page not found” from Traefik.

Cause:

  1. The domain is not correctly pointing to the server IP.
  2. The Service does not have the “Domain” configured.
  3. The Docker container is not healthy.

Resolution:

  1. Verify DNS: Ensure ping yourdomain.com resolves to your server IP.
  2. Check Service: In the Service detail view, ensure the domain is listed in “Networking”.
  3. Check Labels: If you manually imported, ensure traefik.http.routers.svc.rule includes Host('yourdomain.com').

Metrics & Observability

“Metrics not appearing in dashboard”

Symptom: Graphs are empty or show “No Data” for a commissioned server.

Cause:

  1. servos-agent service is not running.
  2. The server cannot reach the Control Plane API via HTTP.
  3. Insufficient Linux capabilities for the agent.

Resolution:

  1. Check Agent Status: SSH into the server and run systemctl status servos-agent.
  2. Check Logs: Run journalctl -u servos-agent to check for “Failed to drop privileges” or “Connection refused” errors.
  3. Verify Connectivity: Ensure the server can curl your SERVOS_API_URL.
  4. Clock Sync: Ensure the server time is synchronized (ntp/chrony). Mismatched timestamps will cause metrics to be rejected.

Connectivity Issues

“Database connection failed”

Symptom: Application logs show “Connection refused” or “Cannot connect to database”.

Cause:

  1. The application is trying to connect to localhost. In Docker, localhost refers to the container itself, not the host or other containers.
  2. Networking isolation.

Resolution:

  1. Use the Service Name: If valid, distinct services in the same environment can talk to each other.
  2. Use Internal IP: Use the server’s internal docker IP (usually 172.17.0.1 for the host, but depends on network mode).
  3. Environment Variables: Ensure DATABASE_URL is correct.

Import Errors

“Ignored unsupported key”

Symptom: During Docker Compose import, you see warnings about networks or depends_on.

Cause: ServOS manages its own networking stack and lifecycle.

Resolution:

  • Depends On: ServOS does not enforce strict startup order. Ensure your application handles retry logic (e.g. wait-for-it script) if the database isn’t ready.

Zombie Containers

“Deleted containers are still running” or “Safety Mode Alert”

Symptom: You deleted a service or deployment, but the container is still running on the server. You might see a “Safety Mode” alert in the dashboard.

Cause: ServOS’s “Zombie Reconciliation” process (which usually deletes unknown containers) automatically disables itself if it detects that >50% of containers on a server would be deleted. This prevents accidental data loss during database restores or misconfigurations.

Resolution: See the Safety Mode Documentation for details on how to resolve the alert and clean up the containers.

Slow Container Shutdown

“Container takes 10+ seconds to stop”

Symptom: During deployments, the old container takes exactly 10 seconds to stop instead of shutting down gracefully.

Cause: Your application is not receiving the SIGTERM signal. This happens when:

  1. Node.js/npm runs as PID 1 and doesn’t forward signals to child processes
  2. The application doesn’t have a signal handler
  3. Shell scripts wrap the main process without exec

Resolution:

Option 1: Use Docker’s init flag (Recommended for ServOS)

Add this to your service’s Docker Config in ServOS:

{
  "HostConfig": {
    "Init": true
  }
}

This injects tini as PID 1, which properly forwards signals to your app.

Option 2: Add tini to your Dockerfile

RUN apk add --no-cache tini
ENTRYPOINT ["/sbin/tini", "--"]
CMD ["node", "server.js"]

Option 3: Handle signals in your application

// Node.js example
process.on('SIGTERM', () => {
  console.log('Received SIGTERM, shutting down gracefully');
  server.close(() => process.exit(0));
});