Linux in Production
Administering — and then hardening — the Ubuntu server that a live application actually runs on. The unglamorous layer that decides whether the thing stays up.
- Host
- Ubuntu Server
- Serving
- Node.js API behind Nginx
- Focus
- Administration & hardening
- Status
- Ongoing
Why this is its own project
Plenty of developers can write an API. Fewer can explain why it stopped responding at 3am, why the log file is empty, or why it works locally and 404s behind the proxy. That gap is Linux administration, and it's the layer I learned before I learned the application layer.
Everything below is applied work — the server hosting MartialXPro in production, not a lab VM.
The host
- Web layer
- Nginx — reverse proxy, TLS termination, static content
- Process layer
- PM2 — supervision, restart on failure and on boot
- Runtime
- Node.js, configured entirely through environment variables
- Access
- SSH with key-based authentication
- Diagnostics
- PM2 process logs and Nginx access/error logs
Nginx as the front door
Nginx sits in front of the Node process rather than exposing the application port directly. That gives one place to terminate TLS, one place to decide what is proxied and what is served as a static file, and one place to control the headers the application sees.
Most of the configuration work was in the details: correct proxy_pass upstreams, forwarding the headers the application depends on, and keeping the split between API traffic and static content unambiguous. A reverse proxy that quietly rewrites part of a request produces bugs that look like application bugs.
Keeping the process alive
PM2 supervises the Node.js application — it restarts the process if it exits, brings it back after a reboot, and gives a consistent place to read runtime logs from. Combined with environment-variable configuration, that means a server restart requires no manual steps to get back to a serving state.
Hardening the box
The security work is not a separate phase bolted on at the end — it's the default configuration of the host. Coming from CEH and web application penetration testing, the baseline I apply is the one I'd otherwise be probing for gaps in:
- Key-based SSH onlyPassword authentication off the table; access is by key pair, which removes the entire brute-force surface.
- Least-privilege file permissionsApplication files, log directories and environment files owned and readable only by the accounts that genuinely need them — not blanket 777 to make an error go away.
- Minimal exposed surfaceOnly the ports that need to be reachable are reachable. Everything else stays closed at the firewall and security-group level.
- No secrets in the repositoryCredentials and environment-specific configuration live in environment variables on the host, never in version control.
- Separation of deploy and runtime accountsThe account that ships code and the account that runs it are distinct, so a compromise of one is not automatically a compromise of the other.
Operating it
Day to day, this means reading logs on a live system and reasoning from them. Permission and ownership problems between the deploy account and the process account, proxy configuration that doesn't match what the application expects, services that come back in the wrong order after a reboot — all of it gets diagnosed on the running host, because there is no second copy of production.
Next on this path is Docker for reproducible runtimes, Terraform so the infrastructure is described rather than clicked, and Kubernetes once there's more than one host worth orchestrating.