Run your app with systemd, not screen or nohup: units, restarts, timers

If an app on your server still lives in screen or nohup ... &, it will not come back after a reboot, will not restart when it crashes, and writes to a log file nobody rotates. systemd is already on every Ubuntu machine. A unit file under 20 lines fixes all three, narrows what the app is allowed to touch, and gives you timers to replace cron.

Tested on Ubuntu 24.04 with systemd 255 (package 255.4-1ubuntu8). Every default quoted here was checked against the systemd 255 man pages.

screen, nohup, pm2 and systemd compared

	screen / nohup	pm2	systemd
Starts at boot	no	via pm2 startup (writes a systemd unit)	yes (enable)
Restarts on crash	no	yes	yes (Restart=)
Runs as its own user	whoever typed it	whoever runs pm2	User=
Logs	file/terminal, never rotated	pm2’s own files	journal, size-capped, searchable by time
Restricts system files	no	no	ProtectSystem=, ReadWritePaths=
Extra install	no	Node + pm2	no, ships with the OS

pm2 still has its place, Node cluster mode for one. For a single-process app, systemd does it all with no middle layer.

Your first unit file

Say the app lives in /srv/myapp and runs with node server.js. First create a system user that cannot log in:

# on the server
sudo useradd --system --home /srv/myapp --shell /usr/sbin/nologin myapp
sudo chown -R myapp:myapp /srv/myapp

Then write /etc/systemd/system/myapp.service:

[Unit]
Description=My app (HTTP API)
After=network-online.target
Wants=network-online.target
StartLimitIntervalSec=300
StartLimitBurst=5

[Service]
User=myapp
Group=myapp
WorkingDirectory=/srv/myapp
EnvironmentFile=/etc/myapp/env
ExecStart=/usr/bin/node /srv/myapp/server.js
Restart=on-failure
RestartSec=5

# hardening
NoNewPrivileges=true
ProtectSystem=strict
ProtectHome=true
PrivateTmp=true
ReadWritePaths=/srv/myapp/data

[Install]
WantedBy=multi-user.target

sudo systemctl daemon-reload
sudo systemctl enable --now myapp
systemctl status myapp

enable hooks the unit into multi-user.target so it starts at boot; --now starts it immediately. With no Type= and an ExecStart=, systemd uses simple: the process it launches is the service. Do not let the app fork itself into the background (no &, no daemon mode).

EnvironmentFile: keep secrets out of the unit

Anyone on the machine can read a unit file (systemctl cat prints it). Do not put passwords in Environment=; move them to a separate file:

sudo mkdir -p /etc/myapp
sudo tee /etc/myapp/env >/dev/null <<'EOF'
# lines starting with # are ignored
PORT=3000
DATABASE_URL=postgres://myapp:change-me@127.0.0.1/myapp
EOF
sudo chmod 600 /etc/myapp/env

One KEY=value per line. systemd, running as root, reads the file before starting the process and passes the values in as environment, so the app itself never needs read access to it. Prefix the path with - (EnvironmentFile=-/etc/myapp/env) if a missing file should not be an error. For keeping each system’s credentials apart, see the separate-credentials article.

Restarts: why RestartSec and StartLimit go together

Restart=on-failure restarts when the process exits non-zero, is killed by a signal (other than SIGHUP, SIGINT, SIGTERM or SIGPIPE), times out, or trips the watchdog. A clean exit 0 is not restarted.

The trap is two defaults that work as a pair:

• RestartSec= defaults to 100 ms
• the rate limit allows at most StartLimitBurst= 5 starts within StartLimitIntervalSec= 10 seconds (both set in [Unit]). Past that, systemd stops restarting and the service stays failed

Worked example: the app dies immediately on start because the database is not up yet.

1. Defaults: starts at 0, 0.1, 0.2, 0.3, 0.4 s. Five starts in 0.4 s hits the limit and the service is failed for good, even though the database would have been ready 3 seconds later
2. RestartSec=5 alone: starts at 0, 5, 10 s. Any 10-second window holds at most 3 starts, never 5, so it never hits the limit and loops every 5 seconds forever. If the cause is a bad config, your journal fills with the same error
3. RestartSec=5 + StartLimitIntervalSec=300 + StartLimitBurst=5 (the file above): starts at 0, 5, 10, 15, 20 s; the fifth start inside 300 s makes it give up at about 20 s. If the database was merely slow, the second or third attempt succeeds

RestartSec decides how long to wait; StartLimit decides when to give up. Always set both.

Once it hits the limit, fix the cause, then sudo systemctl reset-failed myapp before starting again. To hear about a real outage, wire up an alert as in the Telegram monitoring article.

Basic hardening

directive	what it does
NoNewPrivileges=true	the process and its children cannot gain privileges via setuid/setgid or file capabilities
ProtectSystem=strict	the whole filesystem is read-only except /dev, /proc, /sys
ReadWritePaths=	the folders that stay writable under strict
ProtectHome=true	/home, /root, /run/user appear empty and inaccessible
PrivateTmp=true	its own /tmp and /var/tmp, removed when the service stops

(For comparison: ProtectSystem=true locks only /usr and the boot directories, full adds /etc, strict locks everything.)

If the app is compromised, the attacker is user myapp and can write only to /srv/myapp/data; system files and the app’s own code are off-limits. Check the score with:

systemd-analyze security myapp

It walks every directive and gives an exposure score, lower is better. Treat it as a checklist of what else you could close, one at a time, testing the app after each; don’t chase zero. For data folders there is also StateDirectory=myapp: systemd creates /var/lib/myapp, owned by User=, and writable even under strict.

Timers instead of cron

Scheduled jobs (backups, cache cleanup) are a service + timer pair with the same name.

/etc/systemd/system/myapp-backup.service:

[Unit]
Description=Nightly backup for myapp

[Service]
Type=oneshot
User=myapp
ExecStart=/srv/myapp/bin/backup.sh

/etc/systemd/system/myapp-backup.timer:

[Unit]
Description=Run myapp backup every night

[Timer]
OnCalendar=*-*-* 03:00:00
Persistent=true
RandomizedDelaySec=15min

[Install]
WantedBy=timers.target

sudo systemctl daemon-reload
sudo systemctl enable --now myapp-backup.timer
systemctl list-timers myapp-backup.timer
systemd-analyze calendar "*-*-* 03:00:00"   # see what the expression means

The timer finds the service with the same name by default (Unit=). What you get over cron:

• Persistent=true (default false) stores the last run time on disk. If the machine is off at 03:00 and boots at 07:00, the job runs right after boot. cron simply skips it
• RandomizedDelaySec=15min (default 0) adds a random 0-15 minute delay, so many machines with the same schedule don’t hit your storage at the same moment
• Job output goes to the journal, no >> /var/log/x.log 2>&1, and systemctl status myapp-backup tells you whether the last run succeeded

Watch AccuracySec=, which defaults to 1 minute: systemd may shift the wake-up by up to a minute to batch timers. Set AccuracySec=1s if the job must fire on the second.

Reading logs with journalctl

A service’s stdout and stderr go to the journal by default; no log files to manage yourself.

journalctl -u myapp -f                 # follow live
journalctl -u myapp -b                 # since this boot
journalctl -u myapp --since "1 hour ago"
journalctl -u myapp -p err             # errors and worse only
journalctl -u myapp -n 50 --no-pager   # last 50 lines

The journal has its own size cap; for keeping logs from filling the disk see the disk-full article.

Common mistakes

• Editing a unit and forgetting daemon-reload. systemd keeps the old version; systemctl status warns that the unit file changed on disk
• ~, $(...) or pipes in ExecStart=. The command does not go through a shell, so none of these work. Use full paths; if you need a pipe or &&, call /bin/sh -c '...' or move it into a script
• export in the EnvironmentFile. It is not a shell script; lines must be plain KEY=value
• The app breaks under strict with read-only errors. Look in journalctl -u myapp for EROFS or Read-only file system and add only that path to ReadWritePaths=
• StartLimit settings in [Service]. In current systemd they belong in [Unit]
• A timer that was never enabled. Creating the file does nothing on its own; check with systemctl list-timers --all

If you deploy by swapping folders as in the atomic deploy article, point WorkingDirectory= and ExecStart= at the fixed path that gets swapped (not a versioned folder) and systemctl restart myapp after the swap.

Sources

• systemd.service (systemd 255) — freedesktop.org
• systemd.exec (systemd 255) — freedesktop.org
• systemd.timer (systemd 255) — freedesktop.org
• systemd.unit (systemd 255) — freedesktop.org and systemd-system.conf
• journalctl (systemd 255) — freedesktop.org
• systemd package in Ubuntu Noble — Launchpad