Linux systemd: Create and Manage Custom Services

What Is systemd and Why It Matters

systemd is the init system and service manager on most modern Linux distributions, including Ubuntu, Debian, Fedora, CentOS, and Arch Linux. It replaces the older SysV init scripts with a unified, dependency-based approach to managing processes.

When your Linux machine boots, systemd is the first process that starts (PID 1). It launches and supervises all other services in the correct order based on their declared dependencies. As a developer or system administrator, you will frequently need to create custom services for your applications, background workers, or monitoring scripts.

Understanding Unit Files

systemd manages everything through unit files. A unit file is a plain-text configuration file that describes a resource systemd should manage. The most common unit types are:

• service - A process or daemon
• timer - A scheduled task (replaces cron)
• socket - A network or IPC socket
• target - A group of units (similar to runlevels)
• mount - A filesystem mount point

Unit files live in specific directories:

Directory	Purpose
/etc/systemd/system/	Custom and overridden units (highest priority)
/run/systemd/system/	Runtime units (generated dynamically)
/usr/lib/systemd/system/	Units installed by packages

Always place your custom service files in /etc/systemd/system/.

Anatomy of a Service Unit File

A service unit file has three main sections:

[Unit]
Description=My Application Server
Documentation=https://example.com/docs
After=network.target postgresql.service
Wants=postgresql.service

[Service]
Type=simple
User=appuser
Group=appuser
WorkingDirectory=/opt/myapp
Environment=NODE_ENV=production
EnvironmentFile=/opt/myapp/.env
ExecStartPre=/opt/myapp/pre-start.sh
ExecStart=/usr/bin/node /opt/myapp/server.js
ExecReload=/bin/kill -HUP $MAINPID
Restart=on-failure
RestartSec=5
StandardOutput=journal
StandardError=journal

[Install]
WantedBy=multi-user.target

The Unit Section

The [Unit] section provides metadata and dependency information:

• Description - A human-readable name for the service.
• After - This service starts after the listed units. This is ordering only, not a requirement.
• Wants - Soft dependency. systemd tries to start these units but does not fail if they are unavailable.
• Requires - Hard dependency. If the required unit fails, this service also stops.

The Service Section

The [Service] section defines how the process runs:

• Type - How systemd determines the service is “started”:

  • simple - The process specified in ExecStart is the main process (default).
  • forking - The process forks and the parent exits. Use PIDFile= to track the child.
  • oneshot - The process exits after completing its task. Useful for setup scripts.
  • notify - The process sends a notification to systemd when ready.

• ExecStart - The command to start the service. Must use absolute paths.

• ExecReload - The command to reload the service configuration without restarting.

• Restart - When to restart: no, on-success, on-failure, on-abnormal, on-abort, always.

• RestartSec - How long to wait before restarting.

• User/Group - Run the service as this user and group instead of root.

The Install Section

The [Install] section controls how systemctl enable links the service:

• WantedBy=multi-user.target means the service starts when the system reaches the multi-user state (equivalent to the old runlevel 3). This is the correct target for most server applications.

Creating Your First Custom Service

Let’s create a service for a Python web application. First, write a simple Flask app:

# /opt/webapp/app.py
from flask import Flask

app = Flask(__name__)

@app.route('/')
def index():
    return 'Hello from systemd-managed Flask!'

if __name__ == '__main__':
    app.run(host='0.0.0.0', port=5000)

Now create the service file:

sudo nano /etc/systemd/system/webapp.service

[Unit]
Description=Flask Web Application
After=network.target

[Service]
Type=simple
User=www-data
Group=www-data
WorkingDirectory=/opt/webapp
Environment=FLASK_ENV=production
ExecStart=/opt/webapp/venv/bin/python /opt/webapp/app.py
Restart=always
RestartSec=3

[Install]
WantedBy=multi-user.target

Activating the Service

After creating or modifying a unit file, reload the systemd daemon:

# Reload unit file definitions
sudo systemctl daemon-reload

# Start the service
sudo systemctl start webapp

# Enable it to start on boot
sudo systemctl enable webapp

# Check the status
sudo systemctl status webapp

The status command shows whether the service is running, its PID, memory usage, and recent log lines:

● webapp.service - Flask Web Application
     Loaded: loaded (/etc/systemd/system/webapp.service; enabled)
     Active: active (running) since Mon 2026-07-07 10:15:00 UTC; 5s ago
   Main PID: 12345 (python)
      Tasks: 1 (limit: 4915)
     Memory: 28.4M
        CPU: 0.245s
     CGroup: /system.slice/webapp.service
             └─12345 /opt/webapp/venv/bin/python /opt/webapp/app.py

Essential systemctl Commands

# Start, stop, restart
sudo systemctl start webapp
sudo systemctl stop webapp
sudo systemctl restart webapp

# Reload configuration without restart
sudo systemctl reload webapp

# Enable/disable on boot
sudo systemctl enable webapp
sudo systemctl disable webapp

# Check if active or enabled
systemctl is-active webapp
systemctl is-enabled webapp

# List all running services
systemctl list-units --type=service --state=running

# List all failed services
systemctl list-units --type=service --state=failed

# Show the unit file content
systemctl cat webapp

Reading Logs with journalctl

systemd captures stdout and stderr from services into the journal. Use journalctl to read them:

# All logs for a service
journalctl -u webapp

# Follow logs in real time
journalctl -u webapp -f

# Logs since last boot
journalctl -u webapp -b

# Logs from the last hour
journalctl -u webapp --since "1 hour ago"

# Last 50 lines
journalctl -u webapp -n 50

# Only error-level and above
journalctl -u webapp -p err

Hardening Your Service

systemd provides sandboxing features to limit what a service can do. This is especially important for services exposed to the network.

[Service]
# Filesystem restrictions
ProtectSystem=strict
ProtectHome=true
ReadWritePaths=/opt/webapp/data
PrivateTmp=true

# Network restrictions
PrivateNetwork=false
RestrictAddressFamilies=AF_INET AF_INET6

# Capability restrictions
NoNewPrivileges=true
CapabilityBoundingSet=

# System call filtering
SystemCallArchitectures=native
SystemCallFilter=@system-service

• ProtectSystem=strict - Mounts /usr, /boot, and /etc as read-only.
• ProtectHome=true - Makes /home, /root, and /run/user inaccessible.
• PrivateTmp=true - Gives the service its own /tmp directory.
• NoNewPrivileges=true - Prevents the process from gaining additional privileges.

Check the security score of your service:

systemd-analyze security webapp

This command gives a score from 0 (most secure) to 10 (least secure) and lists specific recommendations.

Resource Limits

Prevent a service from consuming too many resources:

[Service]
# Memory limit
MemoryMax=512M
MemoryHigh=400M

# CPU limit (100% = 1 core)
CPUQuota=50%

# Limit number of processes
TasksMax=100

# Limit open files
LimitNOFILE=65536

# Kill the entire cgroup on stop
KillMode=control-group
TimeoutStopSec=30

systemd Timers: Replacing Cron

systemd timers are a modern replacement for cron jobs. They consist of two files: a .service file (what to run) and a .timer file (when to run it).

Creating a Timer

First, create the service that does the work:

# /etc/systemd/system/backup.service
[Unit]
Description=Daily Database Backup

[Service]
Type=oneshot
User=backup
ExecStart=/opt/scripts/backup-database.sh

Then create the timer:

# /etc/systemd/system/backup.timer
[Unit]
Description=Run database backup daily

[Timer]
OnCalendar=*-*-* 02:00:00
Persistent=true
RandomizedDelaySec=300

[Install]
WantedBy=timers.target

• OnCalendar - When to run. Format is DayOfWeek Year-Month-Day Hour:Minute:Second.
• Persistent=true - If the system was off when the timer should have fired, run it immediately on next boot.
• RandomizedDelaySec - Add random delay up to this value to avoid thundering herd.

sudo systemctl daemon-reload
sudo systemctl enable --now backup.timer

# Check timer status
systemctl list-timers

Common Timer Schedules

# Every 15 minutes
OnCalendar=*:0/15

# Every hour on the half hour
OnCalendar=*:30:00

# Monday through Friday at 9 AM
OnCalendar=Mon..Fri *-*-* 09:00:00

# First day of every month
OnCalendar=*-*-01 00:00:00

# Relative timing (5 minutes after boot, then every hour)
OnBootSec=5min
OnUnitActiveSec=1h

Overriding Package-Installed Services

Never edit files in /usr/lib/systemd/system/ directly. Instead, create an override:

sudo systemctl edit nginx

This opens an editor for /etc/systemd/system/nginx.service.d/override.conf. Add only the settings you want to change:

[Service]
LimitNOFILE=65536
Environment=WORKER_CONNECTIONS=4096

To see the combined effective configuration:

systemctl cat nginx

To reset overrides:

sudo systemctl revert nginx

Troubleshooting Failed Services

When a service fails to start, follow these steps:

# Check status and error message
sudo systemctl status webapp

# Read detailed logs
journalctl -u webapp -n 100 --no-pager

# Verify the unit file syntax
systemd-analyze verify /etc/systemd/system/webapp.service

# Check for dependency issues
systemctl list-dependencies webapp

# Analyze boot time to find slow services
systemd-analyze blame
systemd-analyze critical-chain webapp.service

Common Problems and Fixes

Permission denied on ExecStart: Make sure the script is executable and the User has permission to run it.

chmod +x /opt/myapp/start.sh
chown appuser:appuser /opt/myapp/start.sh

Service starts then immediately exits: Check if Type= is correct. If your application forks into the background, use Type=forking. If it stays in the foreground, use Type=simple.

Environment variables not loading: Use EnvironmentFile= with an absolute path. The file format is KEY=value, one per line, no export keyword.

# /opt/myapp/.env
DATABASE_URL=postgres://localhost/mydb
SECRET_KEY=abc123
PORT=5000

Service restarts in a loop: Check RestartSec and add StartLimitIntervalSec and StartLimitBurst to prevent infinite restart loops:

[Service]
Restart=on-failure
RestartSec=5
StartLimitIntervalSec=60
StartLimitBurst=3

This allows 3 restarts within 60 seconds. After that, the service enters a failed state and stops retrying.

Wrapping Up

systemd is the backbone of service management on modern Linux. You create unit files in /etc/systemd/system/, define the process and its dependencies in the [Unit] and [Service] sections, and use systemctl to start, stop, enable, and monitor services. Timers provide a structured alternative to cron, and built-in sandboxing options let you harden services with minimal effort. When something goes wrong, journalctl and systemd-analyze verify are your primary diagnostic tools.