SSH Keys and Secure Server Access

SSH is the single most important tool for working with remote servers, and passwords are the wrong way to use it. Public-key authentication is faster to use, harder to brute force, and far easier to rotate. This guide walks through the modern setup: a strong key, a tidy client config, a hardened server, and a couple of patterns for reaching machines behind a bastion.

Generate a strong key

Skip the old defaults. ed25519 keys are short, fast, and considered the right modern choice. RSA is fine if you stick to 4096 bits, but there is no reason to start there in 2026.

ssh-keygen -t ed25519 -C "you@example.com" -f ~/.ssh/id_ed25519

You will be prompted for a passphrase. Use one. A leaked private key without a passphrase is an immediate compromise; with a passphrase the attacker still has to crack it. Pair the passphrase with ssh-agent so you only type it once per session.

The command creates two files: id_ed25519 (private, mode 600) and id_ed25519.pub (public, safe to share).

Install your public key on the server

The standard tool is ssh-copy-id, which appends your public key to ~/.ssh/authorized_keys on the remote machine with the right permissions.

ssh-copy-id -i ~/.ssh/id_ed25519.pub user@server.example.com

If that command is not available, the manual equivalent is:

cat ~/.ssh/id_ed25519.pub | ssh user@server.example.com \
  "mkdir -p ~/.ssh && chmod 700 ~/.ssh && \
   cat >> ~/.ssh/authorized_keys && chmod 600 ~/.ssh/authorized_keys"

Wrong permissions on ~/.ssh or authorized_keys will cause sshd to silently ignore the file. If key auth mysteriously fails, check modes first.

Use ~/.ssh/config to stay sane

Once you have more than two servers, typing usernames, ports, and key paths gets old. The client-side ~/.ssh/config file solves it.

Host prod-api
    HostName api.prod.example.com
    User deploy
    IdentityFile ~/.ssh/id_ed25519
    Port 22
    ServerAliveInterval 30

Host staging-*
    User ubuntu
    IdentityFile ~/.ssh/id_ed25519_staging
    StrictHostKeyChecking yes

Host bastion
    HostName bastion.example.com
    User jump
    IdentityFile ~/.ssh/id_ed25519

Host *.internal
    ProxyJump bastion
    User deploy

Now ssh prod-api does the right thing. The last block is especially nice — anything matching *.internal automatically tunnels through the bastion.

Hardening sshd on the server

A reasonable baseline for /etc/ssh/sshd_config on a public server:

PermitRootLogin no
PasswordAuthentication no
ChallengeResponseAuthentication no
PubkeyAuthentication yes
AllowUsers deploy
LoginGraceTime 30
MaxAuthTries 3
ClientAliveInterval 300
ClientAliveCountMax 2

After editing, validate and reload:

sudo sshd -t
sudo systemctl reload ssh

Keep an existing SSH session open while you do this. If you locked yourself out, you can fix the config from the still-open shell. Only after a fresh connection succeeds should you close the original.

Changing the listening port is sometimes suggested. It reduces log noise from drive-by scanners but is not real security. Disabling password auth and root login matter far more.

ssh-agent and forwarding

ssh-agent holds your decrypted private keys in memory so you only enter the passphrase once.

eval "$(ssh-agent -s)"
ssh-add ~/.ssh/id_ed25519

Agent forwarding (-A or ForwardAgent yes) lets a remote server use your local agent to authenticate further hops. It is convenient and slightly dangerous — anyone with root on the intermediate host can use your agent while you are connected. Prefer ProxyJump for bastion hops, which keeps the private key on your local machine and never exposes the agent to the intermediate host.

ssh -J bastion.example.com deploy@db.internal

That command opens a connection to bastion and then tunnels a fresh SSH session to db.internal over it. The bastion never sees your key material.

Rotating and revoking keys

Treat keys like passwords: rotate them on a schedule, and revoke them immediately when a laptop is lost or a contractor leaves. Revocation is just removing a line from each server’s authorized_keys. A configuration management tool like Ansible, or a small script that templates the file from a central source of truth, makes this painless.

For teams, a better answer is SSH certificates. An internal CA signs short-lived certificates tied to a user, and servers trust the CA instead of individual keys. Revocation becomes “stop signing new certs for that user,” and yesterday’s certs naturally expire. That setup is heavier than this guide, but worth knowing exists once a fleet outgrows static authorized_keys management.

A small checklist

Use an ed25519 key with a passphrase. Install it with ssh-copy-id. Keep a tidy ~/.ssh/config. Disable root login and password auth on every server. Prefer ProxyJump over agent forwarding. Rotate and revoke as a routine, not as an emergency. Do those six things and SSH stops being the soft part of your infrastructure.