SSH and SSH keys, without the fear

Most homelab machines end up somewhere inconvenient: a closet, a basement shelf, behind a TV. You don’t want to plug in a keyboard every time you change a setting. SSH (Secure Shell) gives you that machine’s terminal from your own computer, encrypted, over the network.

The first connection

On Linux and macOS, SSH is already installed. On Windows 10 and 11, open PowerShell; ssh is built in there too.

ssh pat@192.0.2.20

The part before @ is your username on the server, not on your laptop. The part after is the server’s address. The first time, SSH asks whether you trust the server’s fingerprint. Type yes. It remembers the answer and warns you loudly if that fingerprint ever changes, which is the protection against someone impersonating your server.

If the server doesn’t have SSH yet (Ubuntu Desktop doesn’t, out of the box), install it on the server itself:

sudo apt install -y openssh-server
sudo systemctl enable --now ssh

Why keys beat passwords

A password can be guessed, reused or phished. An SSH key is a pair of files: a private key that never leaves your computer, and a public key you copy to every server you want to log into. The server checks that you hold the matching private key, without the private key ever being sent anywhere.

Make a key on your own computer:

ssh-keygen -t ed25519 -C "laptop"

Press Enter to accept the default location. Adding a passphrase is a good idea on a laptop; it protects the key if the laptop is stolen.

You now have two files in ~/.ssh/: id_ed25519 (private, guard it) and id_ed25519.pub (public, safe to share).

Putting your public key on the server

The classic way is ssh-copy-id:

ssh-copy-id pat@192.0.2.20

It works almost everywhere. I’ll be honest, though: on my own network it fails against two NAS boxes, which reset the connection the moment a client offers a key the way ssh-copy-id does. If you hit something similar, this does the same job through an ordinary password login:

cat ~/.ssh/id_ed25519.pub | ssh pat@192.0.2.20 \
  "mkdir -p ~/.ssh && chmod 700 ~/.ssh && cat >> ~/.ssh/authorized_keys && chmod 600 ~/.ssh/authorized_keys"

Run it from a real terminal window. Some tools that run commands for you (including some editor and AI tool terminals) have no way to show a password prompt, so the login just silently fails.

Now ssh pat@192.0.2.20 should log in without asking for the server’s password.

Give your servers nicknames

Typing addresses gets old. Add entries to ~/.ssh/config on your computer:

Host nas
    HostName 192.0.2.20
    User pat
    IdentityFile ~/.ssh/id_ed25519

Host media
    HostName 192.0.2.30
    User root
    Port 2222

Now it’s just ssh nas. The Port line matters more than you’d think: some systems run SSH on a non-standard port, and “connection refused” is exactly what you get when you forget.

Reading the error messages

Connection refused: nothing is listening on that port. SSH isn’t running, or it’s on a different port.
Connection timed out: something (usually a firewall) is silently dropping the traffic, or the address is wrong.
Permission denied (publickey,password): you reached SSH, but the login failed. Check the username first; it’s case-sensitive. I’ve lost real time to typing “Pat” when the account was “pat”.
Host key verification failed: the server’s fingerprint changed. That’s expected after a reinstall. If you didn’t reinstall anything, stop and find out why before going further.

One step further, when you’re ready

Once keys work everywhere, you can turn off password logins entirely in /etc/ssh/sshd_config (PasswordAuthentication no, then sudo systemctl restart ssh). Keep a second session open while you test it, so a mistake doesn’t lock you out.

Next: what a minimal homelab actually needs.