How to Set Up SSH Keys on Linux and macOS: A Step-by-Step Guide

Passwords are the weakest link on any server that is exposed to the internet. Within minutes of going online, automated bots start guessing usernames and passwords on port 22. SSH keys replace the password with a cryptographic key pair that is practically impossible to guess. This guide shows how to create SSH keys on Linux and macOS, copy them to a server, and then disable password login safely.

How SSH keys work

An SSH key is a pair of files. The private key stays on your own computer and must never be shared. The public key is placed on the server. When you connect, the server sends a challenge that only the matching private key can answer, so your client proves its identity without ever sending the key itself. Anyone may see your public key, but nobody should ever see your private key.

Step 1: Check for existing keys

Before creating a new key, see whether you already have one:

ls -al ~/.ssh

If you see files such as id_ed25519 and id_ed25519.pub, you already have a key pair. You can reuse it, or create a new one with a different file name so you do not overwrite the old one.

Step 2: Generate a new key pair

Ed25519 is the modern default: it is fast, secure and produces short keys.

ssh-keygen -t ed25519 -C "your_email@example.com"

Press Enter to accept the default location, then choose a strong passphrase when asked. The passphrase encrypts your private key on disk, so a stolen laptop does not automatically mean a stolen server. If you must connect to a very old system that does not support Ed25519, use ssh-keygen -t rsa -b 4096 instead.

This creates two files in ~/.ssh: id_ed25519 (private) and id_ed25519.pub (public).

Step 3: Copy the public key to your server

The easiest way is the ssh-copy-id tool. Replace the user name and address with your own:

ssh-copy-id -i ~/.ssh/id_ed25519.pub user@203.0.113.10

If ssh-copy-id is not available, you can do the same thing manually:

cat ~/.ssh/id_ed25519.pub | ssh user@203.0.113.10 "mkdir -p ~/.ssh && chmod 700 ~/.ssh && cat >> ~/.ssh/authorized_keys && chmod 600 ~/.ssh/authorized_keys"

The permissions matter. SSH ignores keys if the .ssh folder or the authorized_keys file is writable by other users.

Step 4: Test the key login

Open a connection:

ssh user@203.0.113.10

If it asks for the passphrase of your key rather than the server password, the setup works. Do not move on until this step succeeds, because the next step removes the password fallback.

Step 5: Use ssh-agent and a config file

Typing the passphrase for every connection gets tiresome. The ssh-agent program keeps your unlocked key in memory for the session. On macOS you can also store the passphrase in the Keychain:

ssh-add --apple-use-keychain ~/.ssh/id_ed25519

On Linux, start the agent and add the key with eval "$(ssh-agent -s)" followed by ssh-add ~/.ssh/id_ed25519. You can also save shortcuts in ~/.ssh/config:

Host myserver
    HostName 203.0.113.10
    User deploy
    IdentityFile ~/.ssh/id_ed25519
    AddKeysToAgent yes

Now ssh myserver is all you need to type.

Step 6: Disable password login

Once key login works, turn off passwords so bots can no longer guess their way in. Keep your current session open while you do this, so you can undo any mistake. On the server, edit /etc/ssh/sshd_config:

PasswordAuthentication no
PubkeyAuthentication yes
PermitRootLogin no

Check the syntax and reload the service:

sudo sshd -t
sudo systemctl reload ssh

On some distributions the service is called sshd instead of ssh. Now open a second terminal and confirm you can still log in with your key. Also run sudo sshd -T | grep -i passwordauthentication. If it still says yes, another file in /etc/ssh/sshd_config.d/, often created by cloud images, is overriding your setting.

A note for Windows users

Windows 10 and 11 include an OpenSSH client, so ssh-keygen and ssh work the same way in PowerShell. Since ssh-copy-id is not available there, open the .pub file, copy its single line, and paste it into ~/.ssh/authorized_keys on the server.

Security best practices

• Always protect your private key with a passphrase.
• Never upload a private key to GitHub, chat tools or cloud storage.
• Use a separate key for each device, so you can revoke one without affecting the others.
• Remove old keys from authorized_keys when a laptop is lost or a team member leaves.
• Consider a hardware security key for high-value servers.

Troubleshooting common errors

Permission denied (publickey): check that you are using the correct user name, that your public key is in that user's authorized_keys, and that ~/.ssh is set to 700 and authorized_keys to 600. Run ssh -v user@host to see which keys are being tried.

WARNING: UNPROTECTED PRIVATE KEY FILE: your private key is readable by others. Fix it with chmod 600 ~/.ssh/id_ed25519.

Summary

Generate an Ed25519 key with a passphrase, copy the public key to your server, test the login, and only then disable password authentication. This one change removes the most common attack path against Linux servers. For the rest of the checklist, read my guide on securing Linux servers for production, or the cybersecurity roadmap for 2026.

← All articles