Securing SSH: Essential Steps for Linux Servers

Last year, I woke up to dozens of failed login attempts on a client’s web server. Bots were hammering port 22 every few seconds – and if even one guess had landed, they’d have had full root access.

These days, one of the first things I do on any fresh server is lock down SSH. The out-of-the-box config is functional, sure – but it leaves way more doors open than I’m comfortable with.

In a previous guide, I covered a few essential SSH tweaks for new Ubuntu setups. In this one, we’re going deeper.

We'll walk through the sshd_config file line by line, and I’ll share the exact changes I make on every server I manage.

Along the way, I’ll include practical, real-world tips – not just theory – so you can confidently harden your SSH setup without locking yourself out.

The Basics – What SSH Really Is (And Why the Server Side Matters)

SSH (Secure Shell) is a protocol that lets you securely connect to and control a remote server over a network – usually from your terminal or an SSH client.

SSH is split into two parts:

• The server-side daemon (sshd) that runs on the server you want to access
• The client (ssh) that you use to connect to it – usually from your own machine

When I'm managing servers, I almost always connect from my Mac’s terminal. If you’re on Windows, a tool like PuTTY (or Windows Terminal with OpenSSH) will do the job.

But here's the important part: security decisions are made on the server side.

You can set whatever preferences you want on your SSH client – but at the end of the day, it’s the server (via the sshd_config file) that controls whether password logins are allowed, which keys are accepted, and so on.

You’ll find SSH settings in these places:

• Server: /etc/ssh/sshd_config
• Client-wide: /etc/ssh/ssh_config
• Per-user: ~/.ssh/config

Before You Start: A Few Things I’ve Learned the Hard Way

I've hardened dozens of servers over the years, and one thing I’ve learned is that not every setting works for every situation.

You need to think through how your server is used before blindly applying changes. A tweak that improves security in one setup might break something important in another.

For example: disabling password authentication is a great move – but if no key-based access is set up yet, you'll lock yourself out.

Here are a few best practices I personally follow every time:

• Back up your config file before touching anything.
• Most options in the file are commented out. Uncomment them before making changes.
• Some server providers (like Hetzner) use config overrides in /etc/ssh/sshd_config.d/. Always double-check that directory if something you changed isn’t working.

How I Reload SSH Safely (So I Don’t Lock Myself Out)

Once you’ve edited your sshd_config file, here’s the exact process I use to apply changes without risking a disconnect.

Test for syntax errors:

sudo sshd -t

If there's a mistake, the command will tell you – before you break SSH access.

Reload the SSH service (don’t restart it):

sudo systemctl reload ssh

Reloading is safer than restarting. It applies the new config but keeps your existing connection alive – which has saved me more than once.

Change the Default SSH Port

SSH runs on port 22 by default – and every scanner on the internet knows it. One easy way to reduce background noise (and random login attempts) is to move SSH to a non-standard port.

Open your SSH config file:

sudo vim /etc/ssh/sshd_config

Look for the Port line – it’s usually commented out and set to 22 by default:

#Port 22

Uncomment it and choose a different port number:

Port 592

Avoid using ports that are already taken by common services (like 80, 443, 3306, etc.). Pick something between 1024 and 65535 that doesn’t conflict with anything else on your server.

From now on, when you SSH into your server, you'll need to include the -p flag with your new port:

ssh -p 592 username@your.server.ip

Add a UFW Rule to Protect SSH

Changing your SSH port helps reduce noise, but a firewall adds an extra layer of protection – especially against brute-force attacks.

If you're using UFW (which ships with Ubuntu), you can rate-limit SSH connections like this:

sudo ufw limit 22/tcp

The limit rule allows connections but throttles repeated attempts from the same IP – perfect for slowing down brute-force bots without locking yourself out.

You can check your firewall status with:

sudo ufw status

And if UFW isn’t enabled yet:

sudo ufw enable

Fine-Tuning SSH Login Behavior

While scrolling through your sshd_config file, you’ll likely spot the following three lines – commented out by default:

#LoginGraceTime 2m
#MaxAuthTries 6
#MaxSessions 10

These settings control how users connect to your server – how long they have to log in, how many times they can try, and how many sessions they can open. The defaults are fairly relaxed, but in most cases, they’re more permissive than they need to be.

Here’s how I typically adjust them to reduce risk without getting in the way of normal usage:

LoginGraceTime 20
MaxAuthTries 3
MaxSessions 5

What these do:

• LoginGraceTime 20: This shortens the authentication window from 2 minutes to just 20 seconds. If someone can’t log in within that time, it’s usually a sign something’s wrong – or someone’s probing your server.
• MaxAuthTries 3: Reducing the number of allowed login attempts from 6 to 3 helps block brute-force attacks more quickly. I’ve found that 3 strikes is plenty if you're using keys or proper credentials.
• MaxSessions 5: By default, users can open up to 10 concurrent sessions. That’s rarely necessary. Cutting it down to 5 adds another layer of control without affecting usability.

Prevent Logins with Empty Passwords

Yes, it’s technically possible on a Linux server to create users without passwords – but it’s never a good idea. That would allow anyone to log in without authentication, which is obviously a huge security risk.

Thankfully, SSH is configured to block empty-password logins by default – but I always like to double-check this setting just to be safe.

Open your sshd_config file and look for:

#PermitEmptyPasswords no

As long as it's set to no (even if it’s still commented out), you’re good. No one with an empty password will be allowed to connect over SSH.

If, for any reason, it's set to yes, change it to:

PermitEmptyPasswords no

Set an Idle Timeout Interval

Leaving an SSH session open and idle – especially on a shared or unattended machine – can create a serious security risk.

To reduce that risk, I always configure SSH to automatically close idle connections after a short period of inactivity.

In your sshd_config file, these two settings control that behavior:

#ClientAliveInterval 0
#ClientAliveCountMax 3

By default, SSH does not close idle connections unless you explicitly configure it to do so. Here’s what the default values mean:

• ClientAliveInterval is set to 0, which means idle timeout is disabled by default
• ClientAliveCountMax is set to 3, but it only takes effect if ClientAliveInterval is greater than 0

Uncomment and update the values like this:

ClientAliveInterval 60
ClientAliveCountMax 3

Here’s what they do:

• ClientAliveInterval: This sets the number of seconds the server waits before sending a keep-alive check to the client. I typically set it to 60, meaning the server checks in once per minute.
• ClientAliveCountMax: This defines how many missed responses are allowed before the server disconnects the session. With a value of 3, that gives users up to 180 seconds (3 minutes) of inactivity before the session is closed.

Disable Rhosts-Based Authentication

Rhosts-based authentication is an old, insecure method that relies on trusting users based on their IP address or hostname – without requiring a password or key. It was more common in the early days of Unix systems, but it’s now considered unsafe and outdated.

SSH disables it by default, but I always recommend double-checking – especially on older servers or images that may carry legacy settings.

In your sshd_config file, look for:

#IgnoreRhosts yes

If it’s already set to yes (even if commented), you’re fine. But if it’s been changed, make sure it looks like this:

IgnoreRhosts yes

This ensures SSH will completely ignore .rhosts files, even if they exist – closing off an old and unnecessary security risk.

Disable X11 Forwarding

By default, SSH allows X11 forwarding, which lets you run graphical applications from a remote server and display them locally.

While that can be useful in specific cases (like running a GUI app on a remote server), it’s rarely needed on production servers – and the X11 protocol isn’t built with security in mind.

If you're not using it (and chances are, you're not), it’s best to disable it:

X11Forwarding no

I disable this on every server I deploy. It’s a simple step that helps reduce your server’s attack surface by turning off a feature you likely don’t need.

Disable Agent and TCP Forwarding

SSH supports agent forwarding and TCP forwarding, both useful in certain workflows – but they also come with security risks if left enabled unnecessarily.

• Agent Forwarding: It allows you to use your local SSH keys when hopping from one server to another – without copying keys to the first server. It’s convenient, but it can expose your keys if the intermediate server is compromised.
• TCP Forwarding (Port Forwarding): It lets you route network traffic between your local machine and remote server – great for tunneling traffic or exposing local services temporarily. But again, if you’re not using it, it’s just one more thing attackers can potentially abuse.

If you don’t explicitly need these features, it’s best to disable them:

AllowAgentForwarding no
AllowTcpForwarding no

These are typically enabled by default, so I always turn them off – unless there's a specific reason to keep them on.

Disable Root SSH Access

When you first set up a Linux server, it's common to log in as the root user – but it’s not something you should keep doing.

The root account has full control over the server, so it's easy to make a mistake that causes serious damage. It’s also a common target for brute-force attacks since the username is the same on every server.

Instead, I always recommend logging in with a non-root user that has sudo privileges. This way:

• You’re prompted for your password before running administrative commands
• Your actions are logged and traceable
• The username isn’t obvious to attackers

Here’s how to add one:

adduser username
usermod -aG sudo username

Now, locate the PermitRootLogin variable, which is usually commented out by default, and change its value to:

PermitRootLogin no

You’ll need to connect using your non-root user – which is much harder for an attacker to guess.

Disable Password Authentication

By default, you can log into a server over SSH using just a username and password. But there's a much more secure way: SSH key authentication.

This method uses a key pair – a public key (stored on the server) and a private key (kept on your machine). When you connect, the server checks if you have the matching private key. If you do, you're in.

This means no passwords to guess – only someone with your private key can log in.

To create a secure key pair, run:

ssh-keygen -b 4096

You’ll be prompted to choose a file path for saving the key – press ENTER to accept the default or enter a custom path to avoid overwriting any existing key.

You’ll also be asked to enter a passphrase. While optional, using a passphrase adds another layer of security and is strongly recommended.

Once done, two files will be created in your ~/.ssh directory:

• id_ed25519 – your private key
• id_ed25519.pub – your public key

To enable key-based access for your non-root user, copy the public key to the server using:

ssh-copy-id -i ~/.ssh/id_ed25519.pub your.non.root.user@your.server.ip

Once you’ve confirmed that key-based login is working, open your sshd_config file and look for:

#PasswordAuthentication yes

Uncomment it and change the value to:

PasswordAuthentication no

This ensures your server only accepts logins from users with a valid SSH key.

Restrict Who Can SSH Into the Server

Not every user on your server needs SSH access. In fact, the fewer people with SSH access, the better – it tightens security and reduces your attack surface.

There are a couple of ways to limit who can log in over SSH.

Option 1: Use AllowUsers (my preferred method)

This method is direct and easy to manage. You specify exactly which users are allowed to connect via SSH – everyone else is denied by default.

At the end of your sshd_config file, add:

AllowUsers ivan john elie

Now, only ivan, john, and elie can log in via SSH.

Option 2: Use AllowGroups

You can also create a special group (like sshusers), add your allowed users to it, and then add this to your config:

AllowGroups sshusers

This works well if you want to manage access by group instead of listing users individually.

Restrict Access by IP (Optional but Powerful)

If you have a static IP address (one that doesn't change), you can lock down SSH access even further – only allowing connections from your trusted IP.

You can do this with the same AllowUsers directive:

AllowUsers ivan@203.0.113.45

Now only ivan connecting from IP 203.0.113.45 can access the server.

Want to allow all users, but only from one IP? Use:

AllowUsers *@203.0.113.45

This is a great defense-in-depth tactic – just make sure you’re not on a dynamic IP (which changes), or you might accidentally lock yourself out.

Reload SSH (Don’t Forget!)

Once you've made any changes, always validate your SSH config before applying it:

sudo sshd -t

If there are no errors, reload SSH to apply your changes:

sudo systemctl reload ssh

You can also check the final, active SSH config with:

sudo sshd -T

This lets you confirm that your new settings – like AllowUsers – are active and working.

Bonus: Use sshd_config.d/ for Cleaner & More Scalable SSH Config

If you're managing multiple servers – or just want a cleaner setup – consider using the sshd_config.d/ directory instead of editing the main config file directly.

Modern Linux distros support config drop-ins via:

/etc/ssh/sshd_config.d/

Instead of changing /etc/ssh/sshd_config, create a new file like this:

sudo vim /etc/ssh/sshd_config.d/hardening.conf

Add your custom settings there – for example:

Port 592
PermitRootLogin no
PasswordAuthentication no
AllowUsers ivan
ClientAliveInterval 60
ClientAliveCountMax 3

This method keeps your config organized, survives server updates, and makes it easier to sync settings across multiple servers with version control or automation.

Conclusion and Final Thoughts

I hope this guide helped you go beyond the basics and gave you a clear, practical path to securing SSH on your Linux server.

Even small tweaks – like changing the SSH port or turning off unused features – can make a big difference in reducing risk and tightening your server’s defenses.

💬 Found this guide helpful?

I'd love to hear about your experiences, questions, or ideas in the discussion section below. Your feedback not only helps improve future guides – it helps fellow admins on their own journey.

Prefer a more direct conversation? Feel free to contact me anytime.