Secure Your Linux Server: Essential Practices

Before you start using your Linux server, make sure to secure it to keep it safe while managing.

In this guide, I will cover essential security practices that should be implemented first on every newly deployed server.

Though there are advanced security measures, focusing on these essentials first lays a strong foundation for safe server management.

Stay tuned till the end of this guide, where I’ll provide guidance on what steps to take next regarding advanced security measures.

Preparation

To make the most of this guide, ensure you have a properly set up Ubuntu server.

If you don’t have one, consider getting a free VPS server to follow along.

Following along on your own server will enhance your understanding and practical experience.

Strong Passwords

This is a well-known practice, and there isn’t much to add.

Avoid weak passwords and use a password manager for generating and securely storing complex passwords. I personally use Bitwarden.

It’s crucial to refrain from using the same password repeatedly.

Always opt for unique, strong passwords that incorporate symbols, not just letters and numbers.

Now, it’s time to learn how to change a user’s password on a Linux server using a simple command.

After accessing your server for the first time, you might want to change the password for the root user.

You can do this with the following command:

passwd root

The passwd command is used to change the password for any user on your server.

Simply replace root with the user for whom you want to change the password.

Adding a Non-Root User

When accessing your server for the first time, you are probably using the root user, which is not recommended due to its total control over the entire server.

It is very easy to make mistakes when running commands using root, as you can accidentally break your server.

To mitigate risks, it’s safer to use a non-root user, requiring the sudo prefix for administrative commands and a password prompt.

Another advantage of using a non-root user is protecting your server from brute force attacks.

Since the root user is the default on all Linux servers, attackers often attempt to brute force it to guess your password.

Note: Adding a new user doesn’t eliminate the root user. I’ll discuss preventing access with the root user in the SSH Hardening section.

We will start by adding a new user using the following command:

adduser <username>

Try to use a randomly generated string for the username instead of well-known usernames like admin, administrator, etc.

The server will ask for a password and some optional details. If you’re in a hurry, just hit the ENTER key to skip.

Now, we need to grant this user root privileges.

This can be achieved in two ways:

• The first method is to add the user to the sudo group.
• The second method is to manually add the user to the /etc/sudoers file.

The first method is simple, as it only requires a single command:

usermod -aG sudo <username>

Now you can run commands that require root privileges without the need for the root user.

Use the exit command or the logout command to exit the current session and reconnect to your server with the new user.

Try running the apt update command.

It should not work directly, as you need to run the command like this: sudo apt update and then enter your password.

Now you can use the non-root user instead of root.

Read more: I discussed managing users effectively and the second method in a detailed guide.

Patching

It is crucial to regularly update your server.

Many hacks happen because servers aren’t patched with the latest security updates.

Not taking patching seriously is like leaving your front door wide open and being shocked when someone walks in uninvited.

Let’s learn how to update our server. It’s just two simple commands.

Note: I assume from now on that you are connected to your server with the non-root user.

Begin by updating the package list on your server with the following command:

sudo apt update

This command prompts the server to scan the server’s packages and identify those requiring updates, including security patches.

Once this is done, run the following command to update your server’s packages that need updating:

sudo apt upgrade

The server may ask for confirmation by displaying a prompt that requires a yes or no response.

Make sure to type yes.

The updating process may take a while, depending on the number of updates needed.

Read more: I’ve written a detailed guide on automating security updates for a Linux server, as well as enabling the Canonical Livepatch Service.

Using SSH Keys

When it comes to accessing your server, you probably use your password with Secure Shell (SSH).

However, is this the most secure way to access your server? Not necessarily.

A stronger alternative is SSH key authentication, which uses a key pair – a public key and a private key – for enhanced security.

This method removes the need for password authentication.

The public key is stored on the server, and the private key is kept by you, the server administrator.

Authentication occurs when the server sends a message encrypted with the public key, and if the user can decrypt it using the private key, access is granted.

This ensures that only those with the private key can access the server, significantly improving security over password-only authentication.

Therefore, it’s crucial to securely store the private key, while the public key can be freely shared.

When relying solely on password authentication, anyone with our password can access the server.

Using SSH keys, on the other hand, restricts access to only those with the corresponding private key.

Now, let’s generate our key pair and transfer the public key to our server.

First, we want to ensure that we don’t already have a key pair because running this command will override it.

Open your local terminal (without accessing the server) and run the following command to list the contents of the ~/.ssh directory:

ls -l ~/.ssh

If you get a total 0 output, it means that the directory is empty, indicating that you don’t have any SSH keys, which means you are good to go.

If you see the known_hosts and known_hosts.old files listed in the output, you are also good to go as these files contain a list of known hosts (servers) you have previously accessed.

If you get an error indicating that the directory doesn’t exist, don’t worry, as creating the key pair will automatically create the .ssh directory.

However, if you already have any existing key pairs in that directory, ensure to back them up before creating a new key pair.

Now, run the following command:

ssh-keygen -b 4096 

Output
Generating public/private rsa key pair.
Enter file in which to save the key (/Users/ivansalloum/.ssh/id_rsa):

We are using the ssh-keygen command to create a new key pair.

The -b option allows us to specify the bit size of our key, which is 4096 in our case, providing a stronger key than the default size.

After running the command, you will be asked where to save the key.

It defaults to the .ssh directory inside your home directory.

If you don’t specify a path and file name, the command will use the default path and file name.

I mentioned earlier that running this command will override an already generated key pair.

This is because the command has a default location and file name to store the key.

If you consistently stick with the default option, it will override an existing key pair, as the newly created key pair has the same name as the previously created one and is stored in the same location.

When you want to create multiple key pairs for different use cases, it’s always recommended to change their names.

You can change the default name of the key by entering the same path and a name after the colon like this:

Enter file in which to save the key (/Users/ivansalloum/.ssh/id_rsa): /Users/ivansalloum/.ssh/<key_name>

This way, I’m creating a new key pair that won’t be overridden when generating a new key pair.

Note: If you’re creating a key pair on Linux, the path would likely be: /home/user/.ssh

I will stick with the defaults for now, assuming you haven’t generated any key pairs before.

Press the ENTER key to stick with the defaults, as I did, and then you will be prompted to enter a passphrase.

Enter passphrase (empty for no passphrase):

A passphrase adds an extra layer of security and functions similarly to a password.

You have to enter it each time you want to connect to your server, meaning you need both your private key and the passphrase.

This way, if someone gains access to your private key, they would also need the passphrase to access the server.

You can skip adding a passphrase by pressing the ENTER key twice.

After skipping or adding a passphrase, your key pair will be generated.

Now, if we look at the contents of the ~/.ssh directory, we will find the private and public keys we just generated:

id_rsa id_rsa.pub

The id_rsa file is the private key, which shouldn’t be shown to anyone, and the id_rsa.pub file is the public key, which we should copy to our server.

Before we proceed to copy our public key to the server, let me clarify something.

If you place the public key in the .ssh directory within the home directory of the root user, you can only use key authentication when accessing the server with the root user.

On the other hand, if you copy the public key to the .ssh directory in the home directory of another user, that specific user is granted access to the server using key authentication, while the root user is not.

It’s essential to copy the public key only to the .ssh directory of the user you intend to use for accessing the server with key authentication.

Users without a public key can still utilize password authentication.

However, if you’ve disabled password authentication in the sshd_config file, users are required to use key authentication.

Users lacking a public key won’t be able to access the server in this scenario.

Note: Further details on this will be covered in the SSH Hardening section.

As we’ve previously created a non-root user for future use, there’s no need to copy the public key for the root user, as we don’t intend to access our server with the root user.

When copying the public key to our server, you should place it inside a file called authorized_keys within the .ssh directory in the home directory of the user.

You can do this manually, or you can use the ssh-copy-id command, which I prefer.

If you want to do this manually, copy the content of the id_rsa.pub file, access your server with the user for whom you want to enable key authentication, and paste the content of the public key inside the authorized_keys file.

If the .ssh directory doesn’t exist, create it.

If the authorized_keys file doesn’t exist too, create it inside the .ssh directory, add the public key to it, save and close.

Now, try to connect to your server again with the same user, and you should be able to access the server without entering your password.

However, if you decided to use a passphrase, you should enter your passphrase to access your server.

If you don’t want to do it manually, use the following command to copy the public key:

ssh-copy-id -i ~/.ssh/id_rsa.pub <user>@<serverip>

Replace user with the username of the user to whom you want to copy the public key, and replace server_ip with the IP address of your server.

You will be prompted to enter the password for that user, enter it and press the ENTER key.

You can now use key authentication instead of password authentication.

SSH Hardening

For now, we have a non-root user with root privileges that we can use instead of the root user.

Additionally, we have generated an SSH key pair, copied the public key for that user, and can successfully connect to our server using SSH key authentication.

Our next step involves restricting root user access via SSH, as we’ve added a non-root user for future use.

Furthermore, we want to disable password authentication to exclusively use SSH keys.

Preventing root access means any root user attempts will fail and disabling password authentication ensures only our non-root user, with the public key, can access the server.

With all these, we can ensure that only our non-root user can access the server using key authentication, and all other users are unable to access the server.

This will help us stop brute force attacks.

To accomplish this, we should edit the /etc/ssh/sshd_config file and modify two variables.

Open the sshd_config file with your preferred editor and search for the #PasswordAuthentication and #PermitRootLogin variables.

Uncomment these two variables and change both of their values to no like this:

PasswordAuthentication no
PermitRootLogin no

Save and close the file.

Now, we should reload the SSH service for these changes to take effect.

Run the following command to do so:

sudo systemctl reload ssh

If you try to access the server with the root user now, you will receive an error:

root@serverip: Permission denied (publickey).

If you try to access the server with a user that doesn’t have a public key, you will get the same error.

Read more: For an in-depth exploration of SSH security and advanced practices, check out my dedicated guide.

Setting Up a Firewall

A firewall plays a key role in server security.

Think of it as your server’s bouncer – it filters incoming and outgoing traffic, allowing only authorized access to specific ports.

To make this process easy, we’ll use the Uncomplicated Firewall (UFW).

UFW simplifies the setup and management of firewall rules, making it user-friendly.

On Debian-based distributions, like Ubuntu, it often comes pre-packaged.

You can check and install it using:

sudo apt install ufw

Many server providers configure the UFW firewall upon deploying the server to allow only SSH connections, enabling you to connect to the server.

If you have a server from Vultr, UFW is likely to be enabled by default.

In my case with Hetzner, UFW is not enabled.

You can check the status of UFW and your current ruleset using this command:

sudo ufw status

The command’s output will either indicate that UFW is inactive, or that it is active with your current rule set.

If UFW is currently inactive, that’s fine, as we’ll proceed to configure it properly and activate it.

However, if UFW is already active, disable and reset it using the following commands:

sudo ufw disable
sudo ufw reset

You can re-enable it once you have added all the rules and finished configuring it.

UFW comes with a default policy that blocks all incoming traffic while allowing all outgoing traffic.

This means that no one can reach our server, but our server can communicate with the outside world.

Since there is no issue with our server reaching the outside world, there is no need to make any changes to that aspect.

However, since UFW blocks all incoming traffic by default, we should only open the necessary ports and permit traffic through them.

SSH connections use port 22, and we want to ensure that our firewall allows traffic on this port so that we can access our server again.

Allow SSH connections by using the following command:

sudo ufw allow 22/tcp

Now enable the firewall by using this command:

sudo ufw enable

Type y and press the ENTER key to proceed.

If you check the current status of UFW, you will see that it is active, and there is now only one rule allowing traffic on port 22, which we just added.

Output
Status: active

To                        Action      From
--                        ------      ----
22/tcp                    ALLOW       Anywhere                  
22/tcp (v6)               ALLOW       Anywhere (v6)

The firewall is currently blocking all traffic except for SSH.

If you install and configure additional services, you will need to adjust the firewall settings to permit traffic to your server for the new services.

For instance, if you install a web server like NGINX or Apache, you should allow traffic on ports 80 (HTTP) and 443 (HTTPS).

Read more: I’ve written a detailed guide on setting up a firewall using UFW.

The Config File

The config file is basically a list that holds information about the servers we manage and access using SSH.

It simplifies the process of connecting to our servers and organizes server information into a single file for easy access.

This is not a security-related tip or trick, but it could come in handy when troubleshooting issues and for quick access to a specific server when managing a lot of servers.

Open your local terminal (without accessing the server) and create the file using the following command:

touch ~/.ssh/config

Open the file with your preferred editor and add the following:

Host server1
        HostName 81.41.156.93
        User ivan
        Port 22
        IdentityFile ~/.ssh/id_rsa

This is an example of a server I named server1 with the IP address 81.41.156.93.

I want to use the user ivan to access my server using port 22 and the private key id_rsa.

Now, if I want to access my server, instead of using this command:

ssh -p 22 -i ~/.ssh/id_rsa ivan@81.41.156.93

I simply use this:

ssh server1

Do you see how easy it is to access a server now?

I added the -p 22 option just to show you how the command will look without the config file.

SSH will use port 22 by default, so there is no need to add this option when accessing the server normally.

Now, replace the information above with the details of your server and attempt to access the server as I did.

Leave the SSH port as it is if you didn’t change it, as SSH uses port 22 by default.

When you have multiple servers, add a new Host block for each server.

Conclusion and Final Thoughts

Great job reaching the end!

In this guide, you’ve learned crucial initial steps for safely managing a new server.

However, there’s more to securing Linux servers — advanced measures that I’ve covered comprehensively in my complete server security and hardening guide.

It’s a collection of all security and hardening measures with links to detailed guides for each.

By following these guides one by one, your server will be thoroughly secured.

If you found value in this guide or have any questions or feedback, please don’t hesitate to share your thoughts in the comments section below.

Your input is greatly appreciated, and you can also contact me directly if you prefer.