Basics of Securing a Linux Server

(1) Change User Password + Disable SSH password auth

If you received a password for this server, then it's likely that SSH/Cockpit password authentication is enabled for your user.

(Cockpit is a WebUI system management tool that's often pre-installed on Redhat-based systems, instructions related to Cockpit generally don't apply to Ubuntu/Debian-based systems unless you've manually installed Cockpit or a similar remote management system)

(1.1) Disable SSH Password Auth

We STRONGLY recommend that you disable SSH password authentication (use SSH keys instead) to protect your server from compromise via password brute forcing.

We've personally experienced a server with a 40+ character password eventually getting successfully bruteforced by a huge, relentless botnet, with many IP addresses, which allowed it to skirt around being IP banned after 3 failed attempts, simply by switching IPs and continuing the attack.

If you have SSH password authentication enabled, and your SSH port is open to the internet (i.e. not firewalled and restricted to specific IPs), then it's not a matter of "IF" you'll be bruteforced, but a matter of "WHEN".

To disable SSH password authentication, open up the file /etc/ssh/sshd_config

sudo nano /etc/ssh/sshd_config

Find the line starting with 'PasswordAuthentication' - and make sure that it's set to 'no':

PasswordAuthentication no

Restart SSH:

sudo systemctl restart ssh

(1.2) Change your normal user's password

(1.2.1) - Use passwd to change your user's password

You should also change your user's password regardless of whether you disabled SSH password auth, using the command:

sudo passwd yourusername

(1.2.2) - If SSH password auth is off + other password-auth firewalled, set a password that'd be easy to type at a console

If you've disabled SSH password authentication, and you've firewalled any other high-risk services (e.g. server management software, such as RedHat's Cockpit) (or simply disabled them), then you should be safe to set a relatively secure but easy to type word-based password, with a few numbers, and optionally common symbols such as ! and @. For example Orange!Donkey%Potato49.

The reason for this, is that if you or a staff member needs to login to your server via video/serial console, you often cannot paste at the console, so you'd need to type out every character - so a complex password containing a random mix of upper/lowercase letters, symbols, and numbers, could be extremely difficult to type, and sometimes the password prompt may even timeout if it takes you too long to type the password.

There can also be a risk of keyboard layout issues, such as the server having a US keyboard layout configured, or in some cases - a German keyboard layout, while our staff having a UK/CA keyboard layout, which can result in us being unable to type certain symbols such as $, / or even " (quote).

If this happens, some passwords can simply be impossible to type at the console, which can severely impact the time that it takes to get your server back online if something happens which prevents you from being able to access it via SSH. If we're unable to enter any user's password at the console, then we would be forced to reboot the server in order to attempt to reset the password of at least one user to something which can be typed at a serial/video console without issues.

(2) Change Root Password

By default, the password for 'root' is set to 'Oracle69', however, by default - SSH password authentication for 'root' is disabled, as is logging into Cockpit using the root user.

This generally means that the root user can't be logged into remotely using password auth, however, we still strongly recommend that you change the password now that you have the server login details.

To change root's password, run the following two commands:

sudo su -
passwd

Note that generally, the 'root' user's password, is only normally used in emergency cases - i.e. to login via the "hardware" TTY such as the serial console, or VGA graphical console.

At the time of writing (25 JUN 2021), it's not currently possible for customers to access their server's console directly, so anything that needs to be done via the physical console, you must give a Privex staff member your server's root user password, or the password for a non-root user which has 'sudo' / 'su' capabilities (unless the console task doesn't require the use of root).

(3) Setup a Firewall to protect high-risk services or ones which don't need to be exposed

(3.1) Ubuntu/Debian-based distros

(3.1.1) Pyrewall

For Ubuntu / Debian, and any derivative distros of them - we personally recommend our IPTables firewall wrapper software and language - Pyrewall (written in Python).

The Pyre language which we invented - is designed to feel "human" - similar to Python or Ruby, with syntax which almost reads like English, e.g. allow port 80,443 is valid Pyre to whitelist both TCP ports 80 and 443 - and deny port 22 from 12.34.56.1,2a07:e07::/64 is valid Pyre to blacklist the single IPv4 address 12.34.56.1 and IPv6 prefix 2a07:e07::/64 from accessing port 22.

To install Pyrewall, you need Python 3.7 or newer, though it may also work with Python 3.6 if you install the dataclasses python package. Once you have Python installed, you can install the package from PyPi using python3.8 -m pip install -U pyrewall - which will install the Pyrewall code, any package dependencies, as well as install the utilities pyre, pyre4, and pyre6 into your /usr/bin folder, so that you can run them as standard CLI commands from any shell.

(3.1.2) Quickstart install and configure Pyrewall

To simplify the process of installing Pyrewall, we've prepared the commands from start to finish for installing Pyrewall on an ubuntu/debian-based system:

#####
# Become the root user - since nearly all of the following commands need to be ran as root
#####
sudo su -

#####
# Some of these apt install commands may fail, depending on which distro you're running.
# Depending on which version of a given distro that you're running, you might not even
# have python3.8 available.
# If it complains that the python3.8 package doesn't exist - try replacing 'python3.8'
# in the below commands with python3.7 and/or python3.9
#####

apt update
apt install -y python3 python3-dev python3.8 python3.8-dev
apt install -y python3-venv python3.8-venv
apt install -y python3-pip
apt install -y python3.8-pip

#####
# Use the newest version of Python available on your system to install Pyrewall,
# such as Python 3.7, 3.8, 3.9, or even 3.10
#####
python3.8 -m pip install -U pyrewall

#####
# Install .pyre syntax highlighters for both VIM and Nano
#####
mkdir -pv /etc/vim/syntax
wget -O /etc/vim/syntax/pyrewall.vim https://raw.githubusercontent.com/Privex/pyrewall-syntax-highlighters/master/Vim/pyrewall.vim
echo 'autocmd BufNewFile,BufRead *.pyre set syntax=pyrewall' | tee -a /etc/vim/vimrc.local | tee -a ~/.vimrc

mkdir -pv /usr/share/nano
wget -O /usr/share/nano/pyre.nanorc https://raw.githubusercontent.com/Privex/pyrewall-syntax-highlighters/master/Nano/pyre.nanorc

#####
# Generate a default "drop unless whitelisted" PyreWall config at /etc/pyrewall/rules.pyre
#####
mkdir -pv /etc/pyrewall
tee -a /etc/pyrewall/rules.pyre <<"EOF"
@chain INPUT DROP
@chain FORWARD DROP

# sane.pyre includes a lot of important firewall rules that practically every Linux firewall needs,
# such as:
#   - allowing related/established traffic for both INPUT and FORWARD traffic,
#   - allowing INPUT/FORWARD traffic into 'lo' - the loopback interface
#   - dropping INVALID packets,
#   - allowing incoming IPv4 ICMP packets
#   - allowing only SAFE incoming IPv6 ICMPv6 traffic
#      - only allowing 'possibly risky' but important ICMPv6 packet types from fe80::/10 (LAN subnet)
#      - allowing 'safe' and important ICMPv6 packets from anywhere
#
# if you're concerned/curious, you can view the source of the default firewall templates here: 
#     https://github.com/Privex/pyrewall/tree/master/privex/pyrewall/configs/templates
#
@import templates/sane.pyre

rem Allow Privex Corporate IPv4 and IPv6 subnets
allow chain input,forward from 2a07:e00::/32,185.130.44.0/27,185.130.45.0/27

rem (EXAMPLE) Your home computer/network's IP addresses, or your bounce server's
allow chain input,forward from 2a07:e01:2:1::2,185.130.44.40

rem4 Allow LAN / VPN IPv4
allow chain input,forward from 192.168.0.0/16,10.0.0.0/8,fe80::/64,fd00::/12

rem Allow connections to the standard SSH port from anywhere - for the local SSH server
allow port 22

rem Allow Mosh / UDP above port 10,000
allow port 10000-65535 udp

rem Allow HTTP / HTTPS via v4 and v6, with both TCP and UDP
allow port 80,443 both

# Rsync
#allow port 873
# NFS
#allow port 111,2049,33333 both from 2a07:e00::/29,185.130.44.0/22,10.0.0.0/8
# SMB (Samba / CIFS) (445) and NetBIOS (required for some SMB functionality) (139)
#allow port 139,445 both from 2a07:e00::/29,185.130.44.0/22,10.0.0.0/8
# FTP
#allow port 21 both
# IPFS
# allow port 4000-4010 both

EOF

#####
# Install + enable the 'pyrewall.service' SystemD service, and 'pyre' alias service,
# which will also auto-start pyrewall, loading your Pyre rules and executing their
# IPTables equivalent commands.
#####
pyre install_service

(3.2) RedHat Based Distros (RHEL/CentOS/Fedora/Oracle/Rocky)

(3.2.1) Basic Firewalld public and DMZ (trusted) configs

Create the following two files:

/etc/firewalld/zones/public.xml

<?xml version="1.0" encoding="utf-8"?>
<zone>
    <short>Public</short>
    <description>For use in public areas. You do not trust the other computers on networks to not harm your computer. Only selected incoming connections are accepted.</description>
    <service name="ssh"/>
    <service name="dhcpv6-client"/>
    <service name="cockpit"/>
    <service name="http"/>
    <service name="https"/>
</zone>

/etc/firewalld/zones/dmz.xml

<?xml version="1.0" encoding="utf-8"?>
<zone>
    <short>DMZ</short>
    <description>For computers in your demilitarized zone that are publicly-accessible with limited access to your internal network. Only selected incoming connections are accepted.</description>
    <source address="2a07:e00::/32"/>
    <source address="2a07:e01:5::/48"/>
    <source address="185.130.44.0/27"/>
    <service name="ssh"/>
    <service name="cockpit"/>
    <service name="allow-everything-dmz"/>
</zone>

Then restart firewalld:

systemctl restart firewalld

(3.2.2) Firewall Cockpit

We recommend that you firewall Cockpit so that it's only accessible from trusted IP addresses.

Cockpit is a WebUI for remote management, which runs on port 9090 by default, and is enabled + accessible from the internet by default. Unlike SSH, it only appears to support password authentication, and since Cockpit can be used for anything from restarting services, to formatting disks, and creating Linux users - Cockpit being successfully bruteforced would be equally as dangerous as SSH being compromised.

To firewall Cockpit so that it's only accessible from whitelisted IPs:

  1. Open the file /etc/firewalld/zones/public.xml in a text editor such as nano or vim as root, e.g. sudo vim /etc/firewalld/zones/public.xml

  2. Remove the line <service name="cockpit"/> (the line whitelists cockpit access from anywhere)

  3. Open the file /etc/firewalld/zones/dmz.xml in a text editor as root.

    This file contains trusted IP addresses and subnets which are "DMZ'd", meaning that they can connect to every single port/service on the system.

  4. Add each IP address and/or prefix (v4 and v6 supported) which you trust, and would like to be able to access all ports/services on the system, including Cockpit and SSH.

    Example line: <source address="1.2.3.4/32"/> (whitelists the single IPv4 address 1.2.3.4)

  5. Restart firewalld using: 'sudo systemctl restart firewalld'