Linux systemd Targets & SSH Hardening Configuration

Introduction

Service configuration is essential for Linux system administration. This comprehensive guide covers systemd service management, SSH configuration with key-based authentication, Apache web server setup with SSL, MySQL database configuration, NTP time synchronization, and remote logging with rsyslog.

We'll explore systemd targets (runlevels), service dependencies, SSH hardening, Apache virtual hosts, MySQL security, and centralized logging. Each section includes practical configurations and production best practices.

Linux Administration Series

📚 View Complete Linux Administration Guide - Master all 7 parts with our comprehensive learning path.

This is Part IV of our comprehensive 7-part Linux administration guide:

Part I: File System & Process Management
Part II: User Authentication & LDAP
Part III: UFW Firewall & Networking
Part IV: systemd & SSH Hardening (Current Article)
Part V: Postfix Email Server
Part VI: QEMU KVM Virtualization
Part VII: LVM & RAID Storage

← Part III | Part V →

Systemd and System Targets

Systemd Architecture

System targets (runlevels):

Target	Legacy Runlevel	Description
poweroff.target	0	Shutdown system
rescue.target	1	Single-user mode (recovery)
multi-user.target	2, 3, 4	Multi-user, CLI only
graphical.target	5	Multi-user with GUI
reboot.target	6	Reboot system

Managing targets:

# Check current target
systemctl get-default

# List all targets
systemctl list-units --type=target

# List available targets
systemctl list-unit-files --type=target

# Switch to different target (temporary)
sudo systemctl isolate rescue.target
sudo systemctl isolate multi-user.target
sudo systemctl isolate graphical.target

# Set default target (permanent)
sudo systemctl set-default multi-user.target
sudo systemctl set-default graphical.target

# Check target dependencies
systemctl list-dependencies graphical.target

# Show target configuration
systemctl cat multi-user.target

Service Management with Systemd

Service Lifecycle

Service commands:

# Service status (systemctl documentation: https://www.freedesktop.org/software/systemd/man/systemctl.html)
systemctl status nginx
systemctl status nginx.service

# Start/stop service
sudo systemctl start nginx
sudo systemctl stop nginx

# Restart service
sudo systemctl restart nginx

# Reload configuration (no restart)
sudo systemctl reload nginx

# Enable/disable service (autostart on boot)
sudo systemctl enable nginx
sudo systemctl disable nginx

# Check if service is enabled
systemctl is-enabled nginx

# Check if service is active
systemctl is-active nginx

# List all services
systemctl list-units --type=service
systemctl list-units --type=service --state=running
systemctl list-units --type=service --state=failed

# View service logs
journalctl -u nginx
journalctl -u nginx -f  # follow logs
journalctl -u nginx --since today
journalctl -u nginx --since "2025-01-01"

SSH Server Configuration

SSH Architecture

Installing and configuring SSH server:

# Install OpenSSH server (OpenSSH documentation: https://www.openssh.com/)
sudo apt install openssh-server

# Check SSH service status
systemctl status sshd
systemctl status ssh  # Debian/Ubuntu

# Main configuration file
sudo vim /etc/ssh/sshd_config

# Key configuration options:
Port 22                          # Default SSH port
PermitRootLogin no              # Disable root login
PubkeyAuthentication yes        # Enable key-based auth
PasswordAuthentication yes      # Allow password auth
PermitEmptyPasswords no         # Deny empty passwords
X11Forwarding no                # Disable X11 forwarding
MaxAuthTries 3                  # Max authentication attempts
ClientAliveInterval 300         # Keep-alive interval
ClientAliveCountMax 2           # Max keep-alive messages

# Restart SSH after changes
sudo systemctl restart sshd

Access control:

# Allow specific users only
AllowUsers john alice admin

# Deny specific users
DenyUsers baduser hacker

# Allow specific groups
AllowGroups sshusers admins

# Deny specific groups
DenyGroups noremote guests

# Restrict by IP (using Match block)
Match Address 192.168.1.0/24
    PasswordAuthentication yes
Match Address *,!192.168.1.0/24
    PasswordAuthentication no

SSH Key-Based Authentication

SSH Key Architecture

Setting up SSH keys:

# Generate SSH key pair (client) - ssh-keygen man page: https://man.openbsd.org/ssh-keygen
ssh-keygen -t rsa -b 4096 -C "user@example.com"
# OR modern Ed25519 (recommended)
ssh-keygen -t ed25519 -C "user@example.com"

# Key locations:
# Private: ~/.ssh/id_rsa (or id_ed25519)
# Public: ~/.ssh/id_rsa.pub (or id_ed25519.pub)

# Copy public key to server (automated)
ssh-copy-id user@server.com

# Manual method: copy public key content
cat ~/.ssh/id_rsa.pub
# Then on server:
mkdir -p ~/.ssh
chmod 700 ~/.ssh
echo "PUBLIC_KEY_CONTENT" >> ~/.ssh/authorized_keys
chmod 600 ~/.ssh/authorized_keys

# Test key-based login
ssh user@server.com

# Use specific key
ssh -i ~/.ssh/custom_key user@server.com

SSH client configuration (~/.ssh/config):

# Create client config
vim ~/.ssh/config

# Example configuration:
Host myserver
    HostName 192.168.1.100
    Port 22
    User john
    IdentityFile ~/.ssh/id_rsa
    ServerAliveInterval 60

Host github
    HostName github.com
    User git
    IdentityFile ~/.ssh/github_key

Host *.example.com
    User admin
    Port 2222

# Now simply use:
ssh myserver

Disable password authentication (after keys work):

# Edit SSH config
sudo vim /etc/ssh/sshd_config

# Set these values:
PasswordAuthentication no
PubkeyAuthentication yes
ChallengeResponseAuthentication no

# Restart SSH
sudo systemctl restart sshd

Apache Web Server Configuration

Apache Architecture

Installing Apache:

# Install Apache (Apache HTTP Server documentation: https://httpd.apache.org/docs/)
sudo apt install apache2

# Check status
systemctl status apache2

# Main configuration
sudo vim /etc/apache2/apache2.conf

# Test configuration
sudo apache2ctl configtest

# Restart Apache
sudo systemctl restart apache2

Managing Apache modules:

# List available modules
ls /etc/apache2/mods-available/

# List enabled modules
ls /etc/apache2/mods-enabled/
apache2ctl -M

# Enable module
sudo a2enmod rewrite
sudo a2enmod ssl
sudo a2enmod headers

# Disable module
sudo a2dismod status

# Restart after module changes
sudo systemctl restart apache2

Virtual Hosts:

# Create virtual host config
sudo vim /etc/apache2/sites-available/example.com.conf

# Example virtual host:
<VirtualHost *:80>
    ServerName example.com
    ServerAlias www.example.com
    ServerAdmin admin@example.com

    DocumentRoot /var/www/example.com

    <Directory /var/www/example.com>
        Options -Indexes +FollowSymLinks
        AllowOverride All
        Require all granted
    </Directory>

    ErrorLog ${APACHE_LOG_DIR}/example.com-error.log
    CustomLog ${APACHE_LOG_DIR}/example.com-access.log combined
</VirtualHost>

# Create document root
sudo mkdir -p /var/www/example.com
sudo chown -R www-data:www-data /var/www/example.com

# Enable site
sudo a2ensite example.com.conf

# Disable site
sudo a2dissite 000-default.conf

# Reload Apache
sudo systemctl reload apache2

Apache SSL Configuration

SSL/TLS Architecture

Generate self-signed certificate (development):

# Generate SSL certificate
sudo openssl req -x509 -nodes -days 365 -newkey rsa:2048 \
  -keyout /etc/ssl/private/apache-selfsigned.key \
  -out /etc/ssl/certs/apache-selfsigned.crt

# Answer prompts:
# Country Name: US
# State: New York
# Locality: New York City
# Organization: My Company
# Common Name: example.com

Configure Apache SSL:

# Enable SSL module
sudo a2enmod ssl

# Create SSL virtual host
sudo vim /etc/apache2/sites-available/example.com-ssl.conf

# SSL Virtual Host:
<VirtualHost *:443>
    ServerName example.com
    ServerAlias www.example.com

    DocumentRoot /var/www/example.com

    SSLEngine on
    SSLCertificateFile /etc/ssl/certs/apache-selfsigned.crt
    SSLCertificateKeyFile /etc/ssl/private/apache-selfsigned.key

    # Modern SSL configuration
    SSLProtocol all -SSLv2 -SSLv3 -TLSv1 -TLSv1.1
    SSLCipherSuite HIGH:!aNULL:!MD5
    SSLHonorCipherOrder on

    <Directory /var/www/example.com>
        Options -Indexes +FollowSymLinks
        AllowOverride All
        Require all granted
    </Directory>

    ErrorLog ${APACHE_LOG_DIR}/example.com-ssl-error.log
    CustomLog ${APACHE_LOG_DIR}/example.com-ssl-access.log combined
</VirtualHost>

# Enable SSL site
sudo a2ensite example.com-ssl.conf

# Restart Apache
sudo systemctl restart apache2

HTTP to HTTPS redirect:

# Add to port 80 virtual host:
<VirtualHost *:80>
    ServerName example.com
    ServerAlias www.example.com

    Redirect permanent / https://example.com/
</VirtualHost>

Apache Access Control

Host-based access control:

# Allow from specific IP/network
<Directory /var/www/html/admin>
    Require ip 192.168.1.0/24
    Require ip 10.0.0.100
</Directory>

# Deny from specific IP
<Directory /var/www/html>
    <RequireAll>
        Require all granted
        Require not ip 192.168.1.50
    </RequireAll>
</Directory>

# Allow from hostname
<Directory /var/www/html>
    Require host example.com
    Require host .example.com
</Directory>

MySQL/MariaDB Configuration

MySQL Architecture

Installing and securing MySQL:

# Install MySQL server (MySQL documentation: https://dev.mysql.com/doc/)
sudo apt install mysql-server

# Check status
systemctl status mysql

# Secure installation
sudo mysql_secure_installation

# Prompts:
# - Set root password: Yes
# - Remove anonymous users: Yes
# - Disallow root login remotely: Yes
# - Remove test database: Yes
# - Reload privilege tables: Yes

MySQL user management:

# Connect to MySQL as root
sudo mysql -u root -p

# Create database
CREATE DATABASE myapp;

# Create user (local only)
CREATE USER 'appuser'@'localhost' IDENTIFIED BY 'StrongPassword123!';

# Create user (remote access)
CREATE USER 'appuser'@'192.168.1.%' IDENTIFIED BY 'StrongPassword123!';

# Grant privileges
GRANT ALL PRIVILEGES ON myapp.* TO 'appuser'@'localhost';
GRANT SELECT, INSERT, UPDATE ON myapp.* TO 'readonly'@'%';

# Apply changes
FLUSH PRIVILEGES;

# Show grants
SHOW GRANTS FOR 'appuser'@'localhost';

# Remove user
DROP USER 'appuser'@'localhost';

# Change password
ALTER USER 'appuser'@'localhost' IDENTIFIED BY 'NewPassword456!';

MySQL configuration:

# Main config file
sudo vim /etc/mysql/my.cnf
# OR
sudo vim /etc/mysql/mysql.conf.d/mysqld.cnf

# Key settings:
[mysqld]
bind-address = 127.0.0.1        # Listen on localhost only
# bind-address = 0.0.0.0         # Listen on all interfaces

port = 3306                      # Default port

max_connections = 150            # Max concurrent connections
max_allowed_packet = 64M         # Max packet size

# Performance tuning
innodb_buffer_pool_size = 1G     # InnoDB cache (set to 70% of RAM)
innodb_log_file_size = 256M      # Transaction log size

# Logging
log_error = /var/log/mysql/error.log
slow_query_log = 1
slow_query_log_file = /var/log/mysql/slow.log
long_query_time = 2              # Log queries > 2 seconds

# Restart after changes
sudo systemctl restart mysql

Time Synchronization with Chrony

NTP Client Configuration

# Install chrony
sudo apt install chrony

# Configuration file
sudo vim /etc/chrony/chrony.conf

# Configuration:
# Use public NTP pool
pool pool.ntp.org iburst

# Or specific servers
server time.google.com iburst
server time.cloudflare.com iburst

# Allow local network to use this as NTP server
allow 192.168.1.0/24

# Restart chrony
sudo systemctl restart chrony

# Check synchronization status
chronyc tracking
chronyc sources -v

# Force synchronization
sudo chronyc makestep

# System time management
timedatectl
sudo timedatectl set-timezone America/New_York

# List timezones
timedatectl list-timezones | grep New_York

Remote Logging with Rsyslog

Rsyslog Architecture

Configure rsyslog server (receives logs):

# Edit rsyslog configuration
sudo vim /etc/rsyslog.conf

# Enable UDP reception (port 514)
module(load="imudp")
input(type="imudp" port="514")

# Enable TCP reception (more reliable)
module(load="imtcp")
input(type="imtcp" port="514")

# Store remote logs by hostname
$template RemoteLogs,"/var/log/remote/%HOSTNAME%/%PROGRAMNAME%.log"
*.* ?RemoteLogs
& stop

# Create log directory
sudo mkdir -p /var/log/remote

# Open firewall
sudo ufw allow 514/udp
sudo ufw allow 514/tcp

# Restart rsyslog
sudo systemctl restart rsyslog

Configure rsyslog client (sends logs):

# Edit rsyslog configuration
sudo vim /etc/rsyslog.conf

# Send all logs to remote server (UDP)
*.* @192.168.1.100:514

# Send all logs to remote server (TCP, more reliable)
*.* @@192.168.1.100:514

# Send specific facility/priority
auth,authpriv.* @@192.168.1.100:514
*.err @@192.168.1.100:514

# Restart rsyslog
sudo systemctl restart rsyslog

# Test logging
logger -t test "Test message from client"

Production Best Practices

SSH Security:
- Disable root login
- Use key-based authentication only
- Change default port (security through obscurity)
- Implement fail2ban for brute-force protection
- Use SSH certificate authorities for large deployments
Apache/Web Server:
- Always use HTTPS with valid certificates (Let's Encrypt)
- Disable directory listing
- Set proper file permissions (644 files, 755 directories)
- Enable security headers (HSTS, CSP, X-Frame-Options)
- Regular security updates
MySQL/Database:
- Never use root account for applications
- Use least privilege principle
- Enable SSL for remote connections
- Regular backups with mysqldump
- Monitor slow queries
Service Management:
- Use systemd for all services
- Enable services to start on boot
- Monitor service status with monitoring tools
- Implement health checks
- Centralize logging
Time Synchronization:
- Always sync time across all servers
- Use local NTP server for internal networks
- Monitor time drift

Troubleshooting Common Issues

SSH connection refused:

# Check if SSH is running
systemctl status sshd

# Check SSH port
ss -tlnp | grep sshd

# Check firewall
sudo ufw status | grep 22

# Check logs
sudo tail -f /var/log/auth.log

Apache not starting:

# Check configuration
sudo apache2ctl configtest

# Check error logs
sudo tail -f /var/log/apache2/error.log

# Check if port is already in use
sudo ss -tlnp | grep :80

MySQL access denied:

# Reset root password (if forgotten)
sudo systemctl stop mysql
sudo mysqld_safe --skip-grant-tables &
mysql -u root
UPDATE mysql.user SET authentication_string=PASSWORD('newpass') WHERE User='root';
FLUSH PRIVILEGES;
exit
sudo systemctl restart mysql

Frequently Asked Questions

Q: What is systemd and how does it work?

Systemd is the modern Linux init system that manages services, processes, and system startup. It uses unit files to define services, targets for grouping, and dependency management for ordered startup. Commands like "systemctl start/stop/enable service" control services. Systemd replaced SysVinit, offering parallel startup, on-demand activation, and better logging integration.

Q: How do I create a systemd service?

Create a unit file in /etc/systemd/system/myservice.service with [Unit], [Service], and [Install] sections. Define ExecStart for the command, Type for service behavior (simple, forking, oneshot), and WantedBy for dependencies. Run "systemctl daemon-reload" to load, "systemctl enable" for boot startup, and "systemctl start" to run.

Q: What is SSH key authentication and why use it?

SSH key authentication uses public-private key pairs instead of passwords for secure, automated logins. Generate keys with "ssh-keygen", copy public key to server with "ssh-copy-id". Benefits include stronger security, no password transmission, and automated deployments. Disable password authentication in /etc/ssh/sshd_config for maximum security.

Q: How do Apache virtual hosts work?

Apache virtual hosts enable hosting multiple websites on one server using different domains or IPs. Name-based virtual hosts use ServerName directive to match hostnames. Configure in /etc/apache2/sites-available/, enable with "a2ensite", and reload Apache. Each virtual host has separate DocumentRoot, logs, and configuration.

Q: How do I secure MySQL database?

Secure MySQL by running "mysql_secure_installation" to remove anonymous users, disable remote root, and remove test databases. Create users with specific privileges: "GRANT SELECT ON db.* TO 'user'@'localhost'". Use strong passwords, bind to localhost only, enable SSL, and regularly update. Monitor logs for unauthorized access attempts.

Q: What is the difference between systemctl and service commands?

Systemctl is the native systemd command with full control: "systemctl start nginx". Service is a compatibility wrapper supporting both systemd and SysVinit: "service nginx start". Systemctl offers more features like enable/disable, list-dependencies, and show. Use systemctl on modern systems for complete functionality.

Q: How does rsyslog centralized logging work?

Rsyslog collects logs from multiple servers to a central server. Configure clients to forward logs with ". @@logserver:514" in /etc/rsyslog.conf. Server receives with imudp or imtcp input modules. Benefits include centralized search, retention, security analysis, and compliance. Use with ELK stack for advanced analytics and visualization.

Conclusion

Linux service configuration encompasses systemd management, SSH security, web servers, databases, and logging. Understanding service targets, SSH key authentication, Apache virtual hosts, MySQL user management, and centralized logging is essential for production systems.

Choose appropriate configurations: systemd for service management, key-based SSH for security, Apache virtual hosts for multiple sites, MySQL with least privilege, and rsyslog for centralized logging. Implement security best practices, monitor services, and maintain comprehensive documentation.