How to Install an FTP over SSL (FTPS) on Ubuntu Server 18.04

Post Reply
User avatar
LHammonds
Site Admin
Site Admin
Posts: 779
Joined: Fri Jul 31, 2009 6:27 pm
Are you a filthy spam bot?: No
Location: Behind You
Contact:

How to Install an FTP over SSL (FTPS) on Ubuntu Server 18.04

Post: # 731Post LHammonds
Tue Sep 10, 2019 3:23 pm

Greetings and salutations,

I hope this thread will be helpful to those who follow in my foot steps as well as getting any advice based on what I have done / documented.

To discuss this thread, please participate here: Ubuntu Forums

High-level overview

This document will describe how to setup an FTP server which utilizes SSL (FTPS) encryption and local login IDs to allow users to login and upload files but cannot login to the console.

Tools utilized in this process
Helpful links

The list below are sources of information that helped me configure this system.
Assumptions

This documentation will need to make use of some very-specific information that will most-likely be different for each person / location. This variable data will be noted in this section and highlighted in red throughout the document as a reminder that you should plug-in your own value rather than actually using these "place-holder" values.

Under no circumstance should you use the actual values listed below. They are place-holders for the real thing. This is just a checklist template you need to have answered before you start the install process.

Wherever you see RED in this document, you need to substitute it for what your company uses.
  • Minimum Passive Port: 9000
  • Maximum Passive Port: 9020
  • FTP User Account: myftp
  • FTP User Password: mypass123

User avatar
LHammonds
Site Admin
Site Admin
Posts: 779
Joined: Fri Jul 31, 2009 6:27 pm
Are you a filthy spam bot?: No
Location: Behind You
Contact:

Install Ubuntu Server

Post: # 732Post LHammonds
Tue Sep 10, 2019 3:27 pm

Install Ubuntu Server

The 1st thing to do is the installation of Ubuntu Server. The following guide will describe how to setup a server for use in a production environment and this document assumes the guide was followed to setup Ubuntu.

The server install guide also contains an "Assumption" section with variables colored in RED, instead of using those values, this guide will instead use the following as replacements:

Ubuntu Server name: srv-ftps
Ubuntu Server IP address: 192.168.107.28

IMPORTANT: Keep in mind, these are typically different for each site/install. Do not use these exact values yourself...make up your own that matches your environment, write them down and use them instead of the values in the guide.

NOTE: These are the sections you can skip in the server install guide which are not necessary for this server (however, you might want them...it is up to you):
  • Configure Ubuntu for File Sharing (Samba)
  • Configure Windows Server as a Remote Mount Point
  • SSH Public and Private Keys
Follow the How to Install Ubuntu Server guide and come back when finished.

User avatar
LHammonds
Site Admin
Site Admin
Posts: 779
Joined: Fri Jul 31, 2009 6:27 pm
Are you a filthy spam bot?: No
Location: Behind You
Contact:

Install FTP Service

Post: # 733Post LHammonds
Tue Sep 10, 2019 3:29 pm

Install FTP Service

At the login prompt, login with your administrator account (administrator / myadminpass)

At the $ prompt, temporarily grant yourself super user privileges by typing sudo su {ENTER} and then provide the administrator password (myadminpass).

Install vsftpd and make a backup of the original config file.

Code: Select all

apt -y install vsftpd
cp /etc/vsftpd.conf /etc/vsftpd.bak

Generate an SSL Certificate

Run these commands but make sure you substitute your values in place of the "My" values.

Code: Select all

openssl req -x509 -nodes -days 1095 -newkey rsa:2048 -text -subj "/C=MyCountry/ST=MyState/L=MyCity/O=MyOrganization/OU=MyOrganizationalUnit/CN=MyCommonName" -keyout /etc/ssl/private/vsftpd.key -out /etc/ssl/private/vsftpd.pem
chown root:ftp /etc/ssl/private/vsftpd.*
chmod 644 /etc/ssl/private/vsftpd.*

Configure vsftpd

Edit /etc/vsftpd.conf and find/uncomment following lines:

Code: Select all

local_enable=YES
write_enable=YES
local_umask=022
ftpd_banner=Welcome to our FTP server.
chroot_local_user=YES
Find the following existing lines and change the values as follows:

Code: Select all

anonymous_enable=NO
connect_from_port_20=NO
rsa_cert_file=/etc/ssl/private/vsftpd.pem
rsa_private_key_file=/etc/ssl/private/vsftpd.key
Add the following new lines (feel free to adjust values as desired):

Code: Select all

listen_port=990

# Turn on SSL
ssl_enable=YES

# Allow anonymous users to use secured SSL connections
allow_anon_ssl=NO

# All non-anonymous logins are forced to use a secure SSL connection in order to
# send and receive data on data connections.
force_local_data_ssl=YES

# All non-anonymous logins are forced to use a secure SSL connection in order to send the password.
force_local_logins_ssl=YES

# Permit TLS v1 protocol connections. TLS v1 connections are preferred
ssl_tlsv1=YES

# Permit SSL v2 protocol connections. TLS v1 connections are preferred
ssl_sslv2=YES

# permit SSL v3 protocol connections. TLS v1 connections are preferred
ssl_sslv3=YES

# Hide the info about the owner (user and group) of the files.
hide_ids=YES

# Connection limit for each IP:
max_per_ip=50

# Maximum number of clients:
max_clients=50

# When port_enabled is YES, active mode connects are allowed.
port_enable=YES

# When pasv_enable is YES, passive mode connects are allowed.
pasv_enable=YES

# pasv_min_port specifies the lowest possible port sent to the FTP clients for passive mode connections. This setting is used to limit the port range so that firewall rules are easier to create. The value must not be lower than 1024.
pasv_min_port=9000

# pasv_max_port specifies the highest possible port sent to the FTP clients for passive mode connections. This setting is used to limit the port range so that firewall rules are easier to create. The value must not exceed 65535.
pasv_max_port=9020

# require_ssl_reuse, if YES, all SSL data connections are required to exhibit SSL session reuse.  Set to NO if your log shows failures to upload because of no session reuse.

require_ssl_reuse=NO

# Set to YES to get extra information in the logs if you are having issues connecting.
debug_ssl=NO

# This setting allows FileZilla to connect or you would see something like "handshake failed" otherwise.
ssl_ciphers=HIGH
Restart FTP Service

The the following:

Code: Select all

systemctl restart vsftpd

User avatar
LHammonds
Site Admin
Site Admin
Posts: 779
Joined: Fri Jul 31, 2009 6:27 pm
Are you a filthy spam bot?: No
Location: Behind You
Contact:

Firewall Rules

Post: # 734Post LHammonds
Tue Sep 10, 2019 3:31 pm

Firewall Rules

Edit the firewall script that was created during the initial setup of the server (if you followed my instructions):

Code: Select all

vi /var/scripts/prod/en-firewall.sh
Add (or enable) the following:

Code: Select all

echo "Adding FTPS Server rules"
ufw allow proto tcp to any port 990 comment 'FTPS' 1>/dev/null 2>&1
ufw allow proto tcp to any port 9000:9020 comment 'FTP Passive' 1>/dev/null 2>&1
Run the updated rules:

Code: Select all

/var/scripts/prod/en-firewall.sh
Restart Scripts

Edit the service start script to include vsftpd service. (if you followed my instructions)

Code: Select all

vi /var/scripts/prod/servicestart.sh
Add or enable the following:

Code: Select all

systemctl start vsftpd
Edit the service stop scripts to include vsftpd service. (if you followed my instructions)

Code: Select all

vi /var/scripts/prod/servicestop.sh
Add or enable the following:

Code: Select all

systemctl stop vsftpd

User avatar
LHammonds
Site Admin
Site Admin
Posts: 779
Joined: Fri Jul 31, 2009 6:27 pm
Are you a filthy spam bot?: No
Location: Behind You
Contact:

Configure User Accounts

Post: # 735Post LHammonds
Tue Sep 10, 2019 3:31 pm

Configure User Accounts

At the login prompt, login with your administrator account (administrator / myadminpass)

At the $ prompt, temporarily grant yourself super user privileges by typing sudo su {ENTER} and then provide the administrator password (myadminpass).

Add an ftpusers group.

Code: Select all

addgroup ftpusers
Use the default ftp folder of /srv/ftp but move the home directory of the default "ftp" account underneath that folder. The /srv/ftp folder will be become the base folder for all FTP user accounts but nobody will be able to access it directly.

Code: Select all

mkdir -p /srv/ftp/ftp/public
usermod -d /srv/ftp/ftp ftp
chmod 555 /srv/ftp
chmod 555 /srv/ftp/ftp
chmod 755 /srv/ftp/ftp/public
chown -R ftp:ftpusers /srv/ftp/ftp
Type this to add /usr/sbin/nologin to /etc/shells (* This allows users to login via FTP which cannot login directly on the server *)

Code: Select all

echo "/usr/sbin/nologin" >> /etc/shells
Now add a user called myftp with a password of mypass123
mkdir -p /srv/ftp/myftp/public chmod 555 /srv/ftp/myftp chmod 755 /srv/ftp/myftp/public useradd -g ftpusers -d /srv/ftp/myftp -s /usr/sbin/nologin myftp chown -R myftp:ftpusers /srv/ftp/myftp passwd myftp mypass123

User avatar
LHammonds
Site Admin
Site Admin
Posts: 779
Joined: Fri Jul 31, 2009 6:27 pm
Are you a filthy spam bot?: No
Location: Behind You
Contact:

Public / Private Key Management

Post: # 736Post LHammonds
Tue Sep 10, 2019 3:32 pm

Public / Private Key Management

If you need to generate an SSH key with a passphrase in order to connect to another FTP server, it can be a tricky thing to automate scripts that push and pull files to that server since each time you connect, it will ask for the passphrase.

I've seen some recommend to avoid passphrases when scripting or include the passphrase in the script. Either of these methods reduces the security by not having the key protected or by exposing the passphrase by having it on the file system.

My preferred solution is to use the passphrase but also not store it on the server anywhere so if the keys are ever compromised, the passphrases do not go with them.

An SSH agent is used to load the keys in memory and as long as the agent is loaded, the scripts will be able to work automatically in the background.

This still requires manually typing in the passphrase(s) but only after the server reboots.

Here is how it works:

ssh-add is used to load the various private keys you have into memory. This loads a program called "ssh-agent" in memory. In order for SSH programs to access these keys, two variables have to be set. SSH_AUTH_SOCK and SSH_AGENT_PID and they have to be pointed to the lock file and process ID respectfully.

Let's assume you have two passphrase-protected private keys stored here:

Code: Select all

/root/.ssh/ssh2rsa-company1-private.rsa
/root/.ssh/ssh2rsa-company2-private.rsa
When you reboot your server, you will need to manually run a script like the following which will initially load the ssh-agent and the keys:

/var/scripts/prod/load-ssh-keys.sh

Code: Select all

#!/bin/bash
#############################################
## Name          : load-ssk-keys.sh
## Version       : 1.0
## Date          : 2017-02-16
## Author        : LHammonds
## Compatibility : Ubuntu Server 16.04 LTS - 18.04.3 LTS
## Requirements  : Run as root
## Purpose       : Load SSH keys into memory which will
##                 allow FTP login without prompts.
## Run Frequency : As needed
## Exit Codes    : None
################ CHANGE LOG #################
## DATE       WHO  WHAT WAS CHANGED
## ---------- ---- ----------------------------
## 2017-02-16 LTH  Created script.
#############################################

## Requirement Check: Script must run as root user.
if [ "$(id -u)" != "0" ]; then
  ## FATAL ERROR DETECTED: Document problem and terminate script.
  echo -e "\nERROR: Root user required to run this script.\n"
  echo -e "Type 'sudo su' to temporarily become root user.\n"
  exit
fi
clear
ssh-add -l >/dev/null
if [ "$?" == 2 ]; then
  ## Agent not loaded, load it up.
  eval `ssh-agent -s`
fi
echo "Purging any existing keys from memory..."
ssh-add -D
echo "Enter the Company #1 SSH passphrase..."
ssh-add /root/.ssh/ssh2rsa-company1-private.rsa
echo "Enter the Company #2 SSH passphrase..."
ssh-add /root/.ssh/ssh2rsa-company1-private.rsa
echo "Listing all keys in memory..."
ssh-add -l
echo "Complete. Keys will remain in memory until the next reboot."
echo ""
read -p "Press [Enter] key to continue..."
If you logout or if a script runs in a different session (which will likely be the case), it will not be able to "see" the keys loaded in memory because the SSH_AUTH_SOCK and SSH_AGENT_PID variables will not be set in a new session.

To fix this, your scripts need to "find" the currently loaded SSH agent and set the variables correctly.

Example:

Code: Select all

## Point the required variables to the loaded ssh-agent
export SSH_AUTH_SOCK=`find /tmp/ -type s -name agent.\* 2> /dev/null | grep '/tmp/ssh-.*/agent.*'`
export SSH_AGENT_PID=`ps aux | grep ssh-agent | grep -v "grep" | awk '{print $2}'`

if [ ${#SSH_AUTH_SOCK} -lt 1 ]; then
  ## Variable still empty.  Houston, we have a problem.
  echo "ERROR - ssh-agent not pre-loaded with authentication key." | tee -a ${LogFile}
  echo "You must load the SSH keys before running this script."
  exit 0
fi
Here is an example of a loaded ssh-agent:

Code: Select all

root@srv-ftps:/# pgrep -u root ssh-agent
4421
root@srv-ftps:/# ls -l /tmp/ssh-jDCLupzf4420/
total 0
srw------- 1 root root 0 Apr  6 18:24 agent.4420
With the above information, the variables will need to be set as follows:

Code: Select all

SSH_AGENT_PID=4421
SSH_AUTH_SOCK=/tmp/ssh-jDCLupzf4420/agent.4420
It might also be a good idea to remind anyone at the console that runs the reboot process to remind them that once the server comes back up, the keys must be loaded in memory.

Example:

/var/scripts/prod/reboot.sh

Code: Select all

#!/bin/bash
#############################################
## Name          : reboot.sh
## Version       : 1.0
## Date          : 2013-01-08
## Author        : LHammonds
## Compatibility : Ubuntu Server 12.04.1 LTS - 18.04.3 LTS
## Requirements  : Run as root
## Purpose       : Stop services and reboot server.
## Run Frequency : As needed
## Exit Codes    : None
################ CHANGE LOG #################
## DATE       WHO  WHAT WAS CHANGED
## ---------- ---- ----------------------------
## 2013-01-07 LTH  Created script.
#############################################

## Requirement Check: Script must run as root user.
if [ "$(id -u)" != "0" ]; then
  ## FATAL ERROR DETECTED: Document problem and terminate script.
  echo -e "\nERROR: Root user required to run this script.\n"
  echo -e "Type 'sudo su' to temporarily become root user.\n"
  exit
fi
clear
echo ""
echo "WARNING: Be sure to Load SSH Keys after reboot or scripts will fail"
echo ""
sleep 5
echo "Stopping FTP service..."
service vsftpd stop
echo "Rebooting..."
echo "3"
sleep 1
echo "2"
sleep 1
echo "1"
sleep 1
shutdown -r now
If you also want the variables set into the root profile, type the following (while as root):

Code: Select all

vi /root/.profile
Add the following:

Code: Select all

## Point the required variables to the loaded ssh-agent
export SSH_AUTH_SOCK=`find /tmp/ -type s -name agent.\* 2> /dev/null | grep '/tmp/ssh-.*/agent.*'`
export SSH_AGENT_PID=`ps aux | grep ssh-agent | grep -v "grep" | awk '{print $2}'`
echo "Loaded SSH Keys..."
ssh-add -l
Now when you are in your low-rights account and need to sudo into the root account, type the following:

Code: Select all

sudo su --login
The "--login" is important so it executes the code in .profile. Once you are logged in as root, you should be able to list the keys loaded in memory with:

Code: Select all

ssh-add -l

User avatar
LHammonds
Site Admin
Site Admin
Posts: 779
Joined: Fri Jul 31, 2009 6:27 pm
Are you a filthy spam bot?: No
Location: Behind You
Contact:

Command-Line FTP Clients for Automation

Post: # 737Post LHammonds
Wed Sep 11, 2019 9:27 am

Command-Line FTP Clients for Automation

sftp command comes with OpenSSH and should already be installed on your server.

Example:

Code: Select all

echo "lcd /local/path" > /tmp/ftp.cmd
echo "cd /remote/path" >> /tmp/ftp.cmd
echo "mget *" >> /tmp/ftp.cmd
sftp -b /tmp/ftp.cmd ftpuser@192.168.1.21
NOTE: To use this, you might have to modify /etc/ssh/sshd_config and modify the "AllowUsers" and "PasswordAuthentication" lines.

sshpass - This utility allows non-interactive authentication to a keyboard-interactive site. For example, you might need to access an FTP server that requires a password to be typed in interactively. sshpass is used to automate such systems but it isn't without risk. Rather than exposing the password to anyone that is watching the process list, we can export a variable with the password in it to help hide the password from other users on the system. But the script has to be locked down so nobody but the process running it can see the contents of the script...which contains the password.

You have to install sshpass from the repository using the following command:

Code: Select all

apt install ssh-pass
Example usage:
#!/bin/bash echo "lcd /files/to/send/ > /tmp/ftp.cmd echo "cd /upload/path" >> /tmp/ftp.cmd echo "mput *" >> /tmp/ftp.cmd export SSHPASS='ASuperSecretPassword' sshpass -e sftp -oBatchMode=no -b /tmp/ftp.cmd ftpuser@192.168.1.21
ftp-ssl - You have to install ftp-ssl from the repository using the following command:

Code: Select all

apt install ftp-ssl
lftp - is a file transfer program that allows sophisticated FTP, HTTP and other connections to other hosts. You have to install ftp-ssl from the repository using the following command:

Code: Select all

apt install lftp
SSL-Aware FTP Clients for Windows

SmartFTP

FileZilla

FileZilla Settings (tested with 3.44.2):
Host: ?.?.?.?
Port: 990
Protocol: FTP - File Transfer Protocol
Encryption: Require explicit FTP over TLS
Login Type: Normal

WinSCP

WinSCP Settings (tested with 5.13.4 and 5.15.3)
File protocol: FTP
Encryption: TLS/SSL Explicit Encryption
Host name: ?.?.?.?
Port number: 990

User avatar
LHammonds
Site Admin
Site Admin
Posts: 779
Joined: Fri Jul 31, 2009 6:27 pm
Are you a filthy spam bot?: No
Location: Behind You
Contact:

Fail2Ban

Post: # 738Post LHammonds
Wed Sep 11, 2019 4:43 pm

ON HOLD - NEED TO RESEARCH THIS SECTION

Fail2Ban Standard Filters

If you followed my instructions for setting up the Ubuntu Server, you should already have sshd being protected by Fail2Ban. Now we are going to add the pre-defined vsftpd filter.

Edit the jail configuration file:

Code: Select all

sudo vi /etc/fail2ban/jail.local
Add the following section to the bottom:

Code: Select all

[vsftpd-auth]
# detect password authentication failures
enabled  = true
port     = 990
filter   = vsftpd
action   = iptables-multiport[name=vsftpd, port="990"]
bantime  = 3600
maxretry = 3
Restart the Fail2Ban service:

Code: Select all

sudo systemctl restart fail2ban
Check the status:

Code: Select all

sudo fail2ban-client status

Post Reply