Guide to Linux Server Backup Using Rclone and Object Storage

In this comprehensive guide, we will delve into the process of creating both manual and scheduled backups of any Linux server to Object Storage using rclone. This guide caters to a wide range of Linux servers, including Dedicated Servers, VPS, and VDS. The provided commands have been thoroughly tested with Ubuntu 18.04(64 Bit), Ubuntu 20.04(64 Bit), Debian 9, and Debian 10. While these commands are primarily tailored for these distributions, slight adjustments may be required for other distributions like CentOS.

Throughout this guide, it is assumed that you have root access to the Linux server. To elevate your privileges, enter the following command:

sudo -i

Initial Linux Server Backup Configuration

1. Acquiring Object Storage Credentials for Backup Storage

In this tutorial, we will utilize Object Storage as our backup destination. We assume that you already have an active Object Storage subscription.

To proceed, you will need to gather the necessary credentials, including the access_key, secret_key, and URL. These credentials can be obtained from the Object Storage section of your server’s Control Panel (CCP).

2. Creating a Backup File on the Linux Server

Before proceeding with the backup process, we need to install the latest version of rclone using the following command:

curl https://rclone.org/install.sh | sudo bash

Next, we will create a backup of the current user and store the backup file with the current date in the /opt/backup/ directory. Execute the following command to achieve this:

mkdir /opt/backup/ cd /opt/backup && tar -cvzf backup-$(date +%d.%m.%Y).tar.gz --directory=/ --exclude=lost+found --exclude=dev/* --exclude=proc/* --exclude=run/* --exclude=sys/* --exclude=tmp/* --exclude=mnt/* --exclude=media/* --exclude=opt/backup/* .

Here’s a breakdown of the command:

• tar: The tar command is used to create an archive of our data.

• c: This option stands for create and indicates that we are creating a new archive.

• v: This option represents verbose and provides detailed output during the process.

• z: This option signifies gzip, implying that the archive will be compressed using gzip.

• f: This option represents file and specifies the name and location of the archive. It must be the last option passed, as everything following f is treated as a file path.

• /path/filename.tar.gz: This indicates the path where the archive will be stored.

In this command, certain directories have been excluded from the backup process due to storage space considerations or the presence of unwanted large files in the tmp directory.

3. Configuring Rclone on a Linux Server

For this step, we assume that the files and folders you intend to back up are stored in the /opt/backup/ directory.

If you haven’t already, follow the instructions in our product documentation to set up Rclone. Once Rclone is configured, we can proceed with the Linux server backup using Rclone.

4. Transferring Your Backup File from Linux Server to Object Storage

In this step, we will transfer the backup file created in step 2 to the Object Storage. Assuming you have named your object store eu2 and your backup folder backup, execute the following command to create a bucket where we will store all the files:

rclone mkdir eu2:backup

After completing the steps in the “Initial Configuration” section, run the following command to save the files to your Object Storage:

rclone copy -P /opt/backup eu2:backup/server1

This command will transfer all data from /opt/backup to the specified bucket.

Here are the available options for transferring data:

• rclone copy -P /opt/backup eu2:bucketname/server1: This command copies all data without deleting the source files.

• rclone move -P /opt/backup eu2:bucketname/server1: This command moves all data, deleting the source files after the transfer is complete.

• rclone sync -P /opt/backup eu2:bucketname/server1: This command synchronizes the data, ensuring that the files on the server and in the bucket are identical.

5. Scheduling Rclone Backups

   After completing the initial configuration steps, you can set up scheduled backups to run automatically. To achieve this, follow these steps:

   • Create a script file: Start by creating a script file using the following command:

     sudo nano /var/rclone.sh
     

   • Add the script contents: Open the script file in the nano editor and add the following contents:

     #!/bin/bash
     
     # Navigate to the backup directory
     cd /opt/backup
     
     # Create a backup archive with the current date
     tar -cvzf backup-$(date +%d.%m.%Y).tar.gz --directory=/ --exclude=lost+found --exclude=dev/* --exclude=proc/* --exclude=run/* --exclude=sys/* --exclude=tmp/* --exclude=mnt/* --exclude=media/* --exclude=opt/backup/* .
     
     # Sync the backup files to the S3 storage
     /usr/bin/rclone sync -P --update --verbose --transfers 30 --log-file=/var/log/upload.log "/opt/backup" "eu2:backup/"
     

   • Make the script executable: To ensure the script can be executed, use the following command:

     sudo chmod +x /var/rclone.sh
     

   • Add the script to cron: To schedule the script to run at a specific time, open the crontab file using the following command:

     sudo crontab -e
     

   • Add the cron job: At the end of the crontab file, add the following line to schedule the backup script to run daily at 02:00:

     0 2 * * * /var/rclone.sh
     

   • Save the crontab file: Save and close the crontab file.

   With these steps completed, the script will now run automatically every day at 02:00, creating a backup archive and transferring it to the Object Storage. You can modify the cron job to run at a different time or frequency as per your requirements.

   Additional Information

   • For more detailed information on rclone flags and configuration options, refer to the official documentation.

   • In some cases, if your upload speed is slow, you may need to add the -s3-chunk-size 200M flag to the script.

   By following these comprehensive instructions, you can effectively back up your Linux servers using rclone and Object Storage, ensuring the safety and security of your valuable data. Remember to regularly review and update your backup strategy to adapt to changing data needs and storage requirements.

   Conclusion

   By following the detailed instructions provided in this comprehensive guide, you can effectively implement a robust backup strategy for your Linux servers using rclone and Object Storage. This guide covers the entire process, from initial configuration to scheduling automated backups, ensuring that your valuable data remains safe and secure. Remember to regularly review and update your backup strategy to adapt to evolving data needs and storage requirements.