Skip to content

Commit

Permalink
added readme and short install manual
Browse files Browse the repository at this point in the history
  • Loading branch information
K-cermak committed Feb 7, 2024
1 parent bc294bc commit 2953ce8
Show file tree
Hide file tree
Showing 8 changed files with 177 additions and 19 deletions.
3 changes: 2 additions & 1 deletion .gitIgnore
Original file line number Diff line number Diff line change
@@ -1,2 +1,3 @@
credentials-my.php
db.sqlite
db.sqlite
images
79 changes: 79 additions & 0 deletions INSTALL.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,79 @@

---

# WARNING ⚠️⚠️
- In this guide, you are working with the Proxmox installation, in case of an error, data loss can occur. The author of this guide bears no responsibility for data loss and other problems caused.
- Be aware that by installing FTP servers you can cause system vulnerability if you do not follow security rules. Set very strong passwords and make sure that FTP is not accessible from the internet.

---

<br>

## Creating a backup drive
- Go to Proxmox VE.
- In the `Datacenter`, select `Storage` and choose `Add` -> `Directory`.
- Set `ID` to `backup` and `Directory` to the path where backups will be stored. For example `/media/drive/`.
- In `Content`, select `Disk Image` and `VZDump Backup File` and click `OK`.


<br>

## Creating a backup
- Go to the `Backup` tab, click on `Add`.
- For `Storage`, select the name of your disk (e.g., `backup`) and choose the backup time and machines to be backed up.
- In `Retention` tab, set `Keep Last` to the desired number of backups that will be kept on Proxmox.

<br>

## FTP Installation

- Připojte se k Proxmoxu (bare metal) přes SSH či přes službu v Proxmoxu.
- Aktualizujte seznam balíčků: apt-get update
- Nainstalujte vsftpd: apt-get install vsftpd
- Přejděte do konfiguračního souboru: nano /etc/vsftpd.conf. Místo nano můžete použít jiný textový editor.
- Nastavte konfiguraci podle následujícího příkladu. Pokud řádek neexistuje, přidejte jej. Pokud existuje, upravte jej.


- Connect to Proxmox (bare metal) via SSH or through the service in Proxmox.
- Update the package list: `apt-get update`
- Install vsftpd: `apt-get install vsftpd`
- Go to the configuration file: `nano /etc/vsftpd.conf`. Instead of nano, you can use another text editor ().
- Set the configuration according to the following example. If the line does not exist, add it. If it exists, uncomment it and modify it if needed.

```
anonymous_enable=NO
local_enable=YES
write_enable=NO
chroot_local_user=YES
allow_writeable_chroot=YES
local_root=/media/drive/dump ; upravte cestu podle Vašeho nastavení a přidejte /dump na konec
user_sub_token=$USER
```

- Create a new user who will have access to backups: `useradd -d /media/drive/dump ftpuser`. Instead of `ftpuser`, you can use a different name.
- Set a password for the new user: `passwd ftpuser`. Use a strong password.
- Restart vsftpd: `service vsftpd restart`.
- Test the connection to the FTP server using an FTP client (e.g., Total Commander) and the new user credentials.

<br>

## PHP Installation
- Create a new virtual machine that will take care of backup synchronization.
- Update the package list: `sudo apt-get update`.
- Install PHP: `sudo apt-get install php-cli`.
- Install the library for sqlite: `sudo apt-get install php-sqlite3`.

- Create a folder and place the files of this repository in it.
- You can learn how to setup and use this tool in the file [`USAGE.md`](USAGE.md).

<br>

# Configuration of Cron

- Install cron if you don’t have it: `sudo apt-get install cron`.
- Enter the command: `crontab -e`.
- Set cron to run after backup (for example, if Proxmox backup takes place every Sunday at 3:00, set cron on Sunday at 6:00). Example of cron:
```
0 6 * * 0 php /var/www/proxmox-ftp-syncer/syncer.php autorun
```
- Save the file and exit the editor.
Empty file removed Instructions.md
Empty file.
3 changes: 2 additions & 1 deletion License
Original file line number Diff line number Diff line change
Expand Up @@ -20,4 +20,5 @@ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS ORIMPLIE
<br>

### Sources used:
- PHPMailer (https://github.com/PHPMailer/PHPMailer)
- PHPMailer (https://github.com/PHPMailer/PHPMailer)
- Icons by FlatIcon (https://www.flaticon.com/)
59 changes: 43 additions & 16 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,25 +1,52 @@
-- jak vytvořit v proxmoxu složku
# Proxmox FTP Syncer

-- tento návod se vztahuje pouze pokud nepoužíváte k připojení k Proxmoxu FTP
By Karel Cermak | [Karlosoft](https://karlosoft.com).

sudo apt-get install vsftpd
<img src="https://cdn.karlosoft.com/cdn-data/ks/img/proxsync/github.png" width="700" alt="Proxmox FTP Syncer">

sudo nano /etc/vsftpd.conf
<br>

anonymous_enable=NO
local_enable=YES
write_enable=NO
chroot_local_user=YES
allow_writeable_chroot=YES
local_root=/data/backup/dump ; edit this line
user_sub_token=$USER
## What is Proxmox FTP Syncer?
- Proxmox is pretty good at backing up your VMs, but it's not so good at syncing those backups to a remote location without using Proxmox Backup Server. This script is a simple solution to that problem. <b>It uses FTP to sync the backups to a remote server</b>.
- With this CLI tool, you can easily and <b>automatically sync your Proxmox backups to a remote server using FTP</b> (to your NAS, AWS S3, etc.).

<br>

## How it works?
- You will create a new folder on your Proxmox server (bare metal) and set this folder as the backup location for your virtual servers. You will set it up so that old backups are regularly deleted, for example after 5 backups. In the meantime, this tool will synchronize these backups to a remote server using FTP.

-- user
sudo useradd -d /media/hdd/backups ftpuser
sudo passwd ftpuser
<br>

## What can it do?
- Move backups from Proxmox to a remote server using FTP.
- Automatically remove old backups from FTP.
- Email notifications for successful and unsuccessful backups.
- Automatically delay the removal of old backups when new backups are unavailable (which may indicate a problem).
- Simple command line control and easy connection to CRON.

-- finish
sudo service vsftpd restart
<br>

## How to start?
- You can find the installation in this file: [`INSTALL.md`](INSTALL.md).
- You can find the help in this file: [`USAGE.md`](USAGE.md). (or use the command `php syncer.php help`).

<br>

## When is it not good to use this tool?
- You are using Proxmox Backup Server.
- You make backups very often (hourly). This tool is more for backups made at daily or weekly intervals.
- Your backups are really large (several hundred GB). The tool is currently not written for multi-core processing, so if you have large backups, it may take a long time.
- This tool cannot copy data directly between Proxmox and FTP and must store data on its disk. Therefore, it is more suitable to use this tool in a VE that runs on HDD and has enough space (the more the better, for example I use 120 GB drive). Using on an SSD disk can significantly shorten its lifespan.

<br>

## Can I also use this utility to transfer backups of something else?
- Yes, you can. The tool is not limited to Proxmox backups only. You can use it to transfer any files from one server to another using FTP (WordPress backups, etc.).
- Ot is important that what creates the backups is able to delete the backups in the original storage (this utility cannot do that). It should also never create two files with the same name (i.e. the file name should have a timestamp or random ID in it).

<br>
<br>

---

#### This project is not affiliated with Proxmox Server Solutions GmbH. This is just a simple extension for backing up Proxmox VE to FTP, which is not possible by default. Developed by Karel Cermak (info@karlosoft.com).
46 changes: 46 additions & 0 deletions USAGE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
## Basic settings

- Go to the `settings.php file` and set the following variables:

- `CREDENTIALS_FILE` - Path to the file with login credentials.
- `DB_FILE` - Path to the database file. Do not create a database, just enter the absolute path to the file with the .sqlite extension.
- `KEEP_FILES_FOR` - Enter the number of days for which backups will be kept on the FTP server. When set to 0, backups will not be deleted.
- `EXTEND_BACKUP_ON_ERROR` - Enter the number of days by which the retention of backups on the FTP server will be extended if a new backup created by Proxmox is not found. When set to 0, backups will not be extended.
- `SEND_EMAIL` - The value `always` will send an informational email after each backup. The value `on_error` will send an email only in case of an error. The value `no` will not send an email at all.


<br>

## FTP Settings
- Go to the file with your keys (the same as the set variable `CREDENTIALS_FILE` above).
- Enter the login details for the FTP servers. The ORIGIN server is the server from which backups are downloaded (Proxmox), the DESTINATION server is the server (for example, NAS, AWS S3) where backups are stored.

<br>

## Email Settings
- Go to the file with your keys (the same as the set variable `CREDENTIALS_FILE` above).
- Enter the SMTP details for the email server. Your email provider should provide these.
- In `SEND_TO`, enter the email addresses to which informational emails will be sent. If you want to send an email to multiple addresses, separate them with a comma without a space.

<br>

## Database Initialization
- Run the command `php syncer.php create-db`.

<br>

## Verification of Settings
- Run the command `php syncer.php check-connection` to check the connection to the FTP servers.
- Run the command `php syncer.php check-settings` to check the set variables.
- Run the command `php syncer.php check-email` to send a test email.

<br>

## Run the Synchronization
- You can manually start the backup with the command `php syncer.php autorun`.
- You can also set up a CRON job to run the command `php syncer.php autorun` at a specific time (more in the file [`INSTALL.md`](INSTALL.md)).

<br>

## Help
- You can display all available commands using the command `php syncer.php help`.
4 changes: 4 additions & 0 deletions helpers/help.php
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,10 @@ function getHelp() {
boldMessage(" check-connection", false);
echo " - check the connection to the FTP servers\n";

//check email
boldMessage(" check-email", false);
echo " - send test email\n";

//check settings
boldMessage(" check-settings", false);
echo " - check the settings\n";
Expand Down
2 changes: 1 addition & 1 deletion settings.php
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
<?php
define("CREDENTIALS_FILE", "credentials-my.php");
define("CREDENTIALS_FILE", "credentials.php");
define("DB_FILE", "/crons/db.sqlite"); //should be absolute path

define("KEEP_FILES_FOR", 90); //after sync files will be kept for 90 days, 0 will keep files forever
Expand Down

0 comments on commit 2953ce8

Please sign in to comment.