parent
370440e9ef
commit
87a8b1f1d5
@ -0,0 +1,86 @@
|
||||
---
|
||||
title: Using OVH PCA with backupninja
|
||||
description: Accessing the OVH Public cloud archive with the automated backup tool backupninja using its duplicity backend
|
||||
date: 2022-03-03
|
||||
---
|
||||
|
||||
## motivation
|
||||
OVH provides a cheap way for longterm storing backup data, the Public Cloud Archive (PCA). As this is for write-once data (cold data store), it needs to be used together with a little bit more expensive hot data store for meta data when used for sequential backups. The backup solution [duplicity](https://duplicity.gitlab.io/duplicity-web/) is capable of using such multi backends. [backupninja](https://0xacab.org/liberate/backupninja) provides a nice solution for a standardized configuration of duplicity and running it automatically.
|
||||
|
||||
## Prerequisites
|
||||
### On system
|
||||
- Enable email sending for backup status mails by [installing a mta](/msmtp-on-debian/).
|
||||
- Install system dependencies
|
||||
- `apt install librsync-dev gpg backupninja`
|
||||
|
||||
#### GPG
|
||||
- Create a new secret key. Note the password.
|
||||
- `gpg --quick-generate-key cloud@freedomhost.de`
|
||||
- `gpg --export-secret-key keyid > private.key`
|
||||
- Copy the key to the server
|
||||
- `scp private.key server:`
|
||||
- Import it in the local keyring on server
|
||||
- `gpg --allow-secret-key-import --import private.key`
|
||||
- Increase trust level:
|
||||
|
||||
```
|
||||
# gpg --edit-key KEY_ID
|
||||
> trust
|
||||
> 5
|
||||
> quit
|
||||
```
|
||||
|
||||
|
||||
|
||||
### On OVH
|
||||
- Create a new user unter Cloud Archive → Project Management → Users & Roles
|
||||
- Give the user the Role `ObjectStore operator`
|
||||
- Note username and **password**
|
||||
- Get TenandID
|
||||
- click 3 dots on the right of the users row. Click Download OpenStack's RC file. Here you can select a region where the PCA should be set up
|
||||
- Note the `OS_TENANT_ID`, here you also can extract `OS_USERNAME` and `OS_REGION_NAME`
|
||||
|
||||
|
||||
## Installation
|
||||
- Be sure to have at least version 0.8.21 of duplicity
|
||||
- `pip3 install duplicity>=0.8.21 python-swiftclient python-keystoneclient`
|
||||
|
||||
## Configuration
|
||||
- 3 files are used for this:
|
||||
- `/etc/backupninja.conf`: Here you can set the time of day when the backup should be run, and to whom to send status emails. This file is pre-installed and quite self-explaining
|
||||
- `/etc/ovh-config.json` for holding the OVH credentials
|
||||
- `/etc/backup.d/20_ovh_pca.dup` for configuring duplicity for backupninja usage
|
||||
|
||||
|
||||
### `/etc/backup.d/20_ovh_pca.dup`
|
||||
- There is a full example for duplicity configuration in `/usr/share/doc/backupninja/examples/example.dup` find a slightly pre configured version [here](/texts/20_ovh_pca.dup)
|
||||
- most important options are:
|
||||
|
||||
```
|
||||
options = --volsize 200 --archive-dir /tmp --file-prefix-manifest 'hot_' --file-prefix-signature 'hot_' --file-prefix-archive 'cold_'
|
||||
testconnect = no
|
||||
|
||||
[gpg]
|
||||
sign = yes
|
||||
encryptkey = GPGKEY
|
||||
password = GPGKEYPASSWORD
|
||||
|
||||
[source]
|
||||
include = SOMEDIR
|
||||
include = SOMEOTHERDIR
|
||||
|
||||
[dest]
|
||||
desturl = 'multi:///etc/ovh-config.json?mode=mirror&onfail=abort'
|
||||
```
|
||||
|
||||
|
||||
### `/etc/ovh-config.json`
|
||||
- get this [template](/texts/ovh-config.json)
|
||||
- adapt the `url` keys to something meaningful ex. `backedupserver1_cold` and `backedupserver2_hot`. This will be the names of the Cloud Archives resp. Object Stores
|
||||
- change `PCA_TENANTID` and `SWIFT_TENANTID` to noted `OS_TENANT_ID`
|
||||
- change `PCA_USERNAME` and `SWIFT_USERNAME` to noted `OS_USERNAME`
|
||||
- do the same for Password and Regionname
|
||||
|
||||
## testing
|
||||
- do a test run
|
||||
- `backupninja -d -n`
|
@ -0,0 +1,156 @@
|
||||
# passed directly to duplicity
|
||||
|
||||
options = --volsize 200 --archive-dir /tmp --file-prefix-manifest 'hot_' --file-prefix-signature 'hot_' --file-prefix-archive 'cold_'
|
||||
|
||||
# default is 0, but set to 19 if you want to lower the priority.
|
||||
nicelevel = 19
|
||||
|
||||
# default is yes. set to no to skip the test if the remote host is alive.
|
||||
# if 'desturl' is set below, 'testconnect' must be set to 'no' for now.
|
||||
testconnect = no
|
||||
|
||||
## temporary directory used by duplicity, set to some other location if your /tmp is small
|
||||
## default is either /tmp or /usr/tmp, depending on the system
|
||||
##
|
||||
## Default:
|
||||
#tmpdir = /data/BACKUP/.cache
|
||||
|
||||
######################################################
|
||||
## gpg section
|
||||
## (how to encrypt and optionally sign the backups)
|
||||
##
|
||||
## WARNING: old (pre-0.9.4) example.dup used to give wrong information about
|
||||
## the way the following options are used. Please read the following
|
||||
## carefully.
|
||||
##
|
||||
## If the encryptkey variable is set:
|
||||
## - data is encrypted with the GnuPG public key specified by the encryptkey
|
||||
## variable
|
||||
## - if signing is enabled, data is signed with the GnuPG private
|
||||
## key specified by the signkey variable
|
||||
## - the password variable is used to unlock the GnuPG key(s) used
|
||||
## for encryption and (optionnal) signing
|
||||
##
|
||||
## If the encryptkey option is not set:
|
||||
## - data signing is not possible
|
||||
## - the password variable is used to encrypt the data with symmetric
|
||||
## encryption: no GnuPG key pair is needed
|
||||
|
||||
[gpg]
|
||||
|
||||
# when set to yes, encryptkey variable must be set below; if you want to use
|
||||
# two different keys for encryption and signing, you must also set the signkey
|
||||
# variable below.
|
||||
# default is no, for backwards compatibility with backupninja <= 0.5.
|
||||
sign = yes
|
||||
|
||||
# ID of the GnuPG public key used for data encryption.
|
||||
# if not set, symmetric encryption is used, and data signing is not possible.
|
||||
encryptkey =
|
||||
|
||||
# ID of the GnuPG private key used for data signing.
|
||||
# if not set, encryptkey will be used.
|
||||
signkey =
|
||||
|
||||
# password
|
||||
# NB: neither quote this, nor should it include any quotes
|
||||
password =
|
||||
|
||||
######################################################
|
||||
## source section
|
||||
## (where the files to be backed up are coming from)
|
||||
|
||||
[source]
|
||||
|
||||
# A few notes about includes and excludes:
|
||||
# 1. include, exclude and vsinclude statements support globbing with '*'
|
||||
# 2. Symlinks are not dereferenced. Moreover, an include line whose path
|
||||
# contains, at any level, a symlink to a directory, will only have the
|
||||
# symlink backed-up, not the target directory's content. Yes, you have to
|
||||
# dereference yourself the symlinks, or to use 'mount --bind' instead.
|
||||
# Example: let's say /home is a symlink to /mnt/crypt/home ; the following
|
||||
# line will only backup a "/home" symlink ; neither /home/user nor
|
||||
# /home/user/Mail will be backed-up :
|
||||
# include = /home/user/Mail
|
||||
# A workaround is to 'mount --bind /mnt/crypt/home /home' ; another one is to
|
||||
# write :
|
||||
# include = /mnt/crypt/home/user/Mail
|
||||
# 3. All the excludes come after all the includes. The order is not otherwise
|
||||
# taken into account.
|
||||
|
||||
# files to include in the backup
|
||||
# include = /var/spool/cron/crontabs
|
||||
include = /var/backups
|
||||
include = /etc
|
||||
include = /root
|
||||
include = /var/www
|
||||
include = /opt
|
||||
include = /home
|
||||
include = /var/lib/gitea
|
||||
|
||||
#include = /usr/local/*bin
|
||||
#include = /var/lib/dpkg/status*
|
||||
|
||||
# If vservers = yes in /etc/backupninja.conf then the following variables can
|
||||
# be used:
|
||||
# vsnames = all | <vserver1> <vserver2> ... (default = all)
|
||||
# vsinclude = <path>
|
||||
# vsinclude = <path>
|
||||
# ...
|
||||
# Any path specified in vsinclude is added to the include list for each vserver
|
||||
# listed in vsnames (or all if vsnames = all, which is the default).
|
||||
#
|
||||
# For example, vsinclude = /home will backup the /home directory in every
|
||||
# vserver listed in vsnames. If you have 'vsnames = foo bar baz', this
|
||||
# vsinclude will add to the include list /vservers/foo/home, /vservers/bar/home
|
||||
# and /vservers/baz/home.
|
||||
# Vservers paths are derived from .
|
||||
|
||||
|
||||
# files to exclude from the backup
|
||||
#exclude = /home/*/.gnupg
|
||||
#exclude = /home/*/.local/share/Trash
|
||||
#exclude = /home/*/.Trash
|
||||
#exclude = /home/*/.thumbnails
|
||||
#exclude = /srv/vmail/.cache
|
||||
#exclude = /root/.cache
|
||||
|
||||
######################################################
|
||||
## destination section
|
||||
## (where the files are copied to)
|
||||
|
||||
[dest]
|
||||
|
||||
# perform an incremental backup? (default = yes)
|
||||
# if incremental = no, perform a full backup in order to start a new backup set
|
||||
incremental = yes
|
||||
|
||||
# how many days of incremental backups before doing a full backup again ;
|
||||
# default is 30 days (one can also use the time format of duplicity).
|
||||
# if increments = keep, never automatically perform a new full backup ;
|
||||
# only perform incremental backups.
|
||||
#increments = 30
|
||||
#increments = keep
|
||||
increments = 90
|
||||
|
||||
# how many days of data to keep ; default is 60 days.
|
||||
# (you can also use the time format of duplicity)
|
||||
# 'keep = yes' means : do not delete old data, the remote host will take care of this
|
||||
#keep = 60
|
||||
#keep = 1Y
|
||||
#keep = yes
|
||||
keep = 90
|
||||
|
||||
# for how many full backups do we keep their later increments ;
|
||||
# default is all (keep all increments).
|
||||
# increments for older full backups will be deleted : only the more
|
||||
# recent ones (count provided) will be kept
|
||||
#keepincroffulls = all
|
||||
#keepincroffulls = 6
|
||||
#keepincroffulls =
|
||||
|
||||
# full destination URL, in duplicity format; if set, desturl overrides
|
||||
# sshoptions, destdir, desthost and destuser; it also disables testconnect and
|
||||
# bandwithlimit. For details, see duplicity manpage, section "URL FORMAT".
|
||||
|
||||
desturl = 'multi:///etc/ovh-config.json?mode=mirror&onfail=abort'
|
@ -0,0 +1,72 @@
|
||||
[
|
||||
{
|
||||
"description": "Cold storage",
|
||||
"url": "pca://duplicity_cold",
|
||||
"env": [
|
||||
{
|
||||
"name": "PCA_AUTHURL",
|
||||
"value": "https://auth.cloud.ovh.net/v3"
|
||||
},
|
||||
{
|
||||
"name": "PCA_AUTHVERSION",
|
||||
"value": "3"
|
||||
},
|
||||
{
|
||||
"name": "PCA_PROJECT_DOMAIN_NAME",
|
||||
"value": "Default"
|
||||
},
|
||||
{
|
||||
"name": "PCA_TENANTID",
|
||||
"value": ""
|
||||
},
|
||||
{
|
||||
"name": "PCA_USERNAME",
|
||||
"value": ""
|
||||
},
|
||||
{
|
||||
"name": "PCA_PASSWORD",
|
||||
"value": ""
|
||||
},
|
||||
{
|
||||
"name": "PCA_REGIONNAME",
|
||||
"value": "DE"
|
||||
}
|
||||
],
|
||||
"prefixes": ["cold_"]
|
||||
},
|
||||
{
|
||||
"description": "Hot storage",
|
||||
"url": "swift://duplicity_hot",
|
||||
"env": [
|
||||
{
|
||||
"name": "SWIFT_AUTHURL",
|
||||
"value": "https://auth.cloud.ovh.net/v3"
|
||||
},
|
||||
{
|
||||
"name": "SWIFT_AUTHVERSION",
|
||||
"value": "3"
|
||||
},
|
||||
{
|
||||
"name": "SWIFT_PROJECT_DOMAIN_NAME",
|
||||
"value": "Default"
|
||||
},
|
||||
{
|
||||
"name": "SWIFT_TENANTID",
|
||||
"value": ""
|
||||
},
|
||||
{
|
||||
"name": "SWIFT_USERNAME",
|
||||
"value": ""
|
||||
},
|
||||
{
|
||||
"name": "SWIFT_PASSWORD",
|
||||
"value": ""
|
||||
},
|
||||
{
|
||||
"name": "SWIFT_REGIONNAME",
|
||||
"value": "DE"
|
||||
}
|
||||
],
|
||||
"prefixes": ["hot_"]
|
||||
}
|
||||
]
|
Loading…
Reference in new issue