parent
370440e9ef
commit
87a8b1f1d5
@ -0,0 +1,86 @@
|
|||||||
|
---
|
||||||
|
title: Using OVH PCA with backupninja
|
||||||
|
description: Accessing the OVH Public cloud archive with the automated backup tool backupninja using its duplicity backend
|
||||||
|
date: 2022-03-03
|
||||||
|
---
|
||||||
|
|
||||||
|
## motivation
|
||||||
|
OVH provides a cheap way for longterm storing backup data, the Public Cloud Archive (PCA). As this is for write-once data (cold data store), it needs to be used together with a little bit more expensive hot data store for meta data when used for sequential backups. The backup solution [duplicity](https://duplicity.gitlab.io/duplicity-web/) is capable of using such multi backends. [backupninja](https://0xacab.org/liberate/backupninja) provides a nice solution for a standardized configuration of duplicity and running it automatically.
|
||||||
|
|
||||||
|
## Prerequisites
|
||||||
|
### On system
|
||||||
|
- Enable email sending for backup status mails by [installing a mta](/msmtp-on-debian/).
|
||||||
|
- Install system dependencies
|
||||||
|
- `apt install librsync-dev gpg backupninja`
|
||||||
|
|
||||||
|
#### GPG
|
||||||
|
- Create a new secret key. Note the password.
|
||||||
|
- `gpg --quick-generate-key cloud@freedomhost.de`
|
||||||
|
- `gpg --export-secret-key keyid > private.key`
|
||||||
|
- Copy the key to the server
|
||||||
|
- `scp private.key server:`
|
||||||
|
- Import it in the local keyring on server
|
||||||
|
- `gpg --allow-secret-key-import --import private.key`
|
||||||
|
- Increase trust level:
|
||||||
|
|
||||||
|
```
|
||||||
|
# gpg --edit-key KEY_ID
|
||||||
|
> trust
|
||||||
|
> 5
|
||||||
|
> quit
|
||||||
|
```
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
### On OVH
|
||||||
|
- Create a new user unter Cloud Archive → Project Management → Users & Roles
|
||||||
|
- Give the user the Role `ObjectStore operator`
|
||||||
|
- Note username and **password**
|
||||||
|
- Get TenandID
|
||||||
|
- click 3 dots on the right of the users row. Click Download OpenStack's RC file. Here you can select a region where the PCA should be set up
|
||||||
|
- Note the `OS_TENANT_ID`, here you also can extract `OS_USERNAME` and `OS_REGION_NAME`
|
||||||
|
|
||||||
|
|
||||||
|
## Installation
|
||||||
|
- Be sure to have at least version 0.8.21 of duplicity
|
||||||
|
- `pip3 install duplicity>=0.8.21 python-swiftclient python-keystoneclient`
|
||||||
|
|
||||||
|
## Configuration
|
||||||
|
- 3 files are used for this:
|
||||||
|
- `/etc/backupninja.conf`: Here you can set the time of day when the backup should be run, and to whom to send status emails. This file is pre-installed and quite self-explaining
|
||||||
|
- `/etc/ovh-config.json` for holding the OVH credentials
|
||||||
|
- `/etc/backup.d/20_ovh_pca.dup` for configuring duplicity for backupninja usage
|
||||||
|
|
||||||
|
|
||||||
|
### `/etc/backup.d/20_ovh_pca.dup`
|
||||||
|
- There is a full example for duplicity configuration in `/usr/share/doc/backupninja/examples/example.dup` find a slightly pre configured version [here](/texts/20_ovh_pca.dup)
|
||||||
|
- most important options are:
|
||||||
|
|
||||||
|
```
|
||||||
|
options = --volsize 200 --archive-dir /tmp --file-prefix-manifest 'hot_' --file-prefix-signature 'hot_' --file-prefix-archive 'cold_'
|
||||||
|
testconnect = no
|
||||||
|
|
||||||
|
[gpg]
|
||||||
|
sign = yes
|
||||||
|
encryptkey = GPGKEY
|
||||||
|
password = GPGKEYPASSWORD
|
||||||
|
|
||||||
|
[source]
|
||||||
|
include = SOMEDIR
|
||||||
|
include = SOMEOTHERDIR
|
||||||
|
|
||||||
|
[dest]
|
||||||
|
desturl = 'multi:///etc/ovh-config.json?mode=mirror&onfail=abort'
|
||||||
|
```
|
||||||
|
|
||||||
|
|
||||||
|
### `/etc/ovh-config.json`
|
||||||
|
- get this [template](/texts/ovh-config.json)
|
||||||
|
- adapt the `url` keys to something meaningful ex. `backedupserver1_cold` and `backedupserver2_hot`. This will be the names of the Cloud Archives resp. Object Stores
|
||||||
|
- change `PCA_TENANTID` and `SWIFT_TENANTID` to noted `OS_TENANT_ID`
|
||||||
|
- change `PCA_USERNAME` and `SWIFT_USERNAME` to noted `OS_USERNAME`
|
||||||
|
- do the same for Password and Regionname
|
||||||
|
|
||||||
|
## testing
|
||||||
|
- do a test run
|
||||||
|
- `backupninja -d -n`
|
@ -0,0 +1,156 @@
|
|||||||
|
# passed directly to duplicity
|
||||||
|
|
||||||
|
options = --volsize 200 --archive-dir /tmp --file-prefix-manifest 'hot_' --file-prefix-signature 'hot_' --file-prefix-archive 'cold_'
|
||||||
|
|
||||||
|
# default is 0, but set to 19 if you want to lower the priority.
|
||||||
|
nicelevel = 19
|
||||||
|
|
||||||
|
# default is yes. set to no to skip the test if the remote host is alive.
|
||||||
|
# if 'desturl' is set below, 'testconnect' must be set to 'no' for now.
|
||||||
|
testconnect = no
|
||||||
|
|
||||||
|
## temporary directory used by duplicity, set to some other location if your /tmp is small
|
||||||
|
## default is either /tmp or /usr/tmp, depending on the system
|
||||||
|
##
|
||||||
|
## Default:
|
||||||
|
#tmpdir = /data/BACKUP/.cache
|
||||||
|
|
||||||
|
######################################################
|
||||||
|
## gpg section
|
||||||
|
## (how to encrypt and optionally sign the backups)
|
||||||
|
##
|
||||||
|
## WARNING: old (pre-0.9.4) example.dup used to give wrong information about
|
||||||
|
## the way the following options are used. Please read the following
|
||||||
|
## carefully.
|
||||||
|
##
|
||||||
|
## If the encryptkey variable is set:
|
||||||
|
## - data is encrypted with the GnuPG public key specified by the encryptkey
|
||||||
|
## variable
|
||||||
|
## - if signing is enabled, data is signed with the GnuPG private
|
||||||
|
## key specified by the signkey variable
|
||||||
|
## - the password variable is used to unlock the GnuPG key(s) used
|
||||||
|
## for encryption and (optionnal) signing
|
||||||
|
##
|
||||||
|
## If the encryptkey option is not set:
|
||||||
|
## - data signing is not possible
|
||||||
|
## - the password variable is used to encrypt the data with symmetric
|
||||||
|
## encryption: no GnuPG key pair is needed
|
||||||
|
|
||||||
|
[gpg]
|
||||||
|
|
||||||
|
# when set to yes, encryptkey variable must be set below; if you want to use
|
||||||
|
# two different keys for encryption and signing, you must also set the signkey
|
||||||
|
# variable below.
|
||||||
|
# default is no, for backwards compatibility with backupninja <= 0.5.
|
||||||
|
sign = yes
|
||||||
|
|
||||||
|
# ID of the GnuPG public key used for data encryption.
|
||||||
|
# if not set, symmetric encryption is used, and data signing is not possible.
|
||||||
|
encryptkey =
|
||||||
|
|
||||||
|
# ID of the GnuPG private key used for data signing.
|
||||||
|
# if not set, encryptkey will be used.
|
||||||
|
signkey =
|
||||||
|
|
||||||
|
# password
|
||||||
|
# NB: neither quote this, nor should it include any quotes
|
||||||
|
password =
|
||||||
|
|
||||||
|
######################################################
|
||||||
|
## source section
|
||||||
|
## (where the files to be backed up are coming from)
|
||||||
|
|
||||||
|
[source]
|
||||||
|
|
||||||
|
# A few notes about includes and excludes:
|
||||||
|
# 1. include, exclude and vsinclude statements support globbing with '*'
|
||||||
|
# 2. Symlinks are not dereferenced. Moreover, an include line whose path
|
||||||
|
# contains, at any level, a symlink to a directory, will only have the
|
||||||
|
# symlink backed-up, not the target directory's content. Yes, you have to
|
||||||
|
# dereference yourself the symlinks, or to use 'mount --bind' instead.
|
||||||
|
# Example: let's say /home is a symlink to /mnt/crypt/home ; the following
|
||||||
|
# line will only backup a "/home" symlink ; neither /home/user nor
|
||||||
|
# /home/user/Mail will be backed-up :
|
||||||
|
# include = /home/user/Mail
|
||||||
|
# A workaround is to 'mount --bind /mnt/crypt/home /home' ; another one is to
|
||||||
|
# write :
|
||||||
|
# include = /mnt/crypt/home/user/Mail
|
||||||
|
# 3. All the excludes come after all the includes. The order is not otherwise
|
||||||
|
# taken into account.
|
||||||
|
|
||||||
|
# files to include in the backup
|
||||||
|
# include = /var/spool/cron/crontabs
|
||||||
|
include = /var/backups
|
||||||
|
include = /etc
|
||||||
|
include = /root
|
||||||
|
include = /var/www
|
||||||
|
include = /opt
|
||||||
|
include = /home
|
||||||
|
include = /var/lib/gitea
|
||||||
|
|
||||||
|
#include = /usr/local/*bin
|
||||||
|
#include = /var/lib/dpkg/status*
|
||||||
|
|
||||||
|
# If vservers = yes in /etc/backupninja.conf then the following variables can
|
||||||
|
# be used:
|
||||||
|
# vsnames = all | <vserver1> <vserver2> ... (default = all)
|
||||||
|
# vsinclude = <path>
|
||||||
|
# vsinclude = <path>
|
||||||
|
# ...
|
||||||
|
# Any path specified in vsinclude is added to the include list for each vserver
|
||||||
|
# listed in vsnames (or all if vsnames = all, which is the default).
|
||||||
|
#
|
||||||
|
# For example, vsinclude = /home will backup the /home directory in every
|
||||||
|
# vserver listed in vsnames. If you have 'vsnames = foo bar baz', this
|
||||||
|
# vsinclude will add to the include list /vservers/foo/home, /vservers/bar/home
|
||||||
|
# and /vservers/baz/home.
|
||||||
|
# Vservers paths are derived from .
|
||||||
|
|
||||||
|
|
||||||
|
# files to exclude from the backup
|
||||||
|
#exclude = /home/*/.gnupg
|
||||||
|
#exclude = /home/*/.local/share/Trash
|
||||||
|
#exclude = /home/*/.Trash
|
||||||
|
#exclude = /home/*/.thumbnails
|
||||||
|
#exclude = /srv/vmail/.cache
|
||||||
|
#exclude = /root/.cache
|
||||||
|
|
||||||
|
######################################################
|
||||||
|
## destination section
|
||||||
|
## (where the files are copied to)
|
||||||
|
|
||||||
|
[dest]
|
||||||
|
|
||||||
|
# perform an incremental backup? (default = yes)
|
||||||
|
# if incremental = no, perform a full backup in order to start a new backup set
|
||||||
|
incremental = yes
|
||||||
|
|
||||||
|
# how many days of incremental backups before doing a full backup again ;
|
||||||
|
# default is 30 days (one can also use the time format of duplicity).
|
||||||
|
# if increments = keep, never automatically perform a new full backup ;
|
||||||
|
# only perform incremental backups.
|
||||||
|
#increments = 30
|
||||||
|
#increments = keep
|
||||||
|
increments = 90
|
||||||
|
|
||||||
|
# how many days of data to keep ; default is 60 days.
|
||||||
|
# (you can also use the time format of duplicity)
|
||||||
|
# 'keep = yes' means : do not delete old data, the remote host will take care of this
|
||||||
|
#keep = 60
|
||||||
|
#keep = 1Y
|
||||||
|
#keep = yes
|
||||||
|
keep = 90
|
||||||
|
|
||||||
|
# for how many full backups do we keep their later increments ;
|
||||||
|
# default is all (keep all increments).
|
||||||
|
# increments for older full backups will be deleted : only the more
|
||||||
|
# recent ones (count provided) will be kept
|
||||||
|
#keepincroffulls = all
|
||||||
|
#keepincroffulls = 6
|
||||||
|
#keepincroffulls =
|
||||||
|
|
||||||
|
# full destination URL, in duplicity format; if set, desturl overrides
|
||||||
|
# sshoptions, destdir, desthost and destuser; it also disables testconnect and
|
||||||
|
# bandwithlimit. For details, see duplicity manpage, section "URL FORMAT".
|
||||||
|
|
||||||
|
desturl = 'multi:///etc/ovh-config.json?mode=mirror&onfail=abort'
|
@ -0,0 +1,72 @@
|
|||||||
|
[
|
||||||
|
{
|
||||||
|
"description": "Cold storage",
|
||||||
|
"url": "pca://duplicity_cold",
|
||||||
|
"env": [
|
||||||
|
{
|
||||||
|
"name": "PCA_AUTHURL",
|
||||||
|
"value": "https://auth.cloud.ovh.net/v3"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"name": "PCA_AUTHVERSION",
|
||||||
|
"value": "3"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"name": "PCA_PROJECT_DOMAIN_NAME",
|
||||||
|
"value": "Default"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"name": "PCA_TENANTID",
|
||||||
|
"value": ""
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"name": "PCA_USERNAME",
|
||||||
|
"value": ""
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"name": "PCA_PASSWORD",
|
||||||
|
"value": ""
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"name": "PCA_REGIONNAME",
|
||||||
|
"value": "DE"
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"prefixes": ["cold_"]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"description": "Hot storage",
|
||||||
|
"url": "swift://duplicity_hot",
|
||||||
|
"env": [
|
||||||
|
{
|
||||||
|
"name": "SWIFT_AUTHURL",
|
||||||
|
"value": "https://auth.cloud.ovh.net/v3"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"name": "SWIFT_AUTHVERSION",
|
||||||
|
"value": "3"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"name": "SWIFT_PROJECT_DOMAIN_NAME",
|
||||||
|
"value": "Default"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"name": "SWIFT_TENANTID",
|
||||||
|
"value": ""
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"name": "SWIFT_USERNAME",
|
||||||
|
"value": ""
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"name": "SWIFT_PASSWORD",
|
||||||
|
"value": ""
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"name": "SWIFT_REGIONNAME",
|
||||||
|
"value": "DE"
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"prefixes": ["hot_"]
|
||||||
|
}
|
||||||
|
]
|
Loading…
Reference in new issue