Backups/En

From TuxFamilyFAQ
Jump to navigationJump to search


Principles

Due to financial reasons, TuxFamily does not provide any backup service. While the technical team sure has minimal backups that may be used to repair the whole platform in case of a huge technical problem, this is far from the usual expectations for a backup service: decent retention, ability to cherry-pick data to be restored without contacting an admin, etc. Basically, consider our backups are like this. However, TuxFamily still provides everything you need to setup your own regular backups using your own computer(s).

Why should I backup my data?

Three main reasons:

  • Avoid losing your work, or the work of your users/contributors, in case of a technical issue on TuxFamily's side -- while our infrastructure is pretty reliable, this has always been reason #1 to backup data.
  • Offer yourself the ability to retrieve your web site as it was...
    • before you made that huge misconfiguration in your CMS or whatever
    • or before it got hacked by some script kiddie because you did not respect our Security recommendations.
  • The more data you can retrieve by yourself, the more you are independent from your hoster.

What can I backup and how?

  • Each night, we generate one .sql dump for each of your MySQL/PgSQL database; you can retrieve them using scp/rsync.
  • Webareas and download repositories can be fetched using rsync.
  • SVN repositories can be mirrored using svnsync.
  • Git and Mercurial repositories can be easily cloned.

What can't I backup and why?

There is currently no way for you to backup what you configure in the TuxFamily panel (group description/members, mailing lists, mail domains, domain names and cron jobs parameters) -- however, the underlying database is regularly backed up on our side with good retention, so you should not worry about it. Note you will have to contact the TuxFamily staff to retrieve this kind of data. Also, scripts provided here focus on web areas, download repositories, databases, Subversion repositories and do not treat:

  • CVS, because TuxFamily does not provide any way to backup them completely; you may want to try cvssuck.
  • Git/Mercurial repositories, mailing lists, mailboxes.

Prerequisites

The scripts and techniques detailed below expect your SSH account to be enabled. If you want to schedule them instead of executing them manually, you will also need to set up a SSH key, as described on User/En. Notes:

  • this SSH key must be available to the user account that will run your backups
  • ensure your backups are stored in a secured place (e.g. not world-readable): do not let your backups become another attack vector...

Also, they are Unix techniques, tested under Linux. This implies you need a Linux, Unix or Unix-like operating system, which is expected to be rather common in the Free Software world. Do not try to use your TuxFamily SSH account for this purpose: beyond quota-induced problems, it does not make sense to backup data from the hosting platform to the hosting platform itself.

Web areas, download repositories and databases

When it comes to fetching and archiving data regularly, many possibilities exist, ranging from a custom scp-based script to expensive proprietary solutions. You are of course free to choose the solution that fits your needs, depending on your skills and constraints. What follows is an abbreviated procedure to schedule TuxFamily backups for your_project using your_user account and the rsnapshot tool. You can of course change paths to whatever fits your needs (well, avoid /tmp and other tmpfs though):

Create the root directory for your backup

mkdir -p /data/backups/{rsnapshot,run}
chmod 700 /data/backups

Adapt this configuration to create /data/backups/rsnapshot.conf

# This configuration file comes from a simple:
# zegrep '^[^#]' /usr/share/doc/rsnapshot/examples/rsnapshot.conf.default.gz
# ... on my debian workstation. See man 1 rsnapshot for details.
config_version	1.2
snapshot_root	/data/backups/rsnapshot/
cmd_cp		/bin/cp
cmd_rm		/bin/rm
cmd_rsync	/usr/bin/rsync
cmd_ssh	/usr/bin/ssh
cmd_logger	/usr/bin/logger
retain	daily	60
retain	weekly	8
verbose	2
loglevel	3
lockfile	/data/backups/run/rsnapshot.pid
backup	your_user@ssh.tuxfamily.org:/home/your_project/	tuxfamily/

Take care: the configuration lines above contain tabs characters that must be preserved.

Schedule backups:

crontab -e
# m h  dom mon dow   command
0  8    *   *   *   /usr/bin/rsnapshot -c /data/backups/rsnapshot.conf daily
0 20    *   *   1   /usr/bin/rsnapshot -c /data/backups/rsnapshot.conf weekly

See "man 5 crontab" for more information about the syntax above.

SVN repositories

The SVNSync tool enables you to create and update a copy of an existing repository. You may find many tutorials on the Internet about how it works and how to use it, so this FAQ will just briefly repeat the procedure for TuxFamily, considering you want to backup your_svn repository from your_project:

Prepare the synced repository:

mkdir -p /data/backups/svnsync/your_project
cd /data/backups/svnsync/your_project

# create a new SVN repository
svnadmin create your_svn.synced

# Make this repository allow revision property changes
echo '#!/bin/sh' > your_svn.synced/hooks/pre-revprop-change
chmod ug+x your_svn.synced/hooks/pre-revprop-change

# prepare the new repository to be a read-only mirror of your SVN repository
svnsync init file:///data/backups/svnsync/your_project/your_svn.synced svn://svn.tuxfamily.org/svnroot/your_project/your_svn/

Update it regularly:

crontab -e
# Retrieval of your project SVN repository (incremental update)
30 08 * * * /usr/bin/svnsync --non-interactive synchronize file:///data/backups/svnsync/your_project/your_svn.synced

This mirror will enable you to dump (and optionally filter) your SVN repository. In case you need to restore such a dump to TuxFamily, please contact the TuxFamily staff.