Backup Raspberry Pi automatically, with incremental backups

Question

There are some different questions and answers on this subject. However, when searching I have been unable to find an answer that could be considered "canonical" or that gathers the various options in one place.

I would like to backup the contents of my Raspberry Pi to enable a complete restoration if needed, easily and reliably.

These features would be ideal:

• Backups kept on external location (USB stick, DAS, NAS, Network computer, etc)
• Automatisation, either incremental or via scheduling
• Version numbering of backups

Cloning the SD card seems to be a popular solution, but according to this post by Goldilocks this is not advisable because a clone contains files that are loaded only on runtime, etc.

So I'm assuming that a good solution uses something like rsync or perhaps more robustly, a utility that can handle the critera above.

My particular situation: I have a Raspberry Pi on a LAN where I can backup to a Mac OS X computer, a Synology DiskStation NAS or to a USB memory stick connected to the Raspberry. I would prefer not to use the latter.

Answer 1

Rsnapshot fulfills those criteria:¹

• Can be used locally or remotely.

• Can be automated/scheduled.

• Uses numbered incremental backups.

I don't have much personal experience with it beyond the fact that it's used on servers I work on, where it has saved the day occasionally, but there seems to be a nice introduction here, the Arch linux wiki, which tends to have great documentation, has a page (I'm sure most of this applies equally well to other distros), and the home page refers to a HOWTO which I can't find online, but searching for it turned up numerous tutorials, discussions, etc. in addition to these links (the actual how-to may be kept offline so that people do not get confused using the wrong one with the wrong version).

---

1. Although apparently now (2018) it may be unmaintained in favour of BorgBackup.

Answer 2

If you consider these 2 facts it will open you up to other options:

1. You can mount the same device in multiple directories simultaneously.
2. You can create tmpfs mounts anywhere you want to mask a directory.

So, yes there are many paths that you don't want to backup. You avoid most of them by mounting your primary partition at a 2nd directory. Others will still have data you don't want to backup like /tmp, /dev, and /var/log.

My process went like this...

1. SSH into the Raspberry Pi

2. Mount exactly want I wanted to backup

   sudo mount                     /dev/mmcblk0p2    /tmp/root/
   sudo mount -t tmpfs -o size=1m tmpfs             /tmp/root/var/log/
   sudo mount -t tmpfs -o size=1m tmpfs             /tmp/root/dev/
   sudo mount -t tmpfs -o size=1m tmpfs             /tmp/root/tmp/
   sudo mount                     /dev/mmcblk0p1    /tmp/root/boot/
   

3. Exit the machine

4. Copy the data over via ssh+tar

   ssh pi@raspberry.local 'cd /tmp/root; sudo tar cf - * | gzip;' | pv > rpi.tgz
   # NOTE: The `pv` command gives you a progress meter but can be left out.
   

5. Remove the temporary mounts

   for m in /var/log/ /dev/ /boot/ /; do sudo umount /tmp/root${m}; done
   

Once you are happy with the results, you can put it all in a single file like ~/backup.sh

#!/bin/bash -eu

dir=$(mktemp -d)

cleanup(){
    cd /tmp/ # You can't umount or rm a directory while you are in it.
    for m in /dev/ /tmp/ /var/log/ /boot/ /; do
        sudo umount ${dir}${m}
    done
    rm -rf ${dir}
}

do_mounts(){
    sudo mount                     /dev/mmcblk0p2    ${dir}/
    sudo mount -t tmpfs -o size=1m tmpfs             ${dir}/dev/
    sudo mount -t tmpfs -o size=1m tmpfs             ${dir}/tmp/
    sudo mount -t tmpfs -o size=1m tmpfs             ${dir}/var/log/
    sudo mount                     /dev/mmcblk0p1    ${dir}/boot/
}

send_data(){
    cd ${dir}; sudo tar cf - * | gzip | tee >(md5sum > /tmp/backup.md5);
}

give_feedback(){
    awk '{print "MD5:", $1}' < /tmp/backup.md5 >&2
}

trap cleanup EXIT INT TERM
do_mounts
send_data
give_feedback

And calling goes like this...

$ ssh pi@raspberry.local ./backup.sh | pv | tee rpi.tgz | md5sum | awk '{print "MD5:", $1}'
MD5: d3d9181374f3ec8e4e721c786eca9f71
 348MB 0:04:50 [ 1.2MB/s] [                 <=>                                ]
MD5: d3d9181374f3ec8e4e721c786eca9f71

HINT: While you are experimenting, change tar cf - * to tar cf - etc to save yourself a bunch of time on each test run.

Answer 3

I had the same issue with my Raspis @ home. That's why I wrote raspiBackup which allows to create dd, tar and rsync backups on a regular base without user invention. Just give it a try.

Answer 4

I have tried several backup techniques. I use 2 on a regular basis.

I clone the SD card on my Mac and compress the image. I use this as a security backup or to make duplicate cards. I agree this is not an ideal. My script is below.

More frequently, I use rsync to copy to a HD attached to the Pi. See https://raspberrypi.stackexchange.com/a/28087/8697 for details.

In fact I have slightly different versions to backup Wheezy, Jessie and MATE images (to different directories).

I did try to rsync to my Mac, but this only partially worked. The fundamental problem is that the users on the Mac and Pi do not match, not to mention the differences between rsync on the different platforms. With the changes to security in the latest OS X it probably would have even more problems.

I have successfully used one Pi to make a backup of another over a network.

I do not know how this would work to a NAS, but suspect there may be problems with hardlinks and permissions.

#!/bin/bash
# script to backup Pi SD card
# 2017-06-05
# DSK='disk4'   # manual set disk
OUTDIR=~/temp/Pi
# Find disk with Linux partition (works for Raspbian)
# Modified for PINN/NOOBS
export DSK=`diskutil list | grep "Linux" | sed 's/.*\(disk[0-9]\).*/\1/' | uniq`
if [ $DSK ]; then
    echo $DSK
    echo $OUTDIR
else
    echo "Disk not found"
    exit
fi

diskutil unmountDisk /dev/$DSK
echo please wait - This takes some time
echo Ctl+T to show progress!
time sudo dd if=/dev/r$DSK bs=4m | gzip -9 > $OUTDIR/Piback.img.gz

#rename to current date
echo compressing completed - now renaming
mv -n $OUTDIR/Piback.img.gz $OUTDIR/Piback`date +%Y%m%d`.img.gz