How to backup a Raspberry Pi over SSH?

Question

I tried following:

ssh <pi>@<ip> "sudo dd if=/dev/mmcblk0 bs=1M | gzip -" | dd of=~/Desktop/pibackup.gz
sudo: no tty present and no askpass program specified
0+1 records in
0+1 records out
20 bytes copied, 0,312135 s, 0,1 kB/s

I am pretty sure that I should be more than 20 bytes...

lsblk -p
NAME             MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
/dev/mmcblk0     179:0    0 14.9G  0 disk
├─/dev/mmcblk0p1 179:1    0 41.8M  0 part /boot
└─/dev/mmcblk0p2 179:2    0 14.8G  0 part /

and

Device         Boot Start      End  Sectors  Size Id Type
/dev/mmcblk0p1       8192    93802    85611 41.8M  c W95 FAT32 (LBA)
/dev/mmcblk0p2      98304 30375935 30277632 14.4G 83 Linux

Maybe you know a better solution. Thanks in advance.

Okay, I solved it, but i will leave it open because there could be discussed which solution would be better. If I use ssh -t <pi>@<ip> "sudo dd if=/dev/mmcblk0 bs=1M | gzip -" | dd of=~/Desktop/pibackup.gz it will be stucked and 123 bytes are copied. with ssh <pi>@<ip> "sudo -S dd if=/dev/mmcblk0 bs=1M | gzip -" | dd of=~/Desktop/pibackup.gz and later type in the passwd it works because he need the sudo permissions. Without -S he tries to copy, but it will be aborted, so the copied file couldn not be very large. — Micha93, Feb 04 '21 at 15:32
So you are using dd to copy the SD card of the live system? I expect this will lead to issues, as the system is still live and modifying files, therefor you could end with a corrupt image. May not always happen, but I would not trust this to always work. — Chad G, Feb 04 '21 at 17:28
As others have pointed out, the use of dd to create backups of a live system is an anti-pattern. There is no good way to do it, so don't. — goldilocks, Feb 05 '21 at 19:29

Seamus · Answer 1 · 2021-02-05T19:20:02.093

You will (sooner or later) run into problems with the approach you're taking. Running dd to back up a running system (or even a mounted drive) is rolling the dice. It's been said of one who follows this practice: "If you're lucky, the filesystem corruption will be detected as soon as you try to mount the copy. If you're unlucky, it won't be detected until later." Seriously - this is not "good practice", and there are far better ways to make backups of a "live" system.

This Q&A contains a "step-by-step" procedure for a much more reliable approach using image-backup. I'd suggest you consider this approach. If you don't like image-backup, there are numerous other approaches outlined in this Q&A

score 1 · Answer 2 · answered Feb 15 '21 at 08:01

Your immediate problem stems from the fact that you have enabled sudo password (the Pi OS is configured to never ask for a password when running sudo), and then you try to run sudo non-interactively. Either put NOPASSWD back into your sudoers file, or learn how to provide a password to non-interactive sudo (e.g. sudo -S).

Others already explained how copying a live partition is a bad idea. I'll just add that using dd for this is even worse: dd was specifically made to handle tape drive archives where you might need to to discard some garbage data at the end of each record, and assemble records in a special manner to convert them to an actual file. If you just need to copy something, use cp: it will never omit or alter any data while copying, and is smart enough to determine the optimal block size for the device.

score 0 · Answer 3 · answered Feb 07 '21 at 05:31

How I do it over ssh to backup my Pi 4 to my ssd drive. Firstly you need to put your ssh id the .ssh/id_rsa.pub of the user running the rsync command into the /root/.ssh/authorized_keys file you copy the contents into the /root file or you need to allow root login via ssh. The key file is best for passwordless login.

Next you need to prepare your disk to be copied to, an example of it below made with fdisk. the commands to make it are the same as with the gdisk I show below with one exception you need to do an "a", the single letter, command after the partitioning and before the writing too select the 1 partition and make it active or a MBR drive will not boot, then on to formatting like I show there too and now I think 0b for the type of the first boot partition, FAT 32, and 8300 for the second no difference with gdisk..

root@zeus-H370M-DS3H:/home/zeus# fdisk -l /dev/sdc
Disk /dev/sde: 111.79 GiB, 120034123776 bytes, 234441648 sectors
Disk model:  SH103S3120G    
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 33553920 bytes
Disklabel type: dos
Disk identifier: 0xa62408c8
Device     Boot  Start       End   Sectors   Size Id Type
/dev/sdc1         8192    532479    524288   256M  c W95 FAT32 (LBA)
/dev/sdc2       532480 234441647 233909168 111.5G 83 Linux

To prepare your disk as GPT as I discovered the Pi 4 will boot from GPT partitioned drive when I forgot make the boot partition active with fdisk. It had the warning no GPT found then after it gives up searching it goes onto booting MBR drive, we never get to see it as it is only visible with a text boot.

root@zeus-H370M-DS3H:/home/zeus# dd if=/dev/zero of=/dev/sde bs=512 count=34
34+0 records in
34+0 records out
17408 bytes (17 kB, 17 KiB) copied, 0.00286715 s, 6.1 MB/s
root@zeus-H370M-DS3H:/home/zeus# dd if=/dev/zero of=/dev/sde bs=512 count=34 seek=$((`blockdev --getsz /dev/sde` - 34))
34+0 records in
34+0 records out
17408 bytes (17 kB, 17 KiB) copied, 0.00584954 s, 3.0 MB/s
root@zeus-H370M-DS3H:/home/zeus# gdisk /dev/sde
GPT fdisk (gdisk) version 1.0.5
Partition table scan:
  MBR: not present
  BSD: not present
  APM: not present
  GPT: not present
Creating new GPT entries in memory.
Command (? for help): n
Partition number (1-128, default 1): 1
First sector (34-234441614, default = 2048) or {+-}size{KMGTP}: 
Last sector (2048-234441614, default = 234441614) or {+-}size{KMGTP}: +256M
Current type is 8300 (Linux filesystem)
Hex code or GUID (L to show codes, Enter = 8300): ef00
Changed type of partition to 'EFI system partition'
Command (? for help): n
Partition number (2-128, default 2): 2
First sector (34-234441614, default = 526336) or {+-}size{KMGTP}: 
Last sector (526336-234441614, default = 234441614) or {+-}size{KMGTP}: 
Current type is 8300 (Linux filesystem)
Hex code or GUID (L to show codes, Enter = 8300): 
Changed type of partition to 'Linux filesystem'
Command (? for help): w
Final checks complete. About to write GPT data. THIS WILL OVERWRITE EXISTING
PARTITIONS!!
Do you want to proceed? (Y/N): y
OK; writing new GUID partition table (GPT) to /dev/sde.
The operation has completed successfully.
root@zeus-H370M-DS3H:/home/zeus# mkfs.vfat /dev/sde1
mkfs.fat 4.1 (2017-01-24)
root@zeus-H370M-DS3H:/home/zeus# mkfs.ext4 /dev/sde2
mke2fs 1.45.6 (20-Mar-2020)
Creating filesystem with 29239409 4k blocks and 7315456 inodes
Filesystem UUID: 1df05227-02f4-497a-b9e7-f8e4414bf339
Superblock backups stored on blocks: 
32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208, 
4096000, 7962624, 11239424, 20480000, 23887872
Allocating group tables: done

Creating journal (131072 blocks): done
Writing superblocks and filesystem accounting information: done

What it looks like after it is done.

root@zeus-H370M-DS3H:/home/zeus# fdisk -l /dev/sde
Disk /dev/sde: 111.79 GiB, 120034123776 bytes, 234441648 sectors
Disk model: 2115            
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: B0FAEF78-76E3-45DD-8173-518EC01E3889
Device      Start       End   Sectors   Size Type
/dev/sde1    2048    526335    524288   256M EFI System
/dev/sde2  526336 234441614 233915279 111.5G Linux filesystem

To copy the files you make the mount point needed.

root@zeus-H370M-DS3H:/home/zeus# mkdir /tmp/ssdboot
root@zeus-H370M-DS3H:/home/zeus#  mkdir /tmp/ssdroot
root@zeus-H370M-DS3H:/home/zeus# mount /dev/sde2 /tmp/ssdroot
root@zeus-H370M-DS3H:/home/zeus# mount /dev/sde1 /tmp/ssdboot

For the / (root) copying you need to change the ip to the Pi you have.

root@zeus-H370M-DS3H:/home/zeus# rsync -ahPHAXx --delete --exclude={/dev/*,/proc/*,/sys/*,/tmp/*,/run/*,/mnt/*,/media/*,/lost+found} root@192.168.0.111:/ /tmp/ssdroot/

For the /boot copying you need to change the ip to the Pi you have.

root@zeus-H370M-DS3H:/home/zeus# rsync -ahPHAXx --delete root@192.168.0.111:/boot/* /tmp/ssdboot/

If all seems to have copied over now to changing the UUIDs.

root@zeus-H370M-DS3H:/home/zeus# blkid | grep sde
/dev/sde1: SEC_TYPE="msdos" UUID="EF07-CF76" BLOCK_SIZE="512" TYPE="vfat" PARTLABEL="EFI system partition" PARTUUID="582ecdd1-53dd-4e6c-bedd-6fc4c6e5afbf"
/dev/sde2: UUID="1df05227-02f4-497a-b9e7-f8e4414bf339" BLOCK_SIZE="4096" TYPE="ext4" PARTLABEL="Linux filesystem" PARTUUID="6fc53fb7-324c-466e-b872-79eca8a22d4b"

Added the long PARTUUIDs in the correct places and shutdown the pi and booted. It was way to faster for me this time, nothing about the GPT flashed or anything as it was GPT drive used.

root@zeus-H370M-DS3H:/home/zeus# nano /tmp/ssdboot/cmdline.txt 
root@buster-raspi:~# cat /boot/cmdline.txt 
console=tty0 console=ttyS1,115200 root=PARTUUID=6fc53fb7-324c-466e-b872-79eca8a22d4b rw elevator=deadline fsck.repair=yes net.ifnames=0  rootwait
root@zeus-H370M-DS3H:/home/zeus# nano /tmp/ssdroot/etc/fstab
root@buster-raspi:~# cat /etc/fstab
The root file system has fs_passno=1 as per fstab(5) for automatic fsck.
#LABEL=RASPIROOT / ext4 rw 0 1
PARTUUID=6fc53fb7-324c-466e-b872-79eca8a22d4b / ext4 rw 0 1
All other file systems have fs_passno=2 as per fstab(5) for automatic fsck.
#LABEL=RASPIFIRM /boot/firmware vfat rw 0 2
PARTUUID=582ecdd1-53dd-4e6c-bedd-6fc4c6e5afbf /boot vfat rw 0 2
the /proc filesytem
proc /proc proc defaults 0 0

Use MBR drive for anything less that a Pi 4 or 400 as I think those are the only ones for GPT, I know for certain it works flawlessly for me to clone my SSD to another and boot.

How to backup a Raspberry Pi over SSH?

3 Answers3

The root file system has fs_passno=1 as per fstab(5) for automatic fsck.

All other file systems have fs_passno=2 as per fstab(5) for automatic fsck.

the /proc filesytem