e2fsck on Raspberry Pis

Question

I need to run e2fsck on the raspberry pi, because I want to create a shrinked Backup. I have mounted an external server inside /mnt/BACKUP. There I have created an bash script. It will first create a full image inside /mnt/BACKUP/images/<hostname>/. There you will find an Backup for each day. Here is what I have done so far:

#!/bin/bash
# VERSION=1
#
# Automatisches Sichern der Micro-SD-Karten mit Hilfe von "dd" und "cronjob".
Konfiguration:
Angabe des Verzeichnisses, das gesichert werden soll:
SOURCE="/dev/mmcblk0"
Angabe des Verzeichnisses, in das gesichert werden soll:
BEACHTEN: Dies ist in der Regel das via NFS eingebundene NAS-Verzeichnis.
TARGET="/mnt/BACKUP/automatic/images/$(hostname -f)"
if [ ! -d $TARGET ]; then
  mkdir -p $TARGET
fi
Angabe wie viele zurückgehaltene Backups behalten werden sollen:
RETENTION_DAYS="7"
Dateiname der Logdatei spezifizieren:
LOGFILE="/tmp/dd_bak.log"
Durchführung:
echo "START-BACKUP" > $LOGFILE
echo $(date) >> $LOGFILE
WEEKDAY=$(date +%u )
TIMESTAMP=$(date +%Y-%m-%d_%H-%M-%S_WD$WEEKDAY)
echo "START-dd" >> $LOGFILE
dd if=${SOURCE} of=${TARGET}/$HOSTNAME-$TIMESTAMP.img bs=512
echo "END-dd" >> $LOGFILE
Ext4 formatieren
#mkfs ext4 -F ${TARGET}/$HOSTNAME.$TIMESTAMP.img
Alte Backups löschen
echo "Remove retended images" >> $LOGFILE
find ${TARGET}/$HOSTNAME-*.img -mtime +$RETENTION_DAYS -type f -delete
Entfernt auch -shrinked und -truncated
#find ${TARGET}/$HOSTNAME--shrinked.img -mtime +$RETENTION_DAYS -type f -delete
#find ${TARGET}/$HOSTNAME--truncated.img -mtime +$RETENTION_DAYS -type f -delete
Shrinked Images erstellen, um nicht allokierten Speicher für kleinere SD-Karten zu entfernen
#dd if=${SOURCE} bs=512 conv=sparse | gzip -c > ${TARGET}/$HOSTNAME-$TIMESTAMP-shrinked.img
TRUNCATED=${TARGET}/$HOSTNAME-$TIMESTAMP-truncated.img
cp ${TARGET}/$HOSTNAME-$TIMESTAMP.img $TRUNCATED
echo "Copy to truncated file" >> LOGFILE
partinfo=parted -m $TRUNCATED unit B print
echo $partinfo
partnumber=echo "$partinfo" | grep ext4 | awk -F: ' { print $TRUNCATED } '
echo $partnumber
partstart=echo "$partinfo" | grep ext4 | awk -F: ' { print substr($2,0,length($2)-1) } '
echo $partstart
loopback=losetup -f --show -o $partstart $TRUNCATED
echo $loopback
e2fsck -f $loopback -y
minsize=resize2fs -P $loopback | awk -F': ' ' { print $2 } '
minsize=echo $minsize+1000 | bc
resize2fs -p $loopback $minsize
sleep 1
losetup -d $loopback
partnewsize=echo "$minsize * 4096" | bc
newpartend=echo "$partstart + $partnewsize" | bc
part1=parted $TRUNCATED rm 2
part2=parted $TRUNCATED unit B mkpart primary $partstart $newpartend
endresult=parted -m $TRUNCATED unit B print free | tail -1 | awk -F: ' { print substr($2,0,length($2)-1) } '
truncate -s $endresult $TRUNCATED
echo $(date) >> $LOGFILE
echo "END-BACKUP" >> $LOGFILE
exit 0

Please forgot the first attempt to try creating a shrinked image with gzip because it only will compress an .img file but if you want to write it to your SD card, it will fail because the SC card could be smaller than the uncompressed .img file.

So the interesting part is:

partinfo=`parted -m $TRUNCATED unit B print`
echo $partinfo
partnumber=echo "$partinfo" | grep ext4 | awk -F: ' { print $TRUNCATED } '
echo $partnumber
partstart=echo "$partinfo" | grep ext4 | awk -F: ' { print substr($2,0,length($2)-1) } '
echo $partstart
loopback=losetup -f --show -o $partstart $TRUNCATED
echo $loopback
e2fsck -f $loopback -y
minsize=resize2fs -P $loopback | awk -F': ' ' { print $2 } '
minsize=echo $minsize+1000 | bc
resize2fs -p $loopback $minsize
sleep 1
losetup -d $loopback
partnewsize=echo "$minsize * 4096" | bc
newpartend=echo "$partstart + $partnewsize" | bc
part1=parted $TRUNCATED rm 2
part2=parted $TRUNCATED unit B mkpart primary $partstart $newpartend
endresult=parted -m $TRUNCATED unit B print free | tail -1 | awk -F: ' { print substr($2,0,length($2)-1) } '
truncate -s $endresult $TRUNCATED

If I run it inside a Ubunut virtual machine or server it works to compress one of the created .img files. I get as example following output:

root@<nothing_to_see># TRUNCATED=truncated3.img
root@<nothing_to_see># echo $TRUNCATED
truncated3.img
root@<nothing_to_see># partinfo=`parted -m $TRUNCATED unit B print`
root@<nothing_to_see># echo $partinfo
BYT; /<nothing_to_see>/truncated3.img:15552479232B:file:512:512:msdos::; 1:4194304B:48027135B:43832832B:fat32::lba; 2:50331648B:15552479231B:15502147584B:ext4::;
root@<nothing_to_see># partnumber=`echo "$partinfo" | grep ext4 | awk -F: ' { print $TRUNCATED } '`
root@<nothing_to_see># echo $partnumber
2:50331648B:15552479231B:15502147584B:ext4::;
root@shlServer01:/home/smarthome/Schreibtisch# partstart=`echo "$partinfo" | grep ext4 | awk -F: ' { print substr($2,0,length($2)-1) } '`
root@<nothing_to_see># echo $partstart
50331648
root@<nothing_to_see># loopback=`losetup -f --show -o $partstart $TRUNCATED`
root@<nothing_to_see># echo $loopback
/dev/loop2
root@<nothing_to_see># losetup -a
/dev/loop0: [2304]:325081 (/var/lib/snapd/snaps/core_10583.snap)
/dev/loop1: [2304]:323253 (/var/lib/snapd/snaps/core_10823.snap)
/dev/loop2: [2305]:27525560 (/home/smarthome/Schreibtisch/truncated3.img), offset 50331648
root@<nothing_to_see># sudo fdisk -l /dev/loop2
Disk /dev/loop2: 14,4 GiB, 15502147584 bytes, 30277632 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
root@<nothing_to_see># e2fsck -f $loopback -y
e2fsck 1.42.13 (17-May-2015)
rootfs: recovering journal
Pass 1: Checking inodes, blocks, and sizes
Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
Pass 5: Checking group summary information
Block bitmap differences:  +(11535--11548) +(458752--459741) +(459744--459774) +(460992--461039) +(461056--461103) +(461120--461158) +(461184--461220) +(461248--461285) +(461312--461333) +(461344--461361) +(461376--461413) +(461440--461467) +(461472--461496) +(461504--461542) +(461568--461602) +(461632--461688) +(461696--461729) +(461760--461797) +(461824--461862) +(461888--461924) +(461952--461988) +(462016--462063) +(462080--462114) +(462144--462172) +(462176--462202) +(462208--462256) +(462272--462312) +(462336--462369) +(462400--462432) +(462464--462497) +(462528--462575) +(462592--462610) +(462624--462673) +(462688--462709) +(462720--462738) +(462752--462781) +(462784--462823) +(462848--462933) +(462944--462974) +(462976--463075) +(463104--463261) +(463264--463281) +(463296--463318) +(463328--463348) +(463360--463400) +(463424--463478) +(463488--463594) +(463616--463632) +(463648--463674) +(463680--463725) +(463808--463845) +(463872--464143) +(464160--464180) +(464192--464231) +(464256--464272) +(464288--464307) +(464320--464359) +(464384--464535) +(464544--464570) +(464576--464621) +(464640--464832) +(464864--464883) +(464896--465088) +(465120--465140) +(465280--465297) +(465312--465339) +(465344--465376) +(465792--465850) +(465856--465893) +(465920--465937) +(465952--465969) +(465984--466001) +(466016--466045) +(466048--466173) +(466176--466240) +(466272--466296) +(466304--466425) +(466432--466514) +(466528--466544) +(466560--466664) +(466688--466867) +(466880--466937) +(466944--467091) +(467104--467125) +(467136--467195) +(467200--467337) +(467360--467385) +(467392--467452) +(467456--467600) +(467616--467632) +(467648--467708) +(467712--467829) +(467840--467857) +(467872--467889) +(467904--467920) +(467936--467952) +(467968--467991) +(468000--468029) +(468032--468093) +(468096--468190) +(468192--468215) +(468224--468320) +(468352--468436) +(468448--468471) +(468480--468591) +(468608--468706) +(468736--468835) +(468864--468980) +(468992--469046) +(469056--469109) +(469120--469230) +(469248--469345) +(469376--469496) +(469504--469642) +(469664--469694) +(469696--469854) +(469856--469879) +(469888--470001) +(470016--470162) +(470176--470205) +(470208--470250) +(470272--470424) +(470432--470462) +(470464--470680) +(470688--470928) +(470944--470968) +(470976--471038) +(471040--471192) +(471200--471230) +(471232--471275) +(471296--471431) +(471456--471479) +(471488--471539) +(471552--471686) +(471712--471736) +(471744--471788) +(471808--471953) +(471968--471993) +(472000--472034) +(472064--472219) +(472224--472241) +(472256--472290) +(472320--472561) +(472576--472728) +(472736--472765) +(472768--472800) +(472832--473032) +(473056--473079) +(473088--473230) +(473248--473267) +(473280--473312) +(473344--473487) +(473504--473534) +(473536--473570) +(473600--473711) +(473728--473850) +(473856--474051) +(474080--474228) +(474240--474329) +(474336--474363) +(474368--474503) +(474528--474598) +(474624--474807) +(474816--474835) +(474848--474877) +(474880--475040) +(475072--475090) +(475104--475133) +(475136--475268) +(475296--475324) +(475328--475344) +(475360--475389) +(475392--475537) +(475552--475580) +(475648--475761) +(475776--475840) +(475872--475890) +(475904--476001) +(476032--476143) +(476160--476380) +(476384--476402) +(476416--476457) +(476480--476502) +(476512--476529) +(476544--476564) +(476576--476599) +(476608--476638) +(476640--476660) +(483552--483574) +(483584--483723) +(483744--483760) +(483776--483815) +(484192--484214) +(484224--484258) +(484288--484344) +(484640--484662) +(484864--484964) +(484992--485026) +(485376--485912) +(486400--486836) +(496640--497429) +(497440--497463) +(497472--497514) +(517632--517883) +(520192--524287) +1233891 -1238111
Fix? yes
Free blocks count wrong (2601792, counted=2601510).
Fix? yes
Free inodes count wrong (715854, counted=715841).
Fix? yes
rootfs: ***** FILE SYSTEM WAS MODIFIED *****
rootfs: 199167/915008 files (0.1% non-contiguous), 1183194/3784704 blocks
root@<nothing_to_see># minsize=resize2fs -P $loopback | awk -F': ' ' { print $2 } '
resize2fs 1.42.13 (17-May-2015)
root@<nothing_to_see># minsize=echo $minsize+1000 | bc
root@<nothing_to_see># echo $minsize
1348728
root@<nothing_to_see># resize2fs -p $loopback $minsize
resize2fs 1.42.13 (17-May-2015)
Resizing the filesystem on /dev/loop2 to 1348728 (4k) blocks.
Begin pass 2 (max = 441148)
Relocating blocks             XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
Begin pass 3 (max = 116)
Scanning inode table          XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
Begin pass 4 (max = 28993)
Updating inode references     XXXXXXXXXXXXXXXXXXXXXXXXXX--------------
...I think there we don not have to wait until it will be finished...

The problem is earlier. I think if e2fsck makes problems later resize2fs would also make problems. On the Raspberry Pi I get following error after running e2fsck:

e2fsck 1.43.4 (31-Jan-2017)
ext2fs_open2: Bad magic number in super-block
e2fsck: Superblock invalid, trying backup blocks...
e2fsck: Bad magic number in super-block while trying to open /dev/loop1
The superblock could not be read or does not describe a valid ext2/ext3/ext4
filesystem.  If the device is valid and it really contains an ext2/ext3/ext4
filesystem (and not swap or ufs or something else), then the superblock
is corrupt, and you might try running e2fsck with an alternate superblock:
    e2fsck -b 8193 <device>
 or
    e2fsck -b 32768 <device>

It has nothing to do with the version 1.43.4 and 1.42.13 of the e2fsprogs. I have installed on one of my Raspberry Pis also the older 1.42.13 version. (Maybe updating to a newer version could work, but the old version on the virtual machine does work).

I will give following informations:

echo $loopback
/dev/loop1
losetup -a
/dev/loop1: [0036]:163852 (/mnt/BACKUP/automatic/images/shlPiBad/shlPiBad-2021-02-16_01-00-01_WD2.img), offset 5033164
fdisk -l $loopback
Disk /dev/loop1: 14.9 GiB, 15957438976 bytes, 31166873 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes

So it will make the same until e2fsck is called. If /dev/loop0 oder /dev/loop* is not the problem. Also fine. And if you have a closer look you would also recognize that it tested it on different image files. But you can believe me, creating the .img files on each day with different Raspberry Pis will work fine. Only shrinking it will not work.

The bash script will be executed from each Raspberry Pi each day at 01:00 because I use a cronjob. So everything could be executed with sudo rights. Well, you also see, that this is not the point. And yes you see a few times echo more because I was at first not sure if mounting could be the problem. But the .img gets sucessfully mounted.

One problem could be that e2fsprogs forbids to change something inside rootfs because the mounted and the running image will have it. If you know what I mean.

One solution could be, that the server have to shrink it because it is not possible on the Raspberry Pi. But then I would want to know and learn the reason behind it.

Thanks in advance.

Answer 1

This Question is confusing. You have posted code that doesn't work and code that "works" but doesn't do what you want.
It is unclear what this code is attempting to do but you can't resize a mounted partition.

Someone may be able to help, but your whole approach is wrong.

You are backing up a full image then attempting to shrink it.
This is slow and will result in an image containing lots of useless data.

There are a number of backup techniques.
The approach I use is an incremental backup to a small image, although this could be used to make fresh minimal sized backups.

This is fast, and avoids including any unnecessary data.

See Backup image of SD Card