2

I had my RPi2 (used as a local web-dev server) crashed after making an update using SSH from a distant computer. During the upgrade, the console started to display there was a bus problem, and after having waited for quite a while I hard-rebooted the Pi, and used a screen to see what happensed Well, I'm stuck with the kernel-panic message mentionned in the title. The Pi is running Raspbian (no NOOBS !).

I found here a post with almost the same issue : Kernel panic-not syncing: VFS: unable to mount root fs on unknown- block(179,6) running Raspbian on top of NOOBS. Unfortunately, the solutions proposed don't work for me.

  1. I can't enter recovery mode using SHIFT
  2. I can edit the cmdline.txt file (either on Windows or Ubuntu), I can replace root=/dev/mmcblk0p2 by root=/dev/mmcblk0p6. However, those changes are not taken into account, and I still have the same message when rebooting.

Just in case, I've testes several power supplies, without success.

How can I force cmdline.txt changes to be taken into account ? Is there another way to proceed in order to get my system back ?

EDIT #1 :

I've been using an Ubuntu live USB version. The USB device has a capacity of 16 Gb, so has the micro SD used with the Raspi. I used fdisk to differentiate both. Here are the results for the Pi micro SD :

Disque /dev/sdc : 14,7 GiB, 15811477504 octets, 30881792 secteurs
Unités : sectors of 1 * 512 = 512 octets
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0x0cf63fa8

Périphérique Amorçage  Start      Fin Secteurs  Size Id Type
/dev/sdc1               8192   131071   122880   60M  c W95 FAT32 (LBA)
/dev/sdc2             131072 30881791 30750720 14,7G 83 Linux

Then, I created a directory sudo mkdir /media/sdcard/ and mounted the first one : sudo mount /dev/sdc1 /media/sdcard/. No surprise here, I get that I already had when inserting the device (automatically mounted by Ubuntu) : the boot part, with cmdline.txt, config.txt, and the overlay directory.

I am interested as you may have guessed by the files contained by sdc2. Hence, I repeated the operation for this volume and I got this :

 sudo mount /dev/sdc2 /media/sdcard/
mount: wrong fs type, bad option, bad superblock on /dev/sdc2,
       missing codepage or helper program, or other error

       In some cases useful info is found in syslog - try
       dmesg | tail or so.

Here is what I got after running dmesg | tail as suggested :

[ 3242.691618] blk_update_request: I/O error, dev sdc, sector 3538944
[ 3242.691633] JBD2: IO error reading journal superblock
[ 3242.691639] EXT4-fs (sdc2): error loading journal
[ 3518.191904] sd 9:0:0:0: [sdc] tag#0 FAILED Result: hostbyte=DID_ERROR driverbyte=DRIVER_SENSE
[ 3518.191910] sd 9:0:0:0: [sdc] tag#0 Sense Key : Hardware Error [current] 
[ 3518.191913] sd 9:0:0:0: [sdc] tag#0 Add. Sense: No additional sense information
[ 3518.191918] sd 9:0:0:0: [sdc] tag#0 CDB: Read(10) 28 00 00 36 00 00 00 00 08 00
[ 3518.191921] blk_update_request: I/O error, dev sdc, sector 3538944
[ 3518.191936] JBD2: IO error reading journal superblock
[ 3518.191941] EXT4-fs (sdc2): error loading journal

I've already tried to "repair" the faulty block yesterday, but my skills are limited. It obviously didn't work.

EDIT #2 :

Ran fsck as suggested. "Reading attempt of a block from the filesystem produced a truncated reading. Execute fsck manually"

ubuntu@ubuntu:~/wpscan$ sudo fsck -a /dev/sdc2
fsck de util-linux 2.27.1
Erreur de lecture du bloc 425984 (La tentative de lecture d'un bloc depuis le système de fichiers a produit une lecture tronquée). 

/dev/sdc2: INCONSISTENCE INATTENDUE ; EXÉCUTEZ fsck MANUELLEMENT.
    (i.e., sans options -a ou -p)

Tried to do it manually, but don't know what the best choices are in my case : shall I ignore the error ? shall I force re-writing ? in case of an invalid log for the superblock, shall I erase it ?

Fafanellu
  • 151
  • 1
  • 4
  • +1 for a well-researched question! More to the topic, do you actually have /dev/mmcblk0p6? Since you're not using NOOBS, perhaps your card is partitioned differently. – Dmitry Grigoryev Aug 16 '16 at 08:36
  • Also keep in mind that a failed upgrade can leave you with an unbootable system with incompatible core libraries: be prepared to have more issues when you fix the mounting bug. Do you happen to remember which packages were upgraded? – Dmitry Grigoryev Aug 16 '16 at 08:41
  • so far, I'm only able to see the /boot directory (with a laptpop running Ubuntu Live USB). I've found many other posts about that but I'm stuck with cmdline.txt : don't know how to force it to take into account the changes I made. Unfortunately I don't remember what packages were being updated when the crash happened. Even if I can't reboot, I hope I can recover some files... – Fafanellu Aug 16 '16 at 09:03
  • Do you have gparted, fdisk or a similar tool on your Live USB? Try running it and check what partitions you have. – Dmitry Grigoryev Aug 16 '16 at 10:13
  • yes, thanks. I edited the initial post with all the information I could grab – Fafanellu Aug 16 '16 at 12:15

2 Answers2

2

EXT4 partition on your SD card is damaged to the point it cannot be mounted. If there are files on it that you can't afford to lose, I suggest you make a copy of your SD card with dd before proceeding.

The most straightforward solution is to repair the filesystem with fsck on your live USB stick:

sudo fsck -a /dev/sdc2

If that doesn't work, you have a few options left:

  • try to copy the image created with dd to another SD card. Maybe yours is faulty
  • try to mount the partition without replaying the journal with sudo mount -t ext4 -o ro,noload /dev/sdc2 /media/sdcard/. That won't repair anything, but you may be able to recover some files.

Note that the fix suggested in this question won't help you. It can only fix things for people who do use NOOBS and have their root filesystem on partition 6, while the boot scripts point to partition 2 by mistake. This is not your case.

Dmitry Grigoryev
  • 27,928
  • 6
  • 53
  • 144
  • You have a bad block on your SD card. Use recovery software to save files from it, then get a new one. – Dmitry Grigoryev Aug 16 '16 at 13:24
  • with sudo mount -t ext4 -o ro,noload /dev/sdc2 /media/sdcard/ I was able to mount the device. However, the interesting directory for me is /var/www/ and it couldn't be read : there was an I/O error ! by the way, the reason why I updated/upgraded the Pi was because Apache didn't behave properly...! – Fafanellu Aug 16 '16 at 15:49
  • @Fafanellu Well, I guess this is as far as you can get without professional data recovery services (and there is no guarantee you will get much further with said services). – Dmitry Grigoryev Aug 16 '16 at 15:58
  • of course :( I wonder what caused this bug. I've seen many cases like mine, with a kernel panic after a Raspi upgrade. Maybe I could go a step further if only I could force root=/dev/mmcblk0p6 – Fafanellu Aug 16 '16 at 16:19
  • @Fafanellu No, that wouldn't help. It can only fix things for people who do use NOOBS and have their root filesystem on partition 6. You don't. – Dmitry Grigoryev Aug 16 '16 at 16:30
  • It's not a software bug. Bad blocks on an SD card indicate it is worn out or defective. – goldilocks Aug 16 '16 at 16:42
0

If you have a recovery.cmdline file you could try editing it by changing init=/init to init=/bin/sh. This will skip the init process and boot you straight into shell and from there you might be able to do what is needed to recover.