Kernel version does not update when using apt update, apt full-upgrade

Question

I want my Rpis to automatically update to the latest kernel pushed out by the Rpi Foundation, so I set up unattended-upgrades. I assume that's been working - but the kernel version hasn't updated since I installed the OS.

Hitting it with sudo apt update and then sudo apt full-upgrade also doesn't get me the latest version of the released kernel. Just says there's no new updates.

I have an Rpi 2B and 3B running "Raspberry Pi OS Lite" and when I SSH in it says the Linux kernel is 5.4.51-v7+ #1333 (version I installed with). It's the default Debian Buster-based OS with minimal packages from https://www.raspberrypi.org/software/operating-systems/ . The current release is on 5.4.83, per the website.

I can see from the release notes https://downloads.raspberrypi.org/raspios_lite_armhf/release_notes.txt that the release is on 5.4.83.

Following these (old) recommendations: Why I can't update kernel version? , How do I upgrade Raspbian? , Stuck with the old kernel

I updated with rpi-update specifying the closest to 5.4.83 kernel on the site linked https://github.com/Hexxeh/rpi-firmware/commits/master (5.4.81, 453e49bdd87325369b462b40e809d5f3187df21d).

That Rpi doesn't respond to SSH now, so that's not the way (update: it's now on 5.4.83 through manual reinstallation since it wasn't accessible and manually copying the rpi-update backups didn't fix it). I'd also like the kernel to update with unattended-upgrades anyway (and not have to rely on rpi-update). Also /etc/apt/sources.list.d/raspi.list has deb http://archive.raspberrypi.org/debian/ buster main and I think that's right.

I've still got one Rpi ready to get bjorked, what am I missing? This seems like a basic thing that should "just work".

Edit 3

Upgraded the 2nd Rpi on 5.4.51 still to unbootable. Whoops that's linux for ya

I tried to get around that "not possible" error by the recommendation https://www.raspberrypi.org/forums/viewtopic.php?t=275994#p1672308

sudo nano /var/lib/dpkg/status

[delete libraspberrypi0 libraspberrypi-bin libraspberrypi-dev libraspberrypi-doc raspberrypi-bootloader raspberrypi-kernel entries manually]

sudo dpkg --configure -a

sudo apt update; sudo apt install --reinstall libraspberrypi0 libraspberrypi-{bin,dev,doc} raspberrypi-bootloader raspberrypi-kernel

sudo reboot

Unfortunately though it did install the 5.4.83 kernel with those commands/file editing, the 2nd Rpi did not successfully boot after the reboot command. That's gonna get the same old reinstall, too.

So I'm not sure why 5.4.51 failed to update, but it looks like 5.4.83 will be able to update since my reinstalled Rpi running that responds without error to the --reinstall command. Will see on next release.

Edit 2

I tried sudo apt install --reinstall raspberrypi-bootloader raspberrypi-kernel on the reinstalled 5.4.83 Rpi and the command worked (obv nothing changed). But, on the old 5.4.51 Rpi it gives:

Reinstallation of raspberrypi-bootloader is not possible, it cannot be downloaded.
Reinstallation of raspberrypi-kernel is not possible, it cannot be downloaded.

Is this the crux of the problem? How do I circumvent this "not possible"?

Edit, here are the repo's the 5.4.51 Rpi hits:

Hit:1 http://archive.raspberrypi.org/debian buster InRelease
Hit:2 http://raspbian.raspberrypi.org/raspbian buster InRelease
Hit:3 http://archive.raspbian.org/raspbian testing InRelease

The reinstalled Rpi on 5.4.83 has the added repo:

Hit:1 http://packages.microsoft.com/repos/code stable InRelease

The testing branch is for wireguard, it was added with the code:

echo "deb http://archive.raspbian.org/raspbian testing main" | sudo tee --append /etc/apt/sources.list.d/testing.list
printf 'Package: *\nPin: release a=testing\nPin-Priority: 50\n' | sudo tee --append /etc/apt/preferences.d/limit-testing

per the guide for installing wireguard on 5.4.XX-based Rpi's: https://www.sigmdel.ca/michel/ha/wireguard/wireguard_02_en.html

Answer 1

This problem happened again when kernel 5.10.17 was released, and I made a 2nd plea for help at Kernel version does not update when using apt update, apt full-upgrade [again]. I managed to figure out what caused the issue and was able to upgrade from 5.4.83 to 5.10.17 successfully; same answer as that 2nd plea follows:

I backtracked commands and programs (deleting and unsetting settings) and trying apt full-upgrade each time until I hit what caused the raspberry-kernel to be removed from the upgrade list.

Turns out it was the testing branch for wireguard in Rpi kernel 5.4.XX, it was added with the code:

echo "deb http://archive.raspbian.org/raspbian testing main" | sudo tee --append /etc/apt/sources.list.d/testing.list
printf 'Package: *\nPin: release a=testing\nPin-Priority: 50\n' | sudo tee --append /etc/apt/preferences.d/limit-testing

per the guide for installing wireguard on 5.4.XX-based Rpi's: https://www.sigmdel.ca/michel/ha/wireguard/wireguard_02_en.html

I deleted the entries in the testing.list and limit-testing and raspberry-kernel showed up in apt full-upgrade, added those entries back it and it disappeared.

I'm not sure why this caused an issue, since the raspbian testing repo had a low priority set so it wouldn't effect anything that was also in main repos. But I barely figured this out, so I won't try to divine what went wrongk in apt's raspberry-kernel upgrade calculations.

Good news is that removing wireguard and the testing repo allowed for upgrading to the (currently) latest 5.10.17 kernel, and the 5.10.17 kernel has wireguard included in the kernel. You need to install wireguard-tools to use wireguard but wireguard-tools has been added to the main repos and can be installed with sudo apt install wireguard-tools without any repo shenanigans. This means that for future kernel upgrades I shouldn't have this issue since I don't have to use the testing branch anymore with the latest 5.10.17 kernel.