Bluetooth Blues, Redux

Question

NOTE: All questions here pertain to Raspberry Pi OS, Lite version

I have asked a similar question here, without getting an answer. Eighteen months have passed, new information has come to light and I have a newer version of Raspberry Pi hardware and software. Unfortunately, the results are much the same, but hope springs eternal.

My objective seems simple: Play music files on my RPi using the CLI, and listen to that music through a Bluetooth speaker. In the interests of keeping this question as brief as possible, my question will focus on getting to first base; i.e. How to establish a solid connection between the RPi and the BT speaker.

It may not be germane to my question, but on another Linux platform (Ubuntu 20.04), it took me about 3-4 minutes to sort this all out from the CLI: discover speaker, pair, connect & play music. That may be an apples-to-oranges comparison, but it seems worth mentioning here.

A summary of my system:

Hardware : BCM2711
Revision : b03111
Firmware : Jun 15 2020 14:36:19 (see below for further details) Model : Raspberry Pi 4 Model B Rev 1.1
Distro : Raspberry Pi OS Lite

pi@raspberrypi4b:~ $ uname -a
Linux raspberrypi4b 5.4.51-v7l+ #1333 SMP Mon Aug 10 16:51:40 BST 2020 armv7l GNU/Linux 
...
pi@raspberrypi4b:~ $ vcgencmd bootloader_version
Jun 15 2020 14:36:19
version c302dea096cc79f102cec12aeeb51abf392bd781 (release)
timestamp 1592228179
...
pi@raspberrypi4b:~ $ dpkg -l | grep blue
ii  bluez                          5.50-1.2~deb10u1+rpt2               armhf        Bluetooth tools and daemons
ii  bluez-firmware                 1.2-4+rpt5                          all          Firmware for Bluetooth devices
ii  pi-bluetooth                   0.1.15                              all          Raspberry Pi 3 bluetooth

I have assumed these are all the tools needed by RPi to establish a connection to the speaker. But perhaps this is my first error? Can anyone confirm this?

Connection attempts:

Here's what I've tried, and some questions:

hcitool discovery:

$ hcitool scan
Scanning ...
    B8:F6:53:12:13:F1   JBL Flip 5

Note that after powering the BT speaker up & pressing the BT button on it, I waited several seconds before starting the scan. The scan had to be repeated ~10 times before the BT speaker JBL Flip 5 was discovered.

bluetoothctl pairing and connection:

pi@raspberrypi4b:~ $ bluetoothctl
[bluetooth]# agent on
Agent registration enabled
[bluetooth]# scan on
No default controller available
[bluetooth]#

Note that bluetoothctl does not seem to be aware of its own default controller. Some of the references (see below) show the default controller responsive immediately after entering bluetoothctl. Other references indicate that bluetoothctl must be run w/ sudo. It's unclear why this is necessary, but it does make a difference:

$ sudo bluetoothctl
Agent registered
[bluetooth]# devices
[bluetooth]# list
Controller DC:A6:32:02:F0:97 raspberrypi4b [default]
...
$ bluetoothctl show
Controller DC:A6:32:02:F0:97 (public)
Name: raspberrypi4b
Alias: raspberrypi4b
Class: 0x00000000
Powered: yes
Discoverable: no
Pairable: yes
UUID: Generic Attribute Profile (00001801-0000-1000-8000-00805f9b34fb)
UUID: A/V Remote Control (0000110e-0000-1000-8000-00805f9b34fb)
UUID: PnP Information (00001200-0000-1000-8000-00805f9b34fb)
UUID: A/V Remote Control Target (0000110c-0000-1000-8000-00805f9b34fb)
UUID: Generic Access Profile (00001800-0000-1000-8000-00805f9b34fb)
Modalias: usb:v1D6Bp0246d0532
Discovering: no
...
[bluetooth]# scan on
Discovery started
...  some time after pressing the BT button on the speaker:
[NEW] Device B8:F6:53:12:13:F1 JBL Flip 5
[bluetooth]# scan off
Discovery stopped
...
[bluetooth]# info B8:F6:53:12:13:F1
Device B8:F6:53:12:13:F1 (public)
    Name: JBL Flip 5
    Alias: JBL Flip 5
    Class: 0x00240414
    Icon: audio-card
    Paired: no
    Trusted: no
    Blocked: no
    Connected: no
    LegacyPairing: no
    UUID: Handsfree                 (0000111e-0000-1000-8000-00805f9b34fb)
    UUID: Headset                   (00001108-0000-1000-8000-00805f9b34fb)
    UUID: Audio Sink                (0000110b-0000-1000-8000-00805f9b34fb)
    UUID: Advanced Audio Distribu.. (0000110d-0000-1000-8000-00805f9b34fb)
    UUID: A/V Remote Control        (0000110e-0000-1000-8000-00805f9b34fb)
[bluetooth]# pair B8:F6:53:12:13:F1
Attempting to pair with B8:F6:53:12:13:F1
[CHG] Device B8:F6:53:12:13:F1 Connected: yes
[CHG] Device B8:F6:53:12:13:F1 UUIDs: 00001101-0000-1000-8000-00805f9b34fb
[CHG] Device B8:F6:53:12:13:F1 UUIDs: 0000110b-0000-1000-8000-00805f9b34fb
[CHG] Device B8:F6:53:12:13:F1 UUIDs: 0000110c-0000-1000-8000-00805f9b34fb
[CHG] Device B8:F6:53:12:13:F1 UUIDs: 0000110e-0000-1000-8000-00805f9b34fb
[CHG] Device B8:F6:53:12:13:F1 ServicesResolved: yes
[CHG] Device B8:F6:53:12:13:F1 Paired: yes
Pairing successful

but immediately thereafter:

[CHG] Device B8:F6:53:12:13:F1 ServicesResolved: no
[CHG] Device B8:F6:53:12:13:F1 Connected: no
[bluetooth]# connect B8:F6:53:12:13:F1
Attempting to connect to B8:F6:53:12:13:F1
Failed to connect: org.bluez.Error.Failed

Repeated attempts to restore the connection failed as above. This pattern of a successful pairing & connection, immediately followed by [CHG] to no/failure has been repeated several times.

The Questions:

1. Why am I unable to maintain a stable BT connection between the RPi and speaker?

2. Why does bluetoothctl require root privileges to start properly - is this part of the problem?

My research:

• This Q&A seemed very similar, though I don't need to automate it.

• Several other Q&A and tutorials have been reviewed 1, 2, 3, 4, 5, 6, 7

It strikes me that AFAICT no two of the "references" in this collection have anything in common!

Answer 1

This is not an answer - it's a sequence of journal entries, or just a rambling blog if you prefer!

This doesn't meet SE standards for an answer. Yet, the objective stated in the question above has been met through the steps listed below. I'm posting this as a "prototype answer", which I'll define as a container for information that is useful and pertinent to a proper SE answer. Once a proper SE answer is posted, this material will be moved to my GitHub page. ICYI, further details are posted in a chat session between myself and @ukBaz - who should get credit for the positive result.

0. Is Bluetooth Crippled in the Lite version Raspbian/RPi OS?

@ukBaz brought this up in the discussion, and now with the benefit of hindsight, it seems clear that he's correct. In addition to the approach outlined below, other "solutions" have been posted that require installation of the pulseaudio-module-bluetooth package. Early on, I couldn't imagine why a package that's 670 MB - with ~100 dependencies, including several X libraries - would be required to play a music file over a BT link. But it seems RPi OS Lite must have PulseAudio to enable music-over-Bluetooth! It has been said the ALSA package is an alternative to PulseAudio, but that hasn't been demonstrated. Finally, it is disappointing that the functional limits of the RPi OS Lite distro aren't covered in the official documentation. Analysis and comments are welcome.

1. Are root privileges required to start bluetoothctl? (Partial answer)

For reasons that are presently unclear, the pi user's privileges are different on a Raspbian Desktop installation than they are on a Raspbian Lite installation! For example, user pi on a Desktop RPi system is able to start and control the default Bluetooth controller using bluetoothctl, but not on a Lite RPi system.

As a solution, some posts online suggest starting bluetoothctl with sudo (a.k.a. root privileges). That may be one way to do it, but other posts suggest that user pi be added to the supplementary group named bluetooth.

You can verify this:

1.1 On Raspbian Desktop, using the CLI:

pi@raspberrypi4b:~ $ bluetoothctl
Agent registered
[bluetooth]# list
Controller DC:A6:32:01:02:03 raspberrypi4b [default]

The Controller is active & responsive

1.2 On Raspbian Lite, using the CLI:

pi@raspberrypi4b:~ $ bluetoothctl
[bluetooth]# list
No default controller available

The Controller is non-responsive

1.3 The Resolution:

Add user pi to group bluetooth:

pi@raspberrypi4b:~ $ sudo usermod -G bluetooth -a pi  

Verify:

getent group | grep bluetooth
bluetooth:x:112:pi

Following a reboot, user pi should find bluetoothctl and the Controller responsive without resort to sudo privilege elevation. Analysis and comments are welcome.

2. Error: Failed to connect: org.bluez.Error.Failed (Partial Answer)

This error is displayed in the bluetoothctl dialog. This error has now been resolved, but will be re-visited in the future. and is currently un-resolved. The error is displayed after the BT Speaker is trusted and paired with the controller (see command sequence in the Question above). It's a meaningless error message in the sense that it provides no clues for troubleshooting.

Based on another useful suggest from @ukBaz, the btmon utility was enlisted in an effort to get useful clues for the RPi's failure to make a successful connection to the Bluetooth speaker:

$ sudo btmon -t
Bluetooth monitor ver 5.50
= Note: Linux version 5.4.51-v7l+ (armv7l)
...
= bluetoothd: a2dp-sink profile connect failed for B8:F6:53:12:13:F1: Protocol not available

btmon appears to be a useful tool. The system docs are very lean, but the Ubuntu project has a decent wiki on btmon that is helpful

2.1 a2dp-sink profile connect failed Protocol not available

Using this error as a search term suggested that the solution involved installation of additional software (REF 1, REF2). The name of the software is pulseaudio-module-bluetooth.

Another source, REF 3 suggested that BlueAlsa was a better alternative to PulseAudio. It appears that PulseAudio may be more widely used, so we'll use it for this pass.

2.2 Install pulseaudio-module-bluetooth

Following the obligatory sudo apt update && sudo apt upgrade -y, the trusty apt was set to the installation of this package:

pi@raspberrypi4b:~ $ sudo apt install pulseaudio-module-bluetooth
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following additional packages will be installed:
  dbus-user-session fontconfig libaom0 libasound2-plugins libasyncns0 libavcodec58 libavresample4 libavutil56 libcairo2 libcodec2-0.8.1 libcroco3
...
0 upgraded, 102 newly installed, 0 to remove and 0 not upgraded.
Need to get 46.7 MB of archives.
After this operation, 670 MB of additional disk space will be used.
Do you want to continue? [Y/n] Y

Probably a good idea to reboot at this point.

This seems a staggering amount of disk space (670MB) to stream music files over a Bluetooth link. By comparison, df -H says the entire Raspberry Pi OS (32-bit) Lite installation occupies a bit less than 1,500 MB. That's quite a lot of stuff, and includes numerous dependencies on X window libs - whose utility in the Lite system is unclear. Nevertheless, we'll proceed with this pulseaudio-module-bluetooth approach now, but revisit it for a leaner solution.

The Debian project has an overview page for pulseaudio-module-bluetooth that's an easy read.

2.3 Start the pulseaudio server

If it's not clear, pulseaudio-module-bluetooth is a module in the PulseAudio "family". In fact, installing pulseaudio-module-bluetooth also installed pulseaudio as a "dependency".

For those who are interested, the document 'PulseAudio under the hood' provides a fairly extensive "introduction" to PulseAudio (TL;DR). The Debian overview on PulseAudio is brief & to the point. PulseAudio also has a Wikipedia page that's worth reading. One useful bit of trivia from Wikipedia is that the author of PulseAudio (Lennart Poettering) is also the author of systemd and avahi :0

At a more mundane level, it's also worth noting that the Debian overview on PulseAudio suggests PulseAudio has undergone major changes since jessie. Change is good.

It seems likely that the pulseaudio server would be started by systemd on those distributions that include it. But that's not the case with RPiOS Lite, so we'll do it manually:

pi@raspberrypi4b:~ $ pulseaudio --start

2.4 Test for RPi - Bluetooth connection

Before attempting the connection, start bluetoothctl to see if the Controller has any additional capabilities that may cause a different outcome:

pi@raspberrypi4b:~ pi@raspberrypi4b:~ $ bluetoothctl
Agent registered
[bluetooth]# list
Controller DC:A6:32:02:F0:97 raspberrypi4b [default]
[bluetooth]# show
Controller DC:A6:32:02:F0:97 (public)
    Name: raspberrypi4b
    Alias: raspberrypi4b
    Class: 0x000c0000
    Powered: yes
    Discoverable: no
    Pairable: yes
    UUID: Headset AG                (00001112-0000-1000-8000-00805f9b34fb)
    UUID: Generic Attribute Profile (00001801-0000-1000-8000-00805f9b34fb)
    UUID: A/V Remote Control        (0000110e-0000-1000-8000-00805f9b34fb)
    UUID: Generic Access Profile    (00001800-0000-1000-8000-00805f9b34fb)
    UUID: PnP Information           (00001200-0000-1000-8000-00805f9b34fb)
    UUID: A/V Remote Control Target (0000110c-0000-1000-8000-00805f9b34fb)
    UUID: Audio Source              (0000110a-0000-1000-8000-00805f9b34fb)
    UUID: Audio Sink                (0000110b-0000-1000-8000-00805f9b34fb)
    UUID: Headset                   (00001108-0000-1000-8000-00805f9b34fb)
    Modalias: usb:v1D6Bp0246d0532
    Discovering: no
...
[bluetooth]# devices
Device B8:F6:53:12:13:F1 JBL Flip 5
...
[bluetooth]# info B8:F6:53:12:13:F1
Device B8:F6:53:12:13:F1 (public)
    Name: JBL Flip 5
    Alias: JBL Flip 5
    Class: 0x00240414
    Icon: audio-card
    Paired: yes
    Trusted: yes
    Blocked: no
    Connected: no
    LegacyPairing: no
    UUID: Serial Port               (00001101-0000-1000-8000-00805f9b34fb)
    UUID: Audio Sink                (0000110b-0000-1000-8000-00805f9b34fb)
    UUID: A/V Remote Control Target (0000110c-0000-1000-8000-00805f9b34fb)
    UUID: A/V Remote Control        (0000110e-0000-1000-8000-00805f9b34fb)
    ManufacturerData Key: 0x0057
    ManufacturerData Value:
  31 1f 01 18 8d 00                                1.....
    ServiceData Key: 0000fddf-0000-1000-8000-00805f9b34fb

This looks promising - the Controller (RPi) now has Audio Source and a few other capabilities that may be useful. The BT speaker remains paired and trusted. Let's turn the speaker on & try to connect:

Interestingly, immediately after pushing the power button on the speaker JBL Flip 5, it connected without a request (is this trust or what? :)

[CHG] Device B8:F6:53:12:13:F1 Connected: yes
[JBL Flip 5]# info B8:F6:53:12:13:F1
Device B8:F6:53:12:13:F1 (public)
    Name: JBL Flip 5
    Alias: JBL Flip 5
    Class: 0x00240414
    Icon: audio-card
    Paired: yes
    Trusted: yes
    Blocked: no
    Connected: yes
    LegacyPairing: no
    UUID: Serial Port               (00001101-0000-1000-8000-00805f9b34fb)
    UUID: Audio Sink                (0000110b-0000-1000-8000-00805f9b34fb)
    UUID: A/V Remote Control Target (0000110c-0000-1000-8000-00805f9b34fb)
    UUID: Advanced Audio Distribu.. (0000110d-0000-1000-8000-00805f9b34fb)
    UUID: A/V Remote Control        (0000110e-0000-1000-8000-00805f9b34fb)
    ManufacturerData Key: 0x0057
    ManufacturerData Value:
  31 1f 01 18 8d 00                                1.....
    ServiceData Key: 0000fddf-0000-1000-8000-00805f9b34fb

And so it seems we have made progress! In the next "chapter" below, we'll attempt to actually stream & play music over the speaker - the final objective for this installment.

2.5 The Resolution:

Install pulseaudio-module-bluetooth (and all of its dependencies).

pi@raspberrypi4b:~ sudo apt update && sudo apt upgrade -y
pi@raspberrypi4b:~ sudo apt install pulseaudio-module-bluetooth

Other potential solutions will be evaluated in the future. Analysis and comments are welcome.

3. Stream & play music from RPi to Bluetooth speaker

With a little help from my friends, esp @ukBaz, I am now enjoying Dizzy Gillespie's "Night in Tunisia" being streamed from an RPi 4 "Lite" to the Bluetooth speaker on my desk! Here's the "blow-by-blow" account of this final phase of this "answer":

3.1 mpg321 gives me nothing!

After successfully connecting the Bluetooth stack in PulseAudio on the RPi 4 to a 'JBL Flip 5' speaker, I imagined for a few minutes that I was one step away from nirvana. It wasn't quite that straightforward:

I wanted a command-line music player that could handle MP3-encoded files, and the quaint application mpg321 appeared. Install, review man mpg321, and run:

pi@raspberrypi4b:~ $ sudo apt update && sudo apt upgrade -y
pi@raspberrypi4b:~ $ sudo apt install mpg321
pi@raspberrypi4b:~ $ man mpg321
pi@raspberrypi4b:~ $ mpg321 -g 50 NightTunisia.mp3
... (crickets)

So - no error messages, but no music either!

3.2 connect speaker with cable (aka configure PulseAudio)

And once again, @ukBaz provides the guidance:

How do you select which output (card) the audio is going to?

Does this help? https://wiki.archlinux.org/index.php/PulseAudio/Examples#Set_the_default_output_sink

Indeed it does! Arch Linux has perhaps the best, most up-to-date documentation on all things Linux on the Internet today. The Arch Linux Wiki in particular is a valuable resource! And so, after a couple of tries, it was learned that this is the magic spell that connects the speaker wires from the RPi to the Bluetooth speaker :

pacmd "set-default-source bluez_sink.B8_F6_53_12_13_F1.a2dp_sink.monitor"

Some explanation is in order here as the PulseAudio CLI is incomprehensible to all but the PA-cognoscenti. Unfortunately, I am unable to offer a cogent explanation! Here's the best I can do for now:

The PulseAudio concept of sources and sinks is the first thing that escapes me. While it accomplished the objective, the command pacmd "set-default-source... strikes me as backwards: The RPi should be the source, and the speaker should be the sink!

Pondering this, while perusing man pacmd and man pulse-cli-syntax didn't provide any revelations, but it did lead me to try this:

pi@raspberrypi4b:~ $ pacmd list | grep -e 'index:' -e 'name:'
Default sink name:   bluez_sink.B8_F6_53_12_13_F1.a2dp_sink
Default source name: bluez_sink.B8_F6_53_12_13_F1.a2dp_sink.monitor
...

While this doesn't actually explain anything, it seems apparent that source and sink are software abstractions, and need not have any relevance to the physical world. Yeah - that's weak, but it's all I've got for now, and I am rapidly becoming bored with this :)

3.3 The Resolution:

Now that PulseAudio hopefully has its inputs and outputs connected properly, it's time to give mpg321 another try:

pi@raspberrypi4b:~ $ mpg321 -g 50 NightTunisia.mp3
High Performance MPEG 1.0/2.0/2.5 Audio Player for Layer 1, 2, and 3.
Version 0.3.2-1 (2012/03/25). Written and copyrights by Joe Drew,
now maintained by Nanakos Chrysostomos and others.
Uses code from various people. See 'README' for more!
THIS SOFTWARE COMES WITH ABSOLUTELY NO WARRANTY! USE AT YOUR OWN RISK!
Title   : A Night In Tunisia             Artist : Dizzy Gillespie & Charlie Park
Album   : Town Hall, New York City, June Year    : 2005
Playing MPEG stream from NightTunisia.mp3 ...
MPEG 1.0 layer III, 128 kbit/s, 44100 Hz joint-stereo
ALSA lib pcm.c:2565:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.front
[7:34] Decoding of NightTunisia.mp3 finished.

It works, and the sound quality is impressive! There are numerous reports scattered around the Internet stating that the RPi audio playback on Bluetooth is scratchy, noisy and unacceptable. To my ears, the playback of "Night in Tunisia" sourced from this RPi 4 sounded at least as good as good as the playback from my Ubuntu 20.04 system hosted on a 2011 Macbook Pro. Of course that was 5 days ago, so the setup was a bit longer than the 3-4 minutes required for setup on an Ubuntu Desktop - but who's counting?

Firmware Epilogue:

I'm posting this as an edit because I recall reading that some have speculated the RPi 4 firmware may have something to do with the sound quality issues reported for the RPi. When I posted the Question, the firmware was up-to-date. However, I have just checked, and it seems a new revision has been released since this saga began:

pi@raspberrypi4b:~ $ sudo rpi-eeprom-update
BCM2711 detected
Dedicated VL805 EEPROM detected
*** UPDATE AVAILABLE ***
BOOTLOADER: update available
CURRENT: Mon 15 Jun 13:36:19 UTC 2020 (1592228179)
 LATEST: Thu  3 Sep 12:11:43 UTC 2020 (1599135103)
 FW DIR: /lib/firmware/raspberrypi/bootloader/critical
VL805: update available
CURRENT: 000137ad
 LATEST: 000138a1