I've spent so much time trying to figure out how this FFMPEG and their libraries work :(
I have a RaspberryPi 2B and a Logitech USB Webcam. Here is the input from FFMPEG list from my /dev/video0:
ffmpeg version 3.0.2-2~bpo8+1 Copyright (c) 2000-2016 the FFmpeg developers
built with gcc 4.9.2 (Debian 4.9.2-10)
configuration: --prefix=/usr --extra-version='2~bpo8+1' --toolchain=hardened --libdir=/usr/lib/arm-linux-gnueabihf --incdir=/usr/include/arm-linux-gnueabihf --cc=cc --cxx=g++ --enable-gpl --enable-shared --disable-stripping --disable-decoder=libopenjpeg --disable-decoder=libschroedinger --enable-avresample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libmodplug --enable-libmp3lame --enable-libopenjpeg --enable-libopus --enable-libpulse --enable-librubberband --enable-librtmp --enable-libschroedinger --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxvid --enable-libzvbi --enable-openal --enable-opengl --enable-x11grab --enable-libdc1394 --enable-libiec61883 --enable-libzmq --enable-frei0r --enable-chromaprint --enable-libx264
libavutil 55. 17.103 / 55. 17.103
libavcodec 57. 24.102 / 57. 24.102
libavformat 57. 25.100 / 57. 25.100
libavdevice 57. 0.101 / 57. 0.101
libavfilter 6. 31.100 / 6. 31.100
libavresample 3. 0. 0 / 3. 0. 0
libswscale 4. 0.100 / 4. 0.100
libswresample 2. 0.101 / 2. 0.101
libpostproc 54. 0.100 / 54. 0.100
[video4linux2,v4l2 @ 0x1171bb0] Raw : yuyv422 : YUYV 4:2:2 : 640x480 160x90 160x120 176x144 320x180 320x240 352x288 432x240 640x360 800x448 800x600 864x480 960x720 1024x576 1280x720 1600x896 1920x1080 2304x1296 2304x1536
[video4linux2,v4l2 @ 0x1171bb0] Compressed: h264 : H.264 : 640x480 160x90 160x120 176x144 320x180 320x240 352x288 432x240 640x360 800x448 800x600 864x480 960x720 1024x576 1280x720 1600x896 1920x1080
[video4linux2,v4l2 @ 0x1171bb0] Compressed: mjpeg : Motion-JPEG : 640x480 160x90 160x120 176x144 320x180 320x240 352x288 432x240 640x360 800x448 800x600 864x480 960x720 1024x576 1280x720 1600x896 1920x1080
/dev/video0: Immediate exit requested
and here is the audio input:
**** List of CAPTURE Hardware Devices ****
card 1: C920 [HD Pro Webcam C920], device 0: USB Audio [USB Audio]
Subdevices: 1/1
Subdevice #0: subdevice #0
I've tried this common and basic ffmpeg routine:
ffmpeg -y -f alsa -i hw:1 -i /dev/video0 -t 10 out12.avi
and here is the output:
ffmpeg version 3.0.2-2~bpo8+1 Copyright (c) 2000-2016 the FFmpeg developers
built with gcc 4.9.2 (Debian 4.9.2-10)
configuration: --prefix=/usr --extra-version='2~bpo8+1' --toolchain=hardened --libdir=/usr/lib/arm-linux-gnueabihf --incdir=/usr/include/arm-linux-gnueabihf --cc=cc --cxx=g++ --enable-gpl --enable-shared --disable-stripping --disable-decoder=libopenjpeg --disable-decoder=libschroedinger --enable-avresample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libmodplug --enable-libmp3lame --enable-libopenjpeg --enable-libopus --enable-libpulse --enable-librubberband --enable-librtmp --enable-libschroedinger --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxvid --enable-libzvbi --enable-openal --enable-opengl --enable-x11grab --enable-libdc1394 --enable-libiec61883 --enable-libzmq --enable-frei0r --enable-chromaprint --enable-libx264
libavutil 55. 17.103 / 55. 17.103
libavcodec 57. 24.102 / 57. 24.102
libavformat 57. 25.100 / 57. 25.100
libavdevice 57. 0.101 / 57. 0.101
libavfilter 6. 31.100 / 6. 31.100
libavresample 3. 0. 0 / 3. 0. 0
libswscale 4. 0.100 / 4. 0.100
libswresample 2. 0.101 / 2. 0.101
libpostproc 54. 0.100 / 54. 0.100
Guessed Channel Layout for Input Stream #0.0 : stereo
Input #0, alsa, from 'hw:1':
Duration: N/A, start: 1466107437.084808, bitrate: 1024 kb/s
Stream #0:0: Audio: pcm_s16le, 32000 Hz, 2 channels, s16, 1024 kb/s
Input #1, video4linux2,v4l2, from '/dev/video0':
Duration: N/A, start: 1578.347093, bitrate: 147456 kb/s
Stream #1:0: Video: rawvideo (YUY2 / 0x32595559), yuyv422, 1280x720, 147456 kb/s, 10 fps, 10 tbr, 1000k tbn, 1000k tbc
Output #0, avi, to 'out12.avi':
Metadata:
ISFT : Lavf57.25.100
Stream #0:0: Video: mpeg4 (FMP4 / 0x34504D46), yuv420p, 1280x720, q=2-31, 200 kb/s, 10 fps, 10 tbn, 10 tbc
Metadata:
encoder : Lavc57.24.102 mpeg4
Side data:
unknown side data type 10 (24 bytes)
Stream #0:1: Audio: mp3 (libmp3lame) (U[0][0][0] / 0x0055), 32000 Hz, stereo, s16p
Metadata:
encoder : Lavc57.24.102 libmp3lame
Stream mapping:
Stream #1:0 -> #0:0 (rawvideo (native) -> mpeg4 (native))
Stream #0:0 -> #0:1 (pcm_s16le (native) -> mp3 (libmp3lame))
Press [q] to stop, [?] for help
[alsa @ 0xe9ec10] Thread message queue blocking; consider raising the thread_queue_size option (current value: 8)
frame= 4 fps=0.0 q=2.0 size= 154kB time=00:00:00.40 bitrate=3152.5kbits/sframe= 8 fps=7.7 q=12.0 size= 261kB time=00:00:00.80 bitrate=2670.1kbits/frame= 13 fps=8.3 q=21.4 size= 303kB time=00:00:01.30 bitrate=1911.3kbits/frame= 18 fps=8.7 q=31.0 size= 349kB time=00:00:01.80 bitrate=1586.8kbits/frame= 23 fps=8.8 q=31.0 size= 393kB time=00:00:02.30 bitrate=1401.3kbits/frame= 28 fps=9.0 q=31.0 size= 428kB time=00:00:02.80 bitrate=1253.2kbits/frame= 34 fps=9.4 q=31.0 size= 463kB time=00:00:03.40 bitrate=1116.7kbits/frame= 38 fps=9.2 q=31.0 size= 494kB time=00:00:03.80 bitrate=1065.3kbits/frame= 43 fps=9.3 q=31.0 size= 520kB time=00:00:04.30 bitrate= 990.6kbits/frame= 49 fps=9.5 q=24.8 size= 557kB time=00:00:04.90 bitrate= 931.1kbits/frame= 55 fps=9.6 q=31.0 size= 593kB time=00:00:05.50 bitrate= 883.7kbits/frame= 61 fps=9.7 q=24.8 size= 631kB time=00:00:06.10 bitrate= 847.9kbits/frame= 67 fps=9.8 q=31.0 size= 670kB time=00:00:06.70 bitrate= 818.7kbits/frame= 72 fps=9.8 q=31.0 size= 696kB time=00:00:07.20 bitrate= 791.4kbits/frame= 78 fps=9.9 q=31.0 size= 727kB time=00:00:07.80 bitrate= 763.8kbits/frame= 83 fps=9.9 q=31.0 size= 760kB time=00:00:08.30 bitrate= 750.1kbits/frame= 88 fps=9.8 q=31.0 size= 801kB time=00:00:08.80 bitrate= 745.4kbits/frame= 93 fps=9.8 q=31.0 size= 840kB time=00:00:09.30 bitrate= 740.1kbits/frame= 98 fps=9.8 q=31.0 size= 874kB time=00:00:09.80 bitrate= 730.4kbits/frame= 100 fps=9.7 q=31.0 Lsize= 890kB time=00:00:10.04 bitrate= 725.9kbits/s speed=0.979x
video:757kB audio:114kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 2.143680%
The output seems fine, but the quality is poor, it's recording at 640x480.
Then, I tried this:
ffmpeg -f alsa -i hw:1 -vcodec h264 -f v4l2 -i /dev/video0 -vcodec copy -y -t 10 out4.mkv
The image is wonderful, but, unfortunately, the audio is recorded with a bit lag :(
All I want is to set the RaspberryPi to record a few seconds with the audio encompassed at the highest resolution possible, or at least some guidance to achieve that.
Many thanks!
ffmpegis just a vastly overrated waste of time -- look intouv4linstead (looks to me like that's whatffmpegis using for the video source anyway). No fuss, no latency, max resolution & framerate, low processor usage, streaming web server, etc. – goldilocks Jun 16 '16 at 20:43mjpeg, (which is, e.g., supported by most web browsers), and I believe that can include audio. – goldilocks Jun 16 '16 at 20:50uv4land didn't find time, but I just left one here; you'll have to adapt that to your context. An issue, I think: You may have to read from/dev/uv4lso whether you can get a output format which can be read streamwise that way with sound I don't know. The web server, BTW, is enabled by default and if you have nothing on port 8080 that's where you'll find it (there's an ugly front page, the stream is/streamor the middle button top row -- dunno if you can make use of that for openCV). – goldilocks Jun 25 '16 at 13:51