I got the Pi B+ and the Pi camera and am now trying to find the most efficient (low CPU) and lowest-latency configuration to stream H.264 encoded video from the camera to my home server.
I've read the following:
(All links use gstreamer-1.0 from deb http://vontaene.de/raspbian-updates/ . main
.)
A lot has been done in this regard in the past years.
Originally, we had to pipe the output of raspivid
into gst-launch-1.0
(see link 1).
Then (link 2) the official V4L2 driver was created which is now standard, and it allows to directly obtain the data without a pipe, using just gstreamer (see especially the post by towolf » Sat Dec 07, 2013 3:34 pm in link 2):
Sender (Pi): gst-launch-1.0 -e v4l2src do-timestamp=true ! video/x-h264,width=640,height=480,framerate=30/1 ! h264parse ! rtph264pay config-interval=1 ! gdppay ! udpsink host=192.168.178.20 port=5000
Receiver: gst-launch-1.0 -v udpsrc port=5000 ! gdpdepay ! rtph264depay ! avdec_h264 ! fpsdisplaysink sync=false text-overlay=false
If I understand correctly, both ways use the GPU to do the H264 decoding, but the latter is a bit mor efficient since it doesn't need to go through the kernel another time since there's no pipe between processes involved.
Now I have some questions about this.
Is the latter still the most recent way to efficiently get H264 from the camera? I've read about
gst-omx
, which allows gstreamer pipelines like... video/x-raw ! omxh264enc ! ...
. Does this do anything different to just usingvideo/x-h264
, or might it even be more efficient? What's the difference?How do I find out what gstreamer encoding plugin is actually used when I use the
video/x-h264 ...
pipeline? This seems to be just specifying the format I want, as compared to the other pipeline parts, where I explicitly name the (code) component (likeh264parse
orfpsdisplaysink
).In this reply to link 1 Mikael Lepistö mentions "I removed one unnecessary filter pass from streaming side", meaning that he cut out the
gdppay
andgdpdepay
. What do those do? Why are they needed? Can I really strip them off?He also mentions that by specifying
caps="application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)H264, payload=(int)96"
parameters for theudpsrc
at the receiving side, he's able to start/resume the streaming in the middle of the stream. What do these caps achieve, why these specific choices, where can I read more about them?When I do what's suggested in question 3 and 4 (adding the
caps
, droppinggdppay
andgdpdepay
) then my video latency becomes much worse (and seems to be accumulating, the latency increases over time, and after a few minutes the video stops)! Why could that be? I would like to get the latency I obtained with the original command, but also have the feature of being able to join the stream at any time.I've read that RTSP+RTP usually use a combination of TCP and UDP: TCP for control messages and other things that mustn't get lost, and UDP for the actual video data transmission. In the setups above, am I actually using that, or am I just using UDP only? It's a bit opaque to me whether gstreamer takes care of this or not.
I would appreciate any answer to even a single one of these questions!
|
creates any issue in this context is an incredible piece of B.S. Have you tried anyraspivid | cvlc
methods? I haven't had the camera for very long or much time to play with it, but using that to produce an http stream (viewable on linux at the other end w/vlc
) seems to work okay. – goldilocks Jan 10 '15 at 11:30cat file | grep ...
instead ofgrep ... file
. The pipe adds another layer of copying to and from the kernel, which is easily measureable, especially on devices with low memory bandwidth. If gstreamer can read from the device file direcly, why not use that? Regarding yourraspivid | cvlc
suggestion: I was using this before I switched to the gstreamer based solution, it has up to 3 seconds more latency than gstreamer (I don't know why). – nh2 Jan 10 '15 at 12:19cvlc
uses ~45%, but just running through a pipe at that data rate (keeping in mind again, the pipe is not slowing it down) would barely move the needle, I think. Like <5%. It's not totally insignificant if you want to do this as efficiently as possible of course... – goldilocks Jan 10 '15 at 13:19raspivid | cvlc
one and that's 40-50%. People may respond better to a question that challenges them to improve on a specific figure. Right now you're asking a lot of why, without explaining why each why is significant. – goldilocks Jan 10 '15 at 13:23v4l2src
, which works well so fare - it's the other bits in the gstreamer pipeline that don't work / that I don't understand. – nh2 Jan 10 '15 at 13:55... video/x-raw ! omxh264enc ! ...
andvideo/x-h264
are actually identical. 2: Would help me answer 1 myself. 3: I guess this question needs no explanation. 4: This lets me (re)connect the video at any time, which I need. 5: It solves reconnection, but has much higher latency. I need to know why. 6: To understand what methods I'm actually using right now. – nh2 Jan 10 '15 at 13:59gdppay
: https://github.com/thaytan/gst-rpicamsrc/issues/20 – nh2 May 20 '15 at 14:35gst-rpicamsrc
is a gstreamer element that can can get H264 encoded video from the Raspicam directly into gstreamer with low overhead - I'm using that now, and it works well with low latency. This in combination with thegst-rtsp-server
for serving it to RTSP clients - it uses TCP to control the connection and UDP to send the data. See this: http://cgit.freedesktop.org/gstreamer/gst-rtsp-server/tree/examples/test-launch.c?id=1.4.5 – nh2 May 20 '15 at 14:37