GStreamer: commandline encoding from raw to h264

Setup

We assume that the Jetsons have already been setup with basic Gstreamer 1.0 installation. If not, or if you are trying this on a separate computer, you have to install GStreamer1.0 executable files. (Development files are not mandatory at this point.)

To test GStreamer capabilities on commandline, without developing applications and including libgstreamermm, it may be necessary to complete gstreamer's bad-plugins installation.

You can run gst-inspect-1.0 h264parse to check whether you need to install the bad plugins or not. If the inspect h264parse instruction says it cannot find the h264parse element, you need to install the missing gst plugins on the device. The instruction is:

sudo apt-get install gstreamer1.0-plugins-bad

Encoding tests

Assuming there is a raw videofile called ContinuousGrab in your working directory, recorded with Basler's pixel format (YCbCr422_8). See note on pixelformats below.

Begin Note on pixelformats YCbCr filetype documentations: https://www.loc.gov/preservation/digital/formats/fdd/fdd000352.shtml https://www.ptgrey.com/Content/Images/Uploaded/Downloads/TRM/2016/BFS-U3-51S5/HTML/Model/public/ImageFormatControl.html

YCbCr422_8 == YUYV format aka YUY2

GStreamer's videoconvert (not accelerated) takes among other things YUY2 (Basler's YCbCr422_8) as input and can give NV12 and I420 as output.
nvvidconv takes in UYVY (aka Basler's YUV422packed, we think) format and can give NV12 and I420 as output.
omxh264enc encoder takes only NV12 and I420 as inputs, and produces only h264 as output.

Solutions:

Use basler API to convert from YCbCr to YUV422packed in the source app, then pipe into accelerated nvvidconv to convert into NV12 or I420? (What delay does this give and is NV12 better than I420?) NOT POSSIBLE. Basler converter only outputs Mono or RGB.
Pipe YCbCr data into videoconvert to change from YUY2 to I420 or NV12? (What delay does this have? Is NV12 better than I420 for encoding, for speed?) See example below, doing this.
Custom remapping in appsrc, from YUY2 (YUYV) to UYUV, then pipe into nvvidconv? Conversion is literally a swap-around of every even-odd byte... Possible future path, currently unneeded

Encoding tests cont.

This works, when saving raw Basler data (see ContinuousGrab, just basler's produced image buffer without any converter), on jetson: gst-launch-1.0 filesrc location=ContinuousGrab ! videoparse width=1600 height=1200 framerate=24/1 format=4 ! videoconvert ! omxh264enc ! 'video/x-h264, stream-format=(string)byte-stream' ! h264parse ! qtmux ! filesink location=test.mp4 -e

(videoparse format 4 = YUY2, 15 = RGB, 25=gray8, ).

Example with encoder parameters

With YCbCr source (basler camera in YCbCr422_8 capture mode): gst-launch-1.0 filesrc location=ContinuousGrab ! videoparse width=1600 height=1200 framerate=24/1 format=4 ! videoconvert ! omxh264enc bitrate=20000000 profile=8 quality-level=2 control-rate=2 temporal-tradeoff=0 no-B-Frames=false insert-sps-pps=false low-latency=true ! 'video/x-h264, stream-format=(string)byte-stream' ! h264parse ! qtmux ! filesink location=testLowLatNoTempTradeoff.mp4 -e
With RGB8 source (Basler camera in RGB8 capture mode): gst-launch-1.0 filesrc location=ContinuousGrab_RGB8 ! videoparse width=1600 height=1200 framerate=24/1 format=15 ! videoconvert ! omxh264enc bitrate=20000000 profile=8 quality-level=2 control-rate=2 temporal-tradeoff=0 no-B-Frames=false insert-sps-pps=false low-latency=true ! 'video/x-h264, stream-format=(string)byte-stream' ! h264parse ! qtmux ! filesink location=testRGBEncoded.mp4 -e
Launch sequence using lbstreamermm wrapper and control app (we're using this format in our streaming applets): Glib::ustring launch_command = "appsrc name=camerasource ! video/x-raw,width=1600,height=1200,format=YUY2,framerate=24/1 ! videoconvert ! omxh264enc bitrate=20000000 profile=8 quality-level=2 control-rate=2 temporal-tradeoff=0 no-B-Frames=false insert-sps-pps=false low-latency=true ! video/x-h264,stream-format=(string)byte-stream ! h264parse ! qtmux ! filesink location=smurf.mp4"; //"rtph264pay ! udpsink host=127.0.0.1 port=1234";

Update test