WIP: omxh264enc: support ARGB/ABGR input
This is just a POC at this stage, I have basically no clue what
I'm doing here. This compiles and it allows me to replace a slow
on-CPU videoconvert
with a faster on-GPU glcolorconvert
in
a complex pipeline running on the RPi3b+.
With videoconvert before this patch:
gst-launch-1.0 gltestsrc num-buffers=128 \
! queue ! gldownload \
! videoconvert \
! omxh264enc control-rate=1 target-bitrate=1145000 \
! video/x-h264,profile=baseline ! h264parse \
! avimux ! filesink location=test.avi
With glcolorconvert after this patch:
gst-launch-1.0 gltestsrc num-buffers=128 \
! glcolorconvert ! "video/x-raw(memory:GLMemory),format=ABGR" \
! queue ! gldownload \
! omxh264enc control-rate=1 target-bitrate=1145000 \
! video/x-h264,profile=baseline ! h264parse \
! avimux ! filesink location=test.avi
Open questions:
a) The colors are all wrong, i.e. red becomes orange, green becomes violet and blue becomes yellow. I have no clue why? Dumping the pipeline to dot, I see the following caps description for the omxh264enc output using the old pipeline:
bit-depth-luma: 8
bit-depth-chroma: 8
colorimetry: bt601
chroma-site: jpeg
And this when I use the new pipeline:
interlace-mode: progressive
colorimetry: sRGB
Is this an indication of the problem maybe?
b) ARGB input doesn't work, only ABGR, and I have no idea why? When I try to feed ARGB I get:
0:00:02.013120260 2588 0x1fc3e60 DEBUG GST_CAPS gstutils.c:
3111:gst_pad_peer_query_caps:<omxh264enc-omxh264enc0:src> peer query returned
video/x-h264, width=(int)[ 16, 4096 ], height=(int)[ 16, 4096 ],
framerate=(fraction)[ 0/1, 2147483647/1 ], profile=(string)baseline, stream-
format=(string)byte-stream, alignment=(string)au; video/x-h264, width=(int)[
16, 4096 ], height=(int)[ 16, 4096 ], framerate=(fraction)[ 0/1, 2147483647/1
], profile=(string)baseline
0:00:02.013263854 2588 0x1fc3e60 LOG default gstvideoutilsprivate.c:
128:__gst_video_element_proxy_getcaps:<omxh264enc-omxh264enc0> template caps
video/x-raw, format=(string)I420; video/x-raw, format=(string)NV12; video/x-
raw, format=(string)ABGR
0:00:02.013389583 2588 0x1fc3e60 LOG default gstvideoutilsprivate.c:
129:__gst_video_element_proxy_getcaps:<omxh264enc-omxh264enc0> allowed caps
video/x-h264, width=(int)[ 16, 4096 ], height=(int)[ 16, 4096 ],
framerate=(fraction)[ 0/1, 2147483647/1 ], profile=(string)baseline, stream-
format=(string)byte-stream, alignment=(string)au; video/x-h264, width=(int)[
16, 4096 ], height=(int)[ 16, 4096 ], framerate=(fraction)[ 0/1, 2147483647/1
], profile=(string)baseline
0:00:02.013797604 2588 0x1fc3e60 LOG default gstvideoutilsprivate.c:
138:__gst_video_element_proxy_getcaps:<omxh264enc-omxh264enc0> intersecting
with video/x-raw(memory:GLMemory), format=(string)ARGB, width=(int)[ 1,
2147483647 ], height=(int)[ 1, 2147483647 ], framerate=(fraction)[ 0/1,
2147483647/1 ], texture-target=(string){ 2D, rectangle, external-oes }; video/
x-raw(memory:DMABuf), format=(string)ARGB, width=(int)[ 1, 2147483647 ],
height=(int)[ 1, 2147483647 ], framerate=(fraction)[ 0/1, 2147483647/1 ];
video/x-raw, format=(string)ARGB, width=(int)[ 1, 2147483647 ], height=(int)[
1, 2147483647 ], framerate=(fraction)[ 0/1, 2147483647/1 ]
0:00:02.013934218 2588 0x1fc3e60 LOG default gstvideoutilsprivate.c:
147:__gst_video_element_proxy_getcaps:<omxh264enc-omxh264enc0> proxy caps
EMPTY
0:00:02.014005156 2588 0x1fc3e60 LOG omxvideoenc
gstomxvideoenc.c:3258:gst_omx_video_enc_getcaps:<omxh264enc-omxh264enc0>
Supported caps EMPTY
With ABGR it works and I see:
0:00:02.026753801 2733 0xf85c60 DEBUG GST_CAPS gstutils.c:
3111:gst_pad_peer_query_caps:<omxh264enc-omxh264enc0:src> peer query returned
video/x-h264, width=(int)[ 16, 4096 ], height=(int)[ 16, 4096 ],
framerate=(fraction)[ 0/1, 2147483647/1 ], profile=(string)baseline, stream-
format=(string)byte-stream, alignment=(string)au; video/x-h264, width=(int)[
16, 4096 ], height=(int)[ 16, 4096 ], framerate=(fraction)[ 0/1, 2147483647/1
], profile=(string)baseline
0:00:02.026899114 2733 0xf85c60 LOG default gstvideoutilsprivate.c:
128:__gst_video_element_proxy_getcaps:<omxh264enc-omxh264enc0> template caps
video/x-raw, format=(string)I420; video/x-raw, format=(string)NV12; video/x-
raw, format=(string)ABGR
0:00:02.027026353 2733 0xf85c60 LOG default gstvideoutilsprivate.c:
129:__gst_video_element_proxy_getcaps:<omxh264enc-omxh264enc0> allowed caps
video/x-h264, width=(int)[ 16, 4096 ], height=(int)[ 16, 4096 ],
framerate=(fraction)[ 0/1, 2147483647/1 ], profile=(string)baseline, stream-
format=(string)byte-stream, alignment=(string)au; video/x-h264, width=(int)[
16, 4096 ], height=(int)[ 16, 4096 ], framerate=(fraction)[ 0/1, 2147483647/1
], profile=(string)baseline
0:00:02.027436614 2733 0xf85c60 LOG default gstvideoutilsprivate.c:
138:__gst_video_element_proxy_getcaps:<omxh264enc-omxh264enc0> intersecting
with video/x-raw(memory:GLMemory), format=(string)ABGR, width=(int)[ 1,
2147483647 ], height=(int)[ 1, 2147483647 ], framerate=(fraction)[ 0/1,
2147483647/1 ], texture-target=(string){ 2D, rectangle, external-oes }; video/
x-raw(memory:DMABuf), format=(string)ABGR, width=(int)[ 1, 2147483647 ],
height=(int)[ 1, 2147483647 ], framerate=(fraction)[ 0/1, 2147483647/1 ];
video/x-raw, format=(string)ABGR, width=(int)[ 1, 2147483647 ], height=(int)[
1, 2147483647 ], framerate=(fraction)[ 0/1, 2147483647/1 ]
0:00:02.027611770 2733 0xf85c60 LOG default gstvideoutilsprivate.c:
147:__gst_video_element_proxy_getcaps:<omxh264enc-omxh264enc0> proxy caps
video/x-raw, width=(int)[ 16, 4096 ], height=(int)[ 16, 4096 ],
framerate=(fraction)[ 0/1, 2147483647/1 ], format=(string)ABGR
0:00:02.027717499 2733 0xf85c60 LOG omxvideoenc
gstomxvideoenc.c:3258:gst_omx_video_enc_getcaps:<omxh264enc-omxh264enc0>
Supported caps video/x-raw, width=(int)[ 16, 4096 ], height=(int)[ 16, 4096 ],
framerate=(fraction)[ 0/1, 2147483647/1 ], format=(string)ABGR