qtdemux: video stream that is much shorter than the audio stream is not recognized
I noticed a weird GStreamer behavior. I have generated a QT container containing a very short video stream and a bit longer audio stream:
gst-launch-1.0 -e \
qtmux name=mux ! filesink location="short_video_long_audio2.mp4" \
videotestsrc num-buffers=10 pattern=smpte ! video/x-raw,width=1920,height=1080 ! x264enc ! mux. \
audiotestsrc num-buffers=200 wave=ticks ! stereo ! audioconvert ! avenc_aac ! mux.
mediainfo reports both streams correctly:
mediainfo short_video_long_audio2.mp4
General
Complete name : short_video_long_audio2.mp4
Format : MPEG-4
Format profile : QuickTime
Codec ID : qt 2005.03 (qt )
File size : 129 KiB
Duration : 4 s 667 ms
Overall bit rate : 227 kb/s
Frame rate : 30.000 FPS
Encoded date : 2024-05-31 08:14:12 UTC
Tagged date : 2024-05-31 08:14:13 UTC
Video
ID : 1
Format : AVC
Format/Info : Advanced Video Codec
Format profile : High 4:4:4 Predictive@L4
Format settings : CABAC / 4 Ref Frames
Format settings, CABAC : Yes
Format settings, Reference frames : 4 frames
Codec ID : avc1
Codec ID/Info : Advanced Video Coding
Duration : 333 ms
Bit rate : 2 993 kb/s
Nominal bit rate : 2 048 kb/s
Maximum bit rate : 2 097 kb/s
Width : 1 920 pixels
Height : 1 080 pixels
Display aspect ratio : 16:9
Frame rate mode : Constant
Frame rate : 30.000 FPS
Color space : YUV
Chroma subsampling : 4:4:4
Bit depth : 10 bits
Scan type : Progressive
Bits/(Pixel*Frame) : 0.048
Stream size : 122 KiB (94%)
Writing library : x264 core 164 r3108 31e19f9
Encoding settings : cabac=1 / ref=3 / deblock=1:0:0 / analyse=0x3:0x113 / me=hex / subme=7 / psy=1 / psy_rd=1.00:0.00 / mixed_ref=1 / me_range=16 / chroma_me=1 / trellis=1 / 8x8dct=1 / cqm=0 / deadzone=21,11 / fast_pskip=1 / chroma_qp_offset=4 / threads=18 / lookahead_threads=3 / sliced_threads=0 / nr=0 / decimate=1 / interlaced=0 / bluray_compat=0 / constrained_intra=0 / bframes=3 / b_pyramid=2 / b_adapt=1 / b_bias=0 / direct=1 / weightb=1 / open_gop=0 / weightp=2 / keyint=300 / keyint_min=30 / scenecut=40 / intra_refresh=0 / rc_lookahead=40 / rc=cbr / mbtree=1 / bitrate=2048 / ratetol=1.0 / qcomp=0.60 / qpmin=0 / qpmax=81 / qpstep=4 / vbv_maxrate=2048 / vbv_bufsize=1228 / nal_hrd=none / filler=0 / ip_ratio=1.40 / aq=1:1.00
Encoded date : 2024-05-31 08:14:12 UTC
Tagged date : 2024-05-31 08:14:13 UTC
Color range : Limited
Color primaries : BT.709
Transfer characteristics : BT.709
Matrix coefficients : BT.709
Codec configuration box : avcC
Audio
ID : 2
Format : AAC LC
Format/Info : Advanced Audio Codec Low Complexity
Codec ID : mp4a-40-2
Duration : 4 s 667 ms
Bit rate mode : Constant
Bit rate : 8 882 b/s
Channel(s) : 2 channels
Channel layout : L R
Sampling rate : 44.1 kHz
Frame rate : 43.066 FPS (1024 SPF)
Compression mode : Lossy
Stream size : 5.06 KiB (4%)
Encoded date : 2024-05-31 08:14:12 UTC
Tagged date : 2024-05-31 08:14:13 UTC
The same for ffprobe:
ffprobe ./short_video_long_audio2.mp4
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from './short_video_long_audio2.mp4':
Metadata:
major_brand : qt
minor_version : 537199360
compatible_brands: qt
creation_time : 2024-05-31T08:14:12.000000Z
encoder : x264
Duration: 00:00:04.67, start: 0.000000, bitrate: 227 kb/s
Stream #0:0[0x1](und): Video: h264 (High 4:4:4 Predictive) (avc1 / 0x31637661), yuv444p10le(tv, bt709, progressive), 1920x1080 [SAR 1:1 DAR 16:9], 2992 kb/s, 30 fps, 30 tbr, 3k tbn (default)
Metadata:
creation_time : 2024-05-31T08:14:12.000000Z
handler_name : VideoHandler
vendor_id : [0][0][0][0]
Stream #0:1[0x2](und): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 8 kb/s (default)
Metadata:
creation_time : 2024-05-31T08:14:12.000000Z
handler_name : SoundHandler
vendor_id : [0][0][0][0]
However gst-discoverer sees only audio stream:
gst-discoverer-1.0 short_video_long_audio2.mp4
Analyzing file:///Users/me/short_video_long_audio2.mp4
Done discovering file:///Users/me/short_video_long_audio2.mp4
Properties:
Duration: 0:00:04.667333333
Seekable: yes
Live: no
container #0: Quicktime
audio #1: MPEG-4 AAC
Stream ID: 7b480f53178c6b343fb74ae39a151af0a946e0160054cb500dafe667f0da1c24/002
Language: <unknown>
Channels: 2 (front-left, front-right)
Sample rate: 44100
Depth: 32
Bitrate: 8882
Max bitrate: 0
Running discoverer with the enabled debug messages I see:
0:00:00.011915291 99229 0x6000033b9bd0 WARN qtdemux qtdemux.c:11829:qtdemux_parse_trak:<qtdemux0> Track shorter than 20% (1000/3000 vs. 14002/3000) of the stream found, assuming preview image or something; skipping track
I would expect the behavior consistent with mediainfo and ffprobe, i.e. have the video track discovered as well.