Skip to content

msdk: fix double free for open hw failed

Xu Guangxin requested to merge XuGuangxin/gst-plugins-bad:open_failed into master

if gst_msdk_context_use_vaapi returns fail, we will call msdk_close_session twice.

one in gst_msdk_context_open, one in gst_msdk_context_finalize.

It will introduce double free problem. crash stack like this

0:00:03.511523213  9544      0x5c9a660 ERROR            msdkcontext gstmsdkcontext.c:152:gst_msdk_context_use_vaapi: Couldn't initialize VA DRM display
==9544== Invalid read of size 8
==9544==    at 0x49EEE05: MFX::LoaderCtx::Close() (mfxloader.cpp:262)
==9544==    by 0x49F08DE: MFXClose (mfxloader.cpp:419)
==9544==    by 0x5E5C34F: msdk_close_session (msdk.c:177)
==9544==    by 0x5E3E748: gst_msdk_context_finalize (gstmsdkcontext.c:240)
==9544==    by 0x4B3DC0D: g_object_unref (in /usr/lib/x86_64-linux-gnu/libgobject-2.0.so.0.6200.4)
==9544==    by 0x4899FD1: gst_object_unref (gstobject.c:267)
==9544==    by 0x5E3E84E: gst_msdk_context_new (gstmsdkcontext.c:269)
==9544==    by 0x5E3FC14: gst_msdk_context_ensure_context (gstmsdkcontextutil.c:224)
==9544==    by 0x5E426AC: gst_msdkdec_start (gstmsdkdec.c:791)
==9544==    by 0x5F1FAAD: gst_video_decoder_change_state (gstvideodecoder.c:2542)
==9544==    by 0x48D9D82: gst_element_change_state (gstelement.c:3033)
==9544==    by 0x48D9ADE: gst_element_set_state_func (gstelement.c:2987)
==9544==  Address 0x5c9dde0 is 48 bytes inside a block of size 592 free'd
==9544==    at 0x483BFBF: operator delete(void*) (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
==9544==    by 0x49F0947: operator() (unique_ptr.h:81)
==9544==    by 0x49F0947: operator() (unique_ptr.h:75)
==9544==    by 0x49F0947: ~unique_ptr (unique_ptr.h:284)
==9544==    by 0x49F0947: MFXClose (mfxloader.cpp:418)
==9544==    by 0x5E5C34F: msdk_close_session (msdk.c:177)
==9544==    by 0x5E3E616: gst_msdk_context_open (gstmsdkcontext.c:202)
==9544==    by 0x5E3E837: gst_msdk_context_new (gstmsdkcontext.c:267)
==9544==    by 0x5E3FC14: gst_msdk_context_ensure_context (gstmsdkcontextutil.c:224)
==9544==    by 0x5E426AC: gst_msdkdec_start (gstmsdkdec.c:791)
==9544==    by 0x5F1FAAD: gst_video_decoder_change_state (gstvideodecoder.c:2542)
==9544==    by 0x48D9D82: gst_element_change_state (gstelement.c:3033)
==9544==    by 0x48D9ADE: gst_element_set_state_func (gstelement.c:2987)
==9544==    by 0x48D966E: gst_element_set_state (gstelement.c:2888)
==9544==    by 0x48A4FC2: gst_bin_element_set_state (gstbin.c:2615)
==9544==  Block was alloc'd at
==9544==    at 0x483AE63: operator new(unsigned long) (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
==9544==    by 0x49F09D3: MFXInitEx (mfxloader.cpp:398)
==9544==    by 0x49F0BA5: MFXInit (mfxloader.cpp:388)
==9544==    by 0x5E5C401: msdk_open_session (msdk.c:197)
==9544==    by 0x5E3E5C8: gst_msdk_context_open (gstmsdkcontext.c:186)
==9544==    by 0x5E3E837: gst_msdk_context_new (gstmsdkcontext.c:267)
==9544==    by 0x5E3FC14: gst_msdk_context_ensure_context (gstmsdkcontextutil.c:224)
==9544==    by 0x5E426AC: gst_msdkdec_start (gstmsdkdec.c:791)
==9544==    by 0x5F1FAAD: gst_video_decoder_change_state (gstvideodecoder.c:2542)
==9544==    by 0x48D9D82: gst_element_change_state (gstelement.c:3033)
==9544==    by 0x48D9ADE: gst_element_set_state_func (gstelement.c:2987)
==9544==    by 0x48D966E: gst_element_set_state (gstelement.c:2888)
==9544==

Merge request reports