When gstplayer disables the video stream vaapi fails

0:00:00.828007422  9427 0x560fe2e54ed0 ERROR                  queue gstqueue.c:1028:gst_queue_handle_sink_event:<vaapi-queue> Failed to push event
0:00:00.829051716  9427 0x560fe2e54ed0 ERROR                  queue gstqueue.c:1028:gst_queue_handle_sink_event:<vaapi-queue> Failed to push event
Seek done to position 0
0:00:00.836105180  9427 0x560fe2e3ea30 ERROR            vaapidecode gstvaapidecode.c:736:gst_vaapidecode_handle_frame: push loop error while decoding -1
0:00:00.841012017  9427 0x560fe2e3ea30 ERROR            vaapidecode gstvaapidecode.c:736:gst_vaapidecode_handle_frame: push loop error while decoding -1
0:00:00.846346564  9427 0x560fe2e3ea30 ERROR            vaapidecode gstvaapidecode.c:736:gst_vaapidecode_handle_frame: push loop error while decoding -1

Strangely, when using playbin directly, it doesn't fail which makes me thing that it is related with the surface exhaustion under certain sinks (with gstplayer I use glgsinkbin)