Access side data without decoding the video

[mfoglio]

Right now I am accessing the motion vectors in the following way:

container = av.open(
    rtsp_url, 'r',
    options={
        'rtsp_transport': 'tcp',
        'stimeout': '5000000', 
        'max_delay': '5000000', 
    }
)
stream = container.streams.video[0]
codec_context = stream.codec_context
codec_context.export_mvs = True


for packet in container.demux(video=0):
    for video_frame in packet.decode():
        motion_vectors_raw = video_frame.side_data.get('MOTION_VECTORS')

It seems to me that this does decode the video_frame. Is there a way to obtain the motion vectors without having to decode the entire frame? My goal is to reduce the CPU utilization.

[mfoglio]

I see https://libav.org/documentation/doxygen/master/group__lavc__packet.html that libav AVPacket seems to have a function av_packet_get_side_data. Is this somehow available in Python? I have a good Python knowledge but I am quite rusty in C. Would think it would be difficult for me to add it?

[leidix]

Right now I am accessing the motion vectors in the following way:

container = av.open(
    rtsp_url, 'r',
    options={
        'rtsp_transport': 'tcp',
        'stimeout': '5000000', 
        'max_delay': '5000000', 
    }
)
stream = container.streams.video[0]
codec_context = stream.codec_context
codec_context.export_mvs = True


for packet in container.demux(video=0):
    for video_frame in packet.decode():
        motion_vectors_raw = video_frame.side_data.get('MOTION_VECTORS')

It seems to me that this does decode the video_frame. Is there a way to obtain the motion vectors without having to decode the entire frame? My goal is to reduce the CPU utilization.

Does your approach with motion_vectors_raw = video_frame.side_data.get('MOTION_VECTORS') even work? I tested it with an h.264/rtsp stream but the side data is always empty, even though i verified with https://github.com/LukasBommes/mv-extractor it contains motion vectors. I would love to access them with pyav, your approach using only a few lines of code seems very tempting to me. Thanks for any feddback!