Frame concatenation methods are different between gradio demo and fast_captioner_lmdeploy.py?

[Liam-lyr]

Hi, thanks for the great work!

I'm interested in the Fast Captioning pipeline and I found there if a difference between the frame concatenation methods used in captioner/fast_captioner_lmdeploy.py and captioner/app.py.

In captioner/fast_captioner_lmdeploy.py, you use 30 frames per image to form a 5x6 grid and feed it to the pipeline (https://github.com/ShareGPT4Omni/ShareGPT4Video/blob/88426fd4a8386f3009368d424d5972881cdde311/captioner/fast_captioner_lmdeploy.py#L93C1-L97C84):

    img_grid_w, img_grid_h = 5, 6  # Grid dimensions, defaultly 30 images per video
    img_h, img_w = 600, 800  # Desired height and width

    # Create the dataset
    dataset = ImageGridDataset(args.img_path, img_grid_w, img_grid_h, img_h, img_w)

However in captioner/app.py, you concat the frames in a list (as described in the original paper) (the following snippet and https://github.com/ShareGPT4Omni/ShareGPT4Video/blob/88426fd4a8386f3009368d424d5972881cdde311/captioner/app.py#L155C1-L201C21):

    imgs = load_quota_video(video_path)
    img = img_process(imgs)

Can you please explain why? Thanks a lot.