[Support]: rtps feed distorted since 14.0 RC1/2 for portrait camera live view

[CornBread81032133]

Describe the problem you are having

The detection sub-stream for cameras in portrait orientation are showing a distortion to the right hand side of the image.
This does not show up when viewing the RTSP feed directly, or when using pre-release versions of 14.0 before RC1

How it looks in Frigate 14.0 RC1/2
rtsp://@192.168.0.13/axis-media/media.amp?videocodec=h264&fps=5

How the RTPS feed looks in VLC/older Frigate versions
rtsp://@192.168.0.13/axis-media/media.amp?videocodec=h264&fps=5

Recordings are using a seperate sub-stream and do not exhibit this distortion.

Steps to reproduce

Add a camera in portrait orientation.

Version

0.14.0-da913d8

Frigate config file

mqtt:
  enabled: false

auth:
  enabled: false
  failed_login_rate_limit: 5/minute
  trusted_proxies:
    - <REDACTED>

proxy:
  auth_secret: <REDACTED>
  header_map:
    user: x-forwarded-user
  logout_url: https://<REDACTED>?rd=https://<REDACTED>/

database:
  path: /config/db/frigate.db

detectors:
  ov:
    type: openvino
    device: GPU
    model:
      path: /openvino-model/ssdlite_mobilenet_v2.xml


model:
  width: 300
  height: 300
  input_tensor: nhwc
  input_pixel_format: bgr
  labelmap_path: /openvino-model/coco_91cl_bkgr.txt

cameras:
  front_yard:
    ui:
      order: 1
    ffmpeg:
      inputs:
        - path: 
            rtsp://<REDACTED>@192.168.0.12/axis-media/media.amp?videocodec=h264&fps=5
          roles:
            - detect
        - path: 
            rtsp://<REDACTED>@192.168.0.12/axis-media/media.amp?videocodec=h264
          roles:
            - record
    detect:
      width: 1920
      height: 1080
      fps: 5
      stationary:
        interval: 50
        threshold: 50
    objects:
      track:
        - person
        - car
        - dog
        - cat
      filters:
        car:
          min_area: 10000
          threshold: 0.65
        person:
          min_area: 5000
      mask: 
        0,0.646,0.004,0.004,0.895,0,0.954,0.134,0.911,0.148,0.919,0.178,0.912,0.226,0.773,0.303,0.506,0.408,0.286,0.478,0.082,0.59
    record:
      enabled: true
      retain:
        days: 7
      events:
        retain:
          default: 10
    snapshots:
      enabled: true
      timestamp: true
      bounding_box: true
      retain:
        default: 10

    motion:
      mask:
        - 0.897,0.45,0.871,0.586,0.806,0.662,0.715,0.597,0.664,0.501,0.689,0.412,0.738,0.35,0.77,0.298,0.807,0.288,0.85,0.282,0.898,0.296,0.92,0.325
        - 0.206,0,0.225,0.125,0.271,0.158,0.297,0.24,0.316,0.301,0.344,0.294,0.343,0.224,0.346,0.173,0.362,0.138,0.383,0.19,0.392,0.227,0.421,0.268,0.464,0.25,0.473,0.176,0.459,0.125,0.458,0.063,0.466,0
        - 0.606,0,0.645,0.122,0.68,0.186,0.714,0.184,0.74,0.214,0.762,0.301,0.81,0.282,0.82,0.22,0.832,0.18,0.857,0.178,0.885,0.209,0.914,0.22,0.936,0.137,0.945,0.055,0.944,0.001
        - 0.135,0.915,0.135,1,0,1,0,0.912
    zones:
      Driveway:
        coordinates: 0.082,0.657,0.107,0.997,0.812,1,0.329,0.531
        inertia: 3
        loitering_time: 0
      Garden:
        coordinates: 
          0.329,0.529,0.509,0.408,0.77,0.293,0.841,0.312,0.668,0.502,0.815,0.683,0.739,0.8,0.628,0.81
        objects:
          - cat
          - dog
          - person
        inertia: 3
        loitering_time: 0
      Pathway:
        coordinates: 
          0.63,0.814,0.743,0.803,0.817,0.679,0.809,0.663,0.882,0.517,0.898,0.44,0.919,0.178,1,0.219,0.941,0.652,0.89,0.866,0.849,1,0.815,0.996
      Mailbox:
        coordinates: 0.306,0.505,0.242,0.387,0.331,0.342,0.381,0.47
        loitering_time: 2
        objects: person
        inertia: 3
  Front_Door:
    ui:
      order: 2
    ffmpeg:
      inputs:
        - path: 
            rtsp://<REDACTED>@192.168.0.13/axis-media/media.amp?videocodec=h264&fps=5
          roles:
            - detect
        - path: 
            rtsp://<REDACTED>@192.168.0.13/axis-media/media.amp?videocodec=h264
          roles:
            - record
    detect:
      width: 1080
      height: 1920
      fps: 5
      stationary:
        interval: 50
        threshold: 50
    objects:
      track:
        - person
        - dog
        - cat
        - package
      filters:
        person:
          min_area: 5000
      mask: 0.241,0.366,0.684,0.284,0.997,0.341,0.996,0.002,0.185,0.003
    record:
      enabled: true
      retain:
        days: 7
      events:
        retain:
          default: 10
    snapshots:
      enabled: true
      timestamp: true
      bounding_box: true
      retain:
        default: 10

    motion:
      mask:
        - 0.993,0.593,0.594,0.436,0.477,0.465,0.312,0.398,0.277,0.367,0.253,0.287,0.255,0.247,0.257,0.197,0.338,0.183,0.432,0.18,0.507,0.184,0.595,0.196,0.651,0.22,0.704,0.244,0.727,0.275,0.832,0.265,0.928,0.255,1,0.241
        - 0.617,0.171,0.7,0.155,0.696,0.111,0.676,0.076,0.714,0.051,0.721,0,0.322,0.013,0.345,0.045,0.408,0.052,0.444,0.057,0.481,0.071,0.513,0.075,0.534,0.109,0.548,0.134,0.586,0.165
        - 0.001,0.054,0.257,0.054,0.269,0.028,0.365,0.029,0.395,0.002,0,0
  Garage:
    ui:
      order: 4
    ffmpeg:
      inputs:
        - path: 
            rtsp://<REDACTED>@192.168.0.8/axis-media/media.amp?camera=2&videocodec=h264&fps=5
          roles:
            - detect
        - path: 
            rtsp://<REDACTED>@192.168.0.8/axis-media/media.amp?camera=2&videocodec=h264
          roles:
            - record
    detect:
      width: 1920
      height: 1080
      fps: 5
      stationary:
        interval: 50
        threshold: 50
    objects:
      track:
        - person
        - dog
        - cat
      filters:
        person:
          min_area: 5000
      mask: 0.566,0.169,0.908,0.192,0.88,0.717,0.57,0.649
    record:
      enabled: true
      retain:
        days: 7
      events:
        retain:
          default: 10
    snapshots:
      enabled: true
      timestamp: true
      bounding_box: true
      retain:
        default: 10

    motion:
      threshold: 30
      contour_area: 10
      improve_contrast: 'false'
  Back_door:
    ui:
      order: 5
    ffmpeg:
      inputs:
        - path: 
            rtsp://<REDACTED>@192.168.0.25/axis-media/media.amp?videocodec=h264&fps=5
          roles:
            - detect
        - path: 
            rtsp://<REDACTED>@192.168.0.25/axis-media/media.amp?videocodec=h264
          roles:
            - record
    detect:
      width: 1920
      height: 1080
      fps: 5
      stationary:
        interval: 50
        threshold: 50
    objects:
      track:
        - person
        - dog
        - cat
      filters:
        person:
          min_area: 5000
    record:
      enabled: true
      retain:
        days: 7
      events:
        retain:
          default: 10
    snapshots:
      enabled: true
      timestamp: true
      bounding_box: true
      retain:
        default: 10

    motion:
      mask:
        - 0.29,0.571,0.483,0.732,0.592,0.601,0.587,0.402,0.392,0.321,0.285,0.401
        - 0.847,0,0.852,0.04,0.881,0.042,0.882,0.069,1,0.069,1,0
      threshold: 40
      contour_area: 50
      improve_contrast: 'true'
  Doorbell:
    ui:
      order: 3
    ffmpeg:
      inputs:
        - path: rtsp://<REDACTED>@192.168.0.20/Preview_01_sub
          roles:
            - detect
        - path: rtsp://<REDACTED>@192.168.0.20/Preview_01_main
          roles:
            - record
    detect:
      width: 640
      height: 480
      fps: 10
      stationary:
        interval: 50
        threshold: 50
    objects:
      track:
        - person
        - dog
        - cat
      filters:
        person:
          min_area: 5000
      mask: 0.187,0.611,0.543,0.615,0.547,0.009,0.262,0,0.196,0.255
    record:
      enabled: true
      retain:
        days: 7
      events:
        retain:
          default: 10
    snapshots:
      enabled: true
      timestamp: true
      bounding_box: true
      retain:
        default: 10
    motion:
      threshold: 30
      contour_area: 30
      improve_contrast: 'false'
      mask:
        - 0.184,0.581,0.544,0.573,0.55,0.81,0.218,0.817
        - 0.392,0.409,0.448,0.414,0.492,0.337,0.551,0.378,0.542,0.162,0.626,0.016,0.551,0.001,0.282,0.008,0.251,0.142,0.31,0.296
        - 0.636,0.017,0.638,0.053,0.968,0.05,0.965,0.012
ffmpeg:
  hwaccel_args: preset-vaapi
  output_args:
    record: preset-record-generic-audio-aac
version: 0.14
camera_groups:
  Front:
    order: 1
    icon: LuLeaf
    cameras:
      - Doorbell
      - Front_Door
      - front_yard

Relevant log output

None

Operating system

Proxmox

Install method

Docker Compose

Network connection

Wired

Camera make and model

Axis

Any other information that may be helpful

No response

[hawkeye217]

A change was made after beta 4 to the jsmpeg player to support more streams and devices. The distortion you are seeing is because your detect aspect ratio does not perfectly match the camera substream’s aspect ratio. Either adjust the detect resolution or adjust the output of the camera.

[CornBread81032133]

Here is the output of ffprobe for this RTSP feed.

root@54d152ec55eb:/opt/frigate# ffprobe 'rtsp://<user>:<pass>@192.168.0.13/axis-media/media.amp?videocodec=h264&fps=5'
ffprobe version n5.1-2-g915ef932a3-20220731 Copyright (c) 2007-2022 the FFmpeg developers
  built with gcc 12.1.0 (crosstool-NG 1.25.0.55_3defb7b)
  configuration: --prefix=/ffbuild/prefix --pkg-config-flags=--static --pkg-config=pkg-config --cross-prefix=x86_64-ffbuild-linux-gnu- --arch=x86_64 --target-os=linux --enable-gpl --enable-version3 --disable-debug --enable-iconv --enable-libxml2 --enable-zlib --enable-libfreetype --enable-libfribidi --enable-gmp --enable-lzma --enable-fontconfig --enable-libvorbis --enable-opencl --enable-libpulse --enable-libvmaf --enable-libxcb --enable-xlib --enable-amf --enable-libaom --enable-libaribb24 --enable-avisynth --enable-libdav1d --enable-libdavs2 --disable-libfdk-aac --enable-ffnvcodec --enable-cuda-llvm --enable-frei0r --enable-libgme --enable-libass --enable-libbluray --enable-libjxl --enable-libmp3lame --enable-libopus --enable-mbedtls --enable-librist --enable-libtheora --enable-libvpx --enable-libwebp --enable-lv2 --enable-libmfx --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenh264 --enable-libopenjpeg --enable-libopenmpt --enable-librav1e --enable-librubberband --disable-schannel --enable-sdl2 --enable-libsoxr --enable-libsrt --enable-libsvtav1 --enable-libtwolame --enable-libuavs3d --enable-libdrm --enable-vaapi --enable-libvidstab --enable-vulkan --enable-libshaderc --enable-libplacebo --enable-libx264 --enable-libx265 --enable-libxavs2 --enable-libxvid --enable-libzimg --enable-libzvbi --extra-cflags=-DLIBTWOLAME_STATIC --extra-cxxflags= --extra-ldflags=-pthread --extra-ldexeflags=-pie --extra-libs='-ldl -lgomp' --extra-version=20220731
  libavutil      57. 28.100 / 57. 28.100
  libavcodec     59. 37.100 / 59. 37.100
  libavformat    59. 27.100 / 59. 27.100
  libavdevice    59.  7.100 / 59.  7.100
  libavfilter     8. 44.100 /  8. 44.100
  libswscale      6.  7.100 /  6.  7.100
  libswresample   4.  7.100 /  4.  7.100
  libpostproc    56.  6.100 / 56.  6.100
Input #0, rtsp, from 'rtsp://<user>:<pass>@192.168.0.13/axis-media/media.amp?videocodec=h264&fps=5':
  Metadata:
    title           : Session streamed with GStreamer
    comment         : rtsp-server
  Duration: N/A, start: 0.000000, bitrate: N/A
  Stream #0:0: Video: h264 (High), yuvj420p(pc, bt709, progressive), 1080x1920, 5 fps, 5 tbr, 90k tbn
  Stream #0:1: Audio: pcm_mulaw, 8000 Hz, mono, s16, 64 kb/s

[hawkeye217]

Thanks. There’s still a mismatch of resolutions somewhere, though I can’t tell where it is. It could just be a matter of a few pixels where something in your pipeline is lying about exact resolution.

In any case, using go2rtc for your live views will solve your issue. That’s my best recommendation at this point.

[hawkeye217]

Though I would still recommend setting up go2rtc, there's something else we can try. Use your browser's dev tools to inspect the player element when you see the artifact. It should look similar to this:

Grab a screenshot of that. Let's see if we can figure this out.

[CornBread81032133]

I've grabbed a screenshot on my laptop, which looks to be showing a scaled resolution based on the browser viewport?

<canvas class="rounded-lg md:rounded-2xl" width="405" height="720" style="width: 509px; height: 904px;"></canvas>

[hawkeye217]

Yep, everything there looks correct. Unfortunately that doesn't bring us any closer to finding out what the true resolution of your stream is.

Your best bet is still go2rtc.

[hawkeye217]

Live views would also be greatly improved by using go2rtc: https://deploy-preview-11419--frigate-docs.netlify.app/configuration/live