[Config Support]: seemingly refusing to detect foxes! #14820

ElectronicBattle · 2024-11-05T18:59:46Z

ElectronicBattle
Nov 5, 2024

Describe the problem you are having

Hello

Following the Frigate+ Base Model 2024.3 update, I was expecting to detect lots of foxes but I am seeing none. Prior to the model update, I was detecting foxes every evening (as "cat" or "dog") but having seen several foxes on the cameras yesterday evening and the night before, not one detection has occurred of them (as "cat" or "dog" or "fox").

The cameras are detecting actual cats and dogs and people so it is all otherwise working. Have I made a mistake in my config?

Version

0.13

Frigate config file

#BASELINE8_170824_013 this works OK on Frigate and on frigateviewer: 9ms inference and <50% TN CPU load fastinference8cameras013_config_310724_NO_RESTREAMING_FIXED taken from #config_and_audio_110724# # this works with the #FrigateViewer tablet app - events playback OK
# improved detection (height & width added) # also added ffmpeg out args record audio and logger entries at EOF

objects:
  filters:
    dog:
      min_score: .8
      threshold: .9
    cat:
      min_score: .75
      threshold: .8
    fox:  
      min_score: .65
      threshold: .7
    bird:
      min_score: .65
      threshold: .8
    face:
      min_score: .7
    package:
      min_score: .65
      threshold: .9
    license_plate:
      min_score: .6
    amazon:
      min_score: .75
    ups:
      min_score: .75
    fedex:
      min_score: .75
    person:
      min_score: .65
      threshold: .85
    car:
      min_score: .65
      threshold: .85


ui:
  use_experimental: false
  live_mode: mse

mqtt:
  enabled: false

model:
  path: plus://98f2ff2d47bfacdd1823ef3641b716c4 #mobiledet edgeTPU 2024.3
#  path: plus://9b85aa0fc8a3aaa6ef8dfb8ea80a2dc9 #mobiledet edgeTPU 2024.2
#  path: plus://2cf1f99bc96475f2423babf38b7d7567 #mobiledet cpu 2024.2 


go2rtc:
  streams:
    
    #reference format from NickM-27 @ https://github.com/blakeblackshear/frigate/discussions/12376 
    #              "ffmpeg:rtsp://...                                               #video=h264#audio=copy#hardware" note the quotes and hashes
    greenhouse:    
      - "ffmpeg:rtsp://192.168.1.123:554/media=0/channel=0&level=0/track0#video=h264#audio=copy"
      - "ffmpeg:greenhouse#audio=opus"
    upper_patio:   
      - "ffmpeg:rtsp://192.168.1.123:554/media=0/channel=1&level=0/track0#video=h264#audio=copy"
      - "ffmpeg:upper_patio#audio=opus"
    ponds:         
      - "ffmpeg:rtsp://192.168.1.123:554/media=0/channel=2&level=0/track0#video=h264#audio=copy"
      - "ffmpeg:ponds#audio=opus"
    blue_gate:     
      - "ffmpeg:rtsp://192.168.1.123:554/media=0/channel=3&level=0/track0#video=h264#audio=copy"
      - "ffmpeg:blue_gate#audio=opus"
    lower_steps:   
      - "ffmpeg:rtsp://192.168.1.123:554/media=0/channel=4&level=0/track0#video=h264#audio=copy"
      - "ffmpeg:lower_steps#audio=opus"
    lower_patio:   
      - "ffmpeg:rtsp://192.168.1.123:554/media=0/channel=5&level=0/track0#video=h264#audio=copy"
      - "ffmpeg:lower_patio#audio=opus"
    shed:          
      - "ffmpeg:rtsp://192.168.1.123:554/media=0/channel=6&level=0/track0#video=h264#audio=copy"
      - "ffmpeg:shed#audio=opus"
    drive:         
      - "ffmpeg:rtsp://192.168.1.123:554/media=0/channel=7&level=0/track0#video=h264#audio=copy"
      - "ffmpeg:drive#audio=opus"
#    doorbell:
#      - "ffmpeg:rtsp://admin:[email protected]:554/Streaming/Channels/101#raw -vf lenscorrection=k1=-0.450:k2=0.1"
#      - "ffmpeg:doorbell#audio=opus"
    

####rtsp is 30060 and webRTC is 30061 (TN Frigate app interrnals)
  webrtc:
    candidates:
    - 192.168.1.194:30061
    - stun:30061


ffmpeg:
  #####you can only have one line here#####
  #hwaccel_args: preset-nvidia-h264
  #hwaccel_args: preset-nvidia-hevc #gives black screens on cameras
  #hwaccel_args: preset-nvidia-cuvid #gives black screens on cameras
  output_args:
    record: preset-record-generic-audio-aac
  
detectors:
  coral:
    type: edgetpu
    device: usb
    detect:
      enabled: true

birdseye:
  enabled: true
  mode: continuous #continuous | objects | motion

# record: # <----- Enable recording # was working
#  enabled: true # was working

# added from https://fixtse.com/blog/coral-ia-frigate start#
record:
  enabled: true
  retain:
    days: 5
    mode: motion
  events:
    retain:
      default: 7
      mode: active_objects
# added from https://fixtse.com/blog/coral-ia-frigate #

 
snapshots:
  # Optional: Enable writing jpg snapshot to /media/frigate/clips (default: shown below)
  enabled: true
  # Optional: save a clean PNG copy of the snapshot image (default: shown below)
  clean_copy: true #false  #true
  # Optional: print a timestamp on the snapshots (default: shown below)
  timestamp: false
  # Optional: draw bounding box on the snapshots (default: shown below)
  bounding_box: true
  # Optional: crop the snapshot (default: shown below)
  crop: false
  # Optional: height to resize the snapshot to (default: original size)
  height: 768
  #height: 800
  # Optional: Restrict snapshots to objects that entered any of the listed zones (default: no required zones)
  required_zones: []
  # Optional: Camera override for retention settings (default: global values)
  retain:
    # Required: Default retention days (default: shown below)
    default: 7
    # Optional: Per object retention days
    objects:
      person: 7
      cat: 7
      dog: 7
      bird: 1
      fox: 7
      badger: 7
      car: 7
      license_plate: 7

  # Optional: quality of the encoded jpeg, 0-100 (default: shown below)
  quality: 100

##########################################################
##########################################################
# go2rtc: # this worked from https://docs.frigate.video/configuration/live/#setting-stream-for-live-ui
#  streams:
#    upper_patio:
#    - rtsp://192.168.1.123:554/media=0/channel=1&level=0/track0 # <- stream which supports video & aac audio.
#    - "ffmpeg:upper_patio#audio=opus" # <- copy of the stream which transcodes audio to opus for webrtc
#    restream: true        
#    upper_patio_sub:
#    - rtsp://192.168.1.123:554/media=0/channel=1&level=1/track0 # <- stream which supports video & aac audio.
#    - "ffmpeg:upper_patio_sub#audio=opus"
##########################################################
##########################################################

motion:
  improve_contrast: True

detect:
  width: 640
  height: 360

#record:
#  width: 1920
#  height: 1080

########################################################################################################################################################################################################################################

cameras: #########GREENHOUSE##################GREENHOUSE##################GREENHOUSE##################GREENHOUSE##################GREENHOUSE#########GREENHOUSE#########
  greenhouse:
    enabled: true
    ffmpeg:
      inputs:
      - path: rtsp://192.168.1.123:554/media=0/channel=0&level=1/track0 #level1 = SD 640x360 (checked CORRECT); track0 media=0 are the only working options tested in ffplay
        roles:
        - detect
          

      - path: rtsp://192.168.1.123:554/media=0/channel=0&level=0/track0 #level0 = HD 1920x1080(checked CORRECT);  track0 media=0 are the only working options tested in ffplay
        roles:
        - record

    objects:
      track:
      - face 
      - person
      - cat
      - dog
      - bird
      - deer
      - fox
      - badger #placeholder

    birdseye:
      mode: continuous #continuous | objects | motion
    # masks are black: they block out areas where you aren't interested in detecting things
      order: 1




  upper_patio: #########UPPER_PATIO###############################################UPPER_PATIO###########################UPPER_PATIO###########################UPPER_PATIO###########################
    enabled: true
    ffmpeg:
      inputs:
      - path: rtsp://192.168.1.123:554/media=0/channel=1&level=1/track0 #level1 = SD 640x360 (checked CORRECT); track0 media=0 are the only working options tested in ffplay
        roles:
        - detect
          

      - path: rtsp://192.168.1.123:554/media=0/channel=1&level=0/track0 #level0 = HD 1920x1080(checked CORRECT);  track0 media=0 are the only working options tested in ffplay
        roles:
        - record

    objects:
      track:
      - face  
      - person
      - cat
      - dog
      - bird
      - deer
      - fox
      - badger #placeholder

    birdseye:
      mode: continuous #continuous | objects | motion
    # masks are black: they block out areas where you aren't interested in detecting things
      order: 2

    motion:
      mask: 0,219,72,169,140,139,146,79,218,73,291,72,362,133,430,143,450,121,488,120,523,98,640,95,640,0,0,0

    zones:
      upper patio:
        coordinates: 640,360,640,337,610,312,603,297,594,279,564,256,524,255,510,242,487,220,451,203,435,201,421,187,409,166,392,152,334,146,274,138,215,142,122,145,78,162,23,202,0,360




  ponds: #################PONDS#######################################PONDS#########################PONDS#########################PONDS#########################PONDS#########################
    enabled: true
    ffmpeg:
       inputs:
       - path: rtsp://192.168.1.123:554/media=0/channel=2&level=1/track0 #level1 = SD 640x360 (checked CORRECT); track0 media=0 are the only working options tested in ffplay
         roles:
         - detect
          

       - path: rtsp://192.168.1.123:554/media=0/channel=2&level=0/track0 #level0 = HD 1920x1080(checked CORRECT);  track0 media=0 are the only working options tested in ffplay
         roles:
         - record
    objects:
       track:
       - face  
       - person
       - cat
       - dog
       - bird
       - deer
       - fox 
       - badger #placeholder

    birdseye:
       mode: continuous #continuous | objects | motion
     # masks are black: they block out areas where you aren't interested in detecting things
       order: 3



  blue_gate: #########BLUE_GATE###################################################BLUE_GATE#############################BLUE_GATE#############################BLUE_GATE#############################BLUE_GATE#############################
    enabled: true
    ffmpeg:
      inputs:
      - path: rtsp://192.168.1.123:554/media=0/channel=3&level=1/track0   #level1 = SD; track0 media=0 are the only working options tested in ffplay
        #- path: http://truenas.local:8082/media=0/channel=3&level=1/track0 # didn't work: two streams avalble from .123 not from motioneye
        #- path: http://truenas.local:8082 # didn't work: two streams avalble from .123 not from motioneye
      #  input_args: preset-rtsp-restream

        roles:
        - detect
      - path: rtsp://192.168.1.123:554/media=0/channel=3&level=0/track0   #level0 = HD ;  track0 media=0 are the only working options tested in ffplay
        roles:
        - record

    objects:
      track:
      - face  
      - person
      - cat
      - dog
      - bird
      - deer
      - fox
      - badger # placeholder
      - car
      - license_plate

    birdseye:
      mode: continuous   #continuous | objects | motion
      # masks are black: they block out areas where you aren't interested in detecting things
      order: 4

    motion:
      mask:
      - 0,153,85,148,140,139,239,140,300,142,354,145,398,141,429,141,486,145,518,213,450,226,605,360,640,360,640,0,0,0
      - 479,360,423,337,411,310,390,288,369,268,352,248,340,228,297,223,267,247,231,253,212,204,188,200,152,218,107,227,88,222,73,234,63,250,54,266,45,282,27,292,0,287,0,360

    zones:
      gravel:
        coordinates: 411,273,471,360,523,360,585,360,474,227,473,151,388,141,283,146,228,142,0,151,0,301,296,233



  lower_steps: #########LOWER_STEPS#########LOWER_STEPS#######LOWER_STEPS#######LOWER_STEPS#######LOWER_STEPS#######LOWER_STEPS#######LOWER_STEPS#######LOWER_STEPS#######LOWER_STEPS#######
    enabled: true
    ffmpeg:
      inputs:
      - path: rtsp://192.168.1.123:554/media=0/channel=4&level=1/track0   #level1 = SD; track0 media=0 are the only working options tested in ffplay
        #- path: http://truenas.local:8082/media=0/channel=5&level=1/track0 # didn't work: two streams avalble from .123 not from motioneye
        #- path: http://truenas.local:8082 # didn't work: two streams avalble from .123 not from motioneye
       #input_args: preset-rtsp-restream
        roles:
        - detect
      - path: rtsp://192.168.1.123:554/media=0/channel=4&level=0/track0   #level0 = HD ;  track0 media=0 are the only working options tested in ffplay
        roles:
        - record

    objects:
      track:
      - face  
      - person
      - cat
      - dog
      - bird
      - deer
      - fox
      - badger   #placeholder

    birdseye:
      mode: continuous   #continuous | objects | motion
      order: 5


  lower_patio: #########LOWER_PATIO######LOWER_PATIO######LOWER_PATIO######LOWER_PATIO######LOWER_PATIO######LOWER_PATIO######LOWER_PATIO######LOWER_PATIO######LOWER_PATIO######LOWER_PATIO
    enabled: true
    ffmpeg:
      inputs:
      - path: rtsp://192.168.1.123:554/media=0/channel=5&level=1/track0   #level1 = SD; track0 media=0 are the only working options tested in ffplay
        #- path: http://truenas.local:8082/media=0/channel=5&level=1/track0 # didn't work: two streams avalble from .123 not from motioneye
        #- path: http://truenas.local:8082 # didn't work: two streams avalble from .123 not from motioneye
       #input_args: preset-rtsp-restream
        roles:
        - detect
      - path: rtsp://192.168.1.123:554/media=0/channel=5&level=0/track0   #level0 = HD ;  track0 media=0 are the only working options tested in ffplay
        roles:
        - record

    objects:
      track:
      - face  
      - person
      - cat
      - dog
      - bird
      - deer
      - fox   
      - badger   #placeholder

    birdseye:
      mode: continuous   #continuous | objects | motion
      order: 6

      # masks are black: they block out areas where you aren't interested in detecting things
    motion:
      mask:
      - 0,148,92,104,184,55,253,0,269,154,272,305,318,125,354,107,345,56,406,60,417,131,504,240,570,360,640,360,640,0,0,0
        #- 0,219,72,169,140,139,146,79,218,73,291,72,362,133,430,143,450,121,488,120,523,98,640,95,640,0,0,0

    zones:
      lower patio:
        coordinates: 284,360,303,183,333,157,344,139,357,118,352,99,359,74,382,73,384,110,403,128,499,246,580,360




  shed: #########SHED##########################SHED#########################SHED#########################SHED#########################SHED#########################SHED#########################
    enabled: true
    ffmpeg:
      inputs:
      - path: rtsp://192.168.1.123:554/media=0/channel=6&level=1/track0   #level1 = SD; track0 media=0 are the only working options tested in ffplay
        #- path: http://truenas.local:8082/media=0/channel=5&level=1/track0 # didn't work: two streams avalble from .123 not from motioneye
        #- path: http://truenas.local:8082 # didn't work: two streams avalble from .123 not from motioneye
       #input_args: preset-rtsp-restream
        roles:
        - detect
      - path: rtsp://192.168.1.123:554/media=0/channel=6&level=0/track0   #level0 = HD ;  track0 media=0 are the only working options tested in ffplay
        roles:
        - record

    objects:
      track:
      - face  
      - person
      - cat
      - dog
      - bird
      - deer
      - fox  
      - badger   #placeholder

    birdseye:
      mode: continuous   #continuous | objects | motion
      order: 7



  drive: #########DRIVE#########DRIVE#########DRIVE#########DRIVE#########DRIVE#########DRIVE#########DRIVE#########DRIVE#########DRIVE#########DRIVE#########DRIVE#########DRIVE#########DRIVE
    enabled: true

    objects:
      track:
      - face  
      - person
      - cat
      - dog
      - bird
      - deer
      - fox 
      - badger #placeholder
      - car
      - license_plate

    ffmpeg:
      inputs:
      - path: rtsp://192.168.1.123:554/media=0/channel=7&level=1/track0 #level1 = SD; changing track0 to track1 didn't work)
      #input_args: preset-rtsp-restream
        roles:
        - detect
      - path: rtsp://192.168.1.123:554/media=0/channel=7&level=0/track0 #level0 = HD ;(changing track0 to track1 didn't work)
        roles:
        - record

    birdseye:
      mode: continuous #continuous | objects | motion
      #restream: True
      order: 8

    motion:
      # masks are black: they block out areas where you aren't interested in detecting things
      mask:
      - 98,129,170,113,189,0,0,0,0,0,0,142
      - 273,0,170,0,178,67,258,117,256,191,414,181,640,307,640,0

    # zones are red: they select areas where monitoring of activity is wanted
    zones:
      driveway:
        coordinates: 78,360,248,360,381,360,640,360,640,276,449,177,303,183,298,129,200,64,144,113,0,127,0,360

#  doorbell: #####DOORBELL##########################DOORBELL##########################DOORBELL##########################DOORBELL#####################DOORBELL#####################DOORBELL#####################
#    enabled: true #remember that EZViz won't stream to more than thee (four?) destinations
#    ffmpeg:
#      inputs:
#        - path: rtsp://admin:[email protected]:554/Streaming/Channels/101 # H264 1536x1536
#          roles:
#            - detect
#      output_args:
#        detect: -vf lenscorrection=k1=-0.450:k2=0.1 -f rawvideo -pix_fmt yuv420p
#        - path: rtsp://127.0.0.1:8554/doorbell
#          roles:
#            - record
#      output_args:
#          detect: -vf lenscorrection=k1=-0.450:k2=0.1
#         
#    objects:
#      track:
#        - person
#        - cat
#        - dog
#        - bird
#        - deer
#        - fox #placeholder
#        - badger #placeholder
#    birdseye:
#      mode: continuous     #continuous | objects | motion
#        # masks are black: they block out areas where you aren't interested in detecting things
#      order: 9

# for the doorbell camera. this is what ffplay said (once I'd set the lens correction):

#$  ffplay -i rtsp://admin:[email protected]:554/Streaming/Channels/101 -vf lenscorrection=k1=-0.450:k2=0.095
#  Stream #0:0: Video: h264 (Main), yuv420p(progressive), 1536x1536, 15 fps, 15 tbr, 90k tbn
#  Stream #0:1: Audio: aac (LC), 16000 Hz, mono, fltp



logger:
    logs:
      #frigate.record.maintainer: debug
      frigate.object_detection: debug
      #watchdog.blue_gate: debug







#-----------------------END-----------------------##-----------------------END-----------------------##-----------------------END

Relevant Frigate log output

na

Relevant go2rtc log output

na

Frigate stats

na

Operating system

Other

Install method

Docker CLI

docker-compose file or Docker CLI command

TrueNAS (Debian base) appliance

Object Detector

Coral

Screenshots of the Frigate UI's System metrics pages

Any other information that may be helpful

No response

Answered by ElectronicBattle

Nov 7, 2024

I have now corrected the labels from "cat" or "dog" to "fox" for all of my previous fox images (June of this year until now) and my label stats now look like this, so I have submitted and received a new model:

I have also adjusted the

fox:  
      min_score: .5
      threshold: .6

Hopefully I will now see two things happen:

a return to foxes being detected (I don't know why Frigate seemed to have gone "blind" these last few days since installing the first new 2024.3 model)
correct labelling of them as foxes now that I have corrected my mistakes.

edit: in the three minutes it took me to update my config and reboot and write some of this reply:

Working!

View full answer

NickM-27 · 2024-11-05T19:02:43Z

NickM-27
Nov 5, 2024
Collaborator Sponsor

    fox:  
      min_score: .65
      threshold: .7

this is a bit high, I would suggest min_score of 0.5 and perhaps lower threshold somewhat to see if that improves the detection.

Can you confirm what the contents are of the .json file inconfig/model_cache, specifically looking to see the labelmap

6 replies

NickM-27 Nov 6, 2024
Collaborator Sponsor

Yes you absolutely should correct otherwise it is just confusing the model

blakeblackshear Nov 6, 2024
Maintainer

You aren't alone. There are others that have been substituting labels like this. I have some ideas for how to use suggestions to swap labels in the future.

ElectronicBattle Nov 6, 2024
Author

OK, all understood. I made the mistake of leaving the mis-identified labels alone rather than changing them to say "this isn't a fox", believing that the absence (at the time) of not having a "fox" label to substitute back in would make things worse.

I'll edit as many of the existing foxes as I can and I'll resubmit a new model request in the next few weeks once there are a few hundred corrections.

I presume the old images , when re-labelled, will contribute to the new training along with any new foxes which get detected, so it should not be too hard to correct my mistake, is that true?

Thanks both for the help and I had better get started!

ElectronicBattle Nov 7, 2024
Author

I have now corrected the labels from "cat" or "dog" to "fox" for all of my previous fox images (June of this year until now) and my label stats now look like this, so I have submitted and received a new model:

I have also adjusted the

fox:  
      min_score: .5
      threshold: .6

Hopefully I will now see two things happen:

a return to foxes being detected (I don't know why Frigate seemed to have gone "blind" these last few days since installing the first new 2024.3 model)
correct labelling of them as foxes now that I have corrected my mistakes.

edit: in the three minutes it took me to update my config and reboot and write some of this reply:

Working!

Answer selected by NickM-27

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Config Support]: seemingly refusing to detect foxes! #14820

{{title}}

Replies: 1 comment 6 replies

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

Select a reply

[Config Support]: seemingly refusing to detect foxes! #14820

ElectronicBattle Nov 5, 2024

Describe the problem you are having

Version

Frigate config file

Relevant Frigate log output

Relevant go2rtc log output

Frigate stats

Operating system

Install method

docker-compose file or Docker CLI command

Object Detector

Screenshots of the Frigate UI's System metrics pages

Any other information that may be helpful

Replies: 1 comment · 6 replies

NickM-27 Nov 5, 2024 Collaborator Sponsor

NickM-27 Nov 6, 2024 Collaborator Sponsor

blakeblackshear Nov 6, 2024 Maintainer

ElectronicBattle Nov 6, 2024 Author

ElectronicBattle Nov 7, 2024 Author

ElectronicBattle
Nov 5, 2024

Replies: 1 comment 6 replies

NickM-27
Nov 5, 2024
Collaborator Sponsor

NickM-27 Nov 6, 2024
Collaborator Sponsor

blakeblackshear Nov 6, 2024
Maintainer

ElectronicBattle Nov 6, 2024
Author

ElectronicBattle Nov 7, 2024
Author