Duplicates

[smnorris]

Describe the problem

See output of https://github.com/smnorris/bcfishpass/blob/main/01_prep/02_pscis/sql/05_pscis_points_duplicates.sql

Currently there are >200 crossings that are likely to be duplicates (487 records in the duplicates table, but each record from a set of duplicates is included)

The usual reasons for duplication include (but are not limited to):

• re-assessments that did not include the original stream_crossing_id in the reassessment submission
• separate assessments of two pipes at a single crossing (contrary to PSCIS assessment guide, mostly just in older assessments)
• two crossings have been snapped to the same stream when one is on an unmapped stream

Hot spots (10 or more):

 ALBN                 |    10
 GRAI                 |    10
 LARL                 |    10
 TABR                 |    10
 HORS                 |    12
 REVL                 |    16
 KOTL                 |    16
 SETN                 |    16
 HOLB                 |    17
 KETL                 |    18
 PARS                 |    20
 MORI                 |    20
 LFRA                 |    22
 KOTR                 |    24
 ELKR                 |    28
 SALM                 |    29
 BULK                 |    32
 MCGR                 |    32