[Bug report] - Missing brain regions labels from subset of units in ephys session

[GaelleChapuis]

Describe the bug
@berkgercek reported that in some sessions, a fraction of units do not have a brain region label associated. Some of those sessions are listed below

[berkgercek]

Here is a basic way of finding the proportion of missing labels:

from ibl_pipeline import subject, acquisition, histology
from ibl_pipeline.analyses import behavior
import uuid
from oneibl import one

one = one.ONE()

labelstable = acquisition.Session * histology.ClusterBrainRegion * subject.Subject * behavior.SessionTrainingStatus &\
    {'good_enough_for_brainwide_map': 1, 'insertion_data_source': 'Ephys aligned histology track'}
bwm_ids = [str(eid) for eid in labelstable.fetch('session_uuid')]
bwm_ids = list(set(bwm_ids))

for eid in bwm_ids:
    sessprobes = np.unique(labelstable & {'session_uuid': uuid.UUID(eid)})
    missing_labels[eid] = {}
    for probe_idx in sessprobes:
        missunits = []
        spk_clu = one.load(eid, dataset_types=['spikes.clusters'], offline=True)[probe_idx]
        clu_ids = np.unique(spk_clu)
        probetable = unitlabels & {'session_uuid': uuid.UUID(eid), 'probe_idx': probe_idx}
        regiondf = probetable.proj('cluster_id', 'acronym').fetch(format='frame').reset_index().set_index('cluster_id')
        for unit in clu_ids:
            if unit not in regiondf.index:
                missunits.append(unit)
        missing_labels[eid][probe_idx] = len(missunits) / clu_ids.shape[0]

Unless there is a bug here it seems I am missing between 2% and 50% of unit labels, unless I've fucked up horrendously.

[oliche]

It seems everything is in order in Alyx, this is how I get the locations.
Maybe this has to do with the Datajoint ingestion ?
Note that this session has several sets of channels (histology and ephys aligned), and several ephys alignments.

Here is how I check the completeness of channels:

from oneibl.one import ONE
from brainbox.io import parquet

one = ONE()
eid = 'aad23144-0e52-4eac-80c5-c4ee2decb198'
traj = one.alyx.rest('trajectories', 'list', session=eid, django='probe_insertion__name,probe01')
channels = one.alyx.rest('channels', 'list', session=eid, trajectory_estimate=traj[0]['id'])
ch = parquet.rec2col(list(channels))

## plot
import ibllib.atlas as atlas
import numpy as np
ba = atlas.AllenAtlas(25)
ax = ba.plot_cslice(np.mean(ch['y']) / 1e6)
ax.plot(ch['x'], ch['z'], '+')

## check completeness and acronyms
from brainbox.core import ismember
isin, i = ismember(ch['brain_region'], ba.regions.id)
assert(np.all(isin))
ch['acronym'] = ba.regions.acronym[i]

[berkgercek]

So to be specific, these are cluster locations rather than just channel locations. My guess is that at ingestion DJ has a table which computes from channel locations unit locations. I'll ping Shan about it.

Individual unit labels don't exist in Alyx, right?