Gene Pair Finder Giving "ValueError: negative dimensions are not allowed"

[acgarretson]

I'm running gene pair finder on the output of a SAMap run, and the sam results themselves look great.

I am running find_all or find_pairs like this:
gpf = GenePairFinder(sm, keys=keys) gene_pairs = gpf.find_all(align_thr=0.2)

I get the following error:
>>> gpf.find_all(align_thr=0.9) Calculating gene pairs for the mapping: at;10 to sc;2 Traceback (most recent call last): File "<stdin>", line 1, in <module> File ".micromamba/envs/SAMap/lib/python3.9/site-packages/samap/analysis.py", line 689, in find_all res['{};{}'.format(ct1[i],ct2[i])] = self.find_genes(ct1[i],ct2[i],**kwargs) File ".micromamba/envs/SAMap/lib/python3.9/site-packages/samap/analysis.py", line 746, in find_genes m,gpairs = self._find_link_genes_avg(n1, n2, id1,id2, w1t=w1t, w2t=w2t, expr_thr=0.05) File ".micromamba/envs/SAMap/lib/python3.9/site-packages/samap/analysis.py", line 790, in _find_link_genes_avg X2 = _sparse_sub_standardize(sam2.adata[:, g2].X[x2 == c2, :], mu2, std2) File ".micromamba/envs/SAMap/lib/python3.9/site-packages/anndata/_core/anndata.py", line 606, in X _subset(self._adata_ref.X, (self._oidx, self._vidx)), File ".micromamba/envs/SAMap/lib/python3.9/functools.py", line 888, in wrapper return dispatch(args[0].__class__)(*args, **kw) File ".micromamba/envs/SAMap/lib/python3.9/site-packages/anndata/_core/index.py", line 140, in _subset_spmatrix return a[subset_idx] File ".micromamba/envs/SAMap/lib/python3.9/site-packages/scipy/sparse/_index.py", line 71, in __getitem__ return self._get_sliceXarray(row, col) File ".micromamba/envs/SAMap/lib/python3.9/site-packages/scipy/sparse/_csr.py", line 321, in _get_sliceXarray return self._major_slice(row)._minor_index_fancy(col) File ".micromamba/envs/SAMap/lib/python3.9/site-packages/scipy/sparse/_compressed.py", line 775, in _minor_index_fancy res_indices = np.empty(nnz, dtype=idx_dtype) ValueError: negative dimensions are not allowed

It has happened with a handful of different datasets at this point, and I'm not sure exactly how to troubleshoot.

[daiyc-zoo]

Hi @acgarretson , did you manage to solve this issue? I've run into the same error whilst running different datasets and I'm unsure what the cause of the issue may be. I've checked gene name matches between the single cell dataset and the blastp dataset etc.

One thing I noticed is when subsetting specific cell types from my dataset, the small test dataset can run properly and produce gene pair finder results. However I'm really unsure what is causing the error on the bigger dataset.

[dnjst]

Hi, I'e been getting this error as well, in my case only in one of the species pairs, tried with latest and with older scipy and anndata libraries.

It seems to happen at the step

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
Cell In[188], line 1
----> 1 gpf.find_genes(n1="Hs_dS3",n2="Mf_dS3")

File /~/conda_envs/SAMap/lib/python3.9/site-packages/samap/analysis.py:746, in GenePairFinder.find_genes(self, n1, n2, w1t, w2t, n_genes, thr)
    743 assert n1 in q(self.sams[id1].adata.obs[self.keys[id1]])
    744 assert n2 in q(self.sams[id2].adata.obs[self.keys[id2]])
--> 746 m,gpairs = self._find_link_genes_avg(n1, n2, id1,id2, w1t=w1t, w2t=w2t, expr_thr=0.05)
    748 self.gene_pair_scores = pd.Series(index=gpairs, data=m)
    750 G = q(gpairs[np.argsort(-m)[:n_genes]])

File /~/conda_envs/SAMap/lib/python3.9/site-packages/samap/analysis.py:789, in GenePairFinder._find_link_genes_avg(self, c1, c2, id1, id2, w1t, w2t, expr_thr)
    786 sam1,sam2 = sams[id1],sams[id2]
    787 mu1,std1,mu2,std2 = mus[id1][g1].values,stds[id1][g1].values,mus[id2][g2].values,stds[id2][g2].values
--> 789 X1 = _sparse_sub_standardize(sam1.adata[:, g1].X[x1 == c1, :], mu1, std1)
    790 X2 = _sparse_sub_standardize(sam2.adata[:, g2].X[x2 == c2, :], mu2, std2)
    791 a, b = sam3.adata.obsp["connectivities"][sam3.adata.obs['species']==id1,:][:,sam3.adata.obs['species']==id2][
    792     x1 == c1, :][:, x2 == c2].nonzero()

File /~/conda_envs/SAMap/lib/python3.9/site-packages/anndata/_core/anndata.py:612, in AnnData.X(self)
    609     X = None
    610 elif self.is_view:
    611     X = as_view(
--> 612         _subset(self._adata_ref.X, (self._oidx, self._vidx)),
    613         ElementRef(self, "X"),
    614     )
    615 else:
    616     X = self._X

File /~/conda_envs/SAMap/lib/python3.9/functools.py:888, in singledispatch.<locals>.wrapper(*args, **kw)
    884 if not args:
    885     raise TypeError(f'{funcname} requires at least '
    886                     '1 positional argument')
--> 888 return dispatch(args[0].__class__)(*args, **kw)

File /~/conda_envs/SAMap/lib/python3.9/site-packages/anndata/_core/index.py:140, in _subset_spmatrix(a, subset_idx)
    138 if len(subset_idx) > 1 and all(isinstance(x, cabc.Iterable) for x in subset_idx):
    139     subset_idx = (subset_idx[0].reshape(-1, 1), *subset_idx[1:])
--> 140 return a[subset_idx]

File /~/conda_envs/SAMap/lib/python3.9/site-packages/scipy/sparse/_index.py:71, in IndexMixin.__getitem__(self, key)
     69         return self._get_sliceXslice(row, col)
     70     elif col.ndim == 1:
---> 71         return self._get_sliceXarray(row, col)
     72     raise IndexError('index results in >2 dimensions')
     73 elif row.ndim == 1:

File /~/conda_envs/SAMap/lib/python3.9/site-packages/scipy/sparse/_csr.py:321, in csr_matrix._get_sliceXarray(self, row, col)
    320 def _get_sliceXarray(self, row, col):
--> 321     return self._major_slice(row)._minor_index_fancy(col)

File /~/conda_envs/SAMap/lib/python3.9/site-packages/scipy/sparse/_compressed.py:775, in _cs_matrix._minor_index_fancy(self, idx)
    773 col_order = np.argsort(idx).astype(idx_dtype, copy=False)
    774 nnz = res_indptr[-1]
--> 775 res_indices = np.empty(nnz, dtype=idx_dtype)
    776 res_data = np.empty(nnz, dtype=self.dtype)
    777 csr_column_index2(col_order, col_offsets, len(self.indices),
    778                   self.indices, self.data, res_indices, res_data)

ValueError: negative dimensions are not allowed