Bug to run the SAMap algorithm

[xiangyuma1995]

I am trying to run the SAMap algorithm using the commands shown in the SAMap tutorial (All data and commands were obtained from SAMap tutorial). But command fail with the following error message:
sm.run(pairwise=True)
samap = sm.samap # SAM object with three species stitched together

Calculating gene-gene correlations in the homology graph...

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
Cell In[10], [line 1](vscode-notebook-cell:?execution_count=10&line=1)
----> [1](vscode-notebook-cell:?execution_count=10&line=1) sm.run(pairwise=True)
      [2](vscode-notebook-cell:?execution_count=10&line=2) samap = sm.samap # SAM object with three species stitched together

File [c:\Users\MXY\.conda\envs\myenv2\lib\site-packages\samap\mapping.py:298](file:///C:/Users/MXY/.conda/envs/myenv2/lib/site-packages/samap/mapping.py:298), in SAMAP.run(self, NUMITERS, NHS, crossK, N_GENE_CHUNKS, umap, ncpus, hom_edge_thr, hom_edge_mode, scale_edges_by_corr, neigh_from_keys, pairwise)
    [294](file:///C:/Users/MXY/.conda/envs/myenv2/lib/site-packages/samap/mapping.py:294)         neigh_from_keys[sid] = False
    [296](file:///C:/Users/MXY/.conda/envs/myenv2/lib/site-packages/samap/mapping.py:296) start_time = time.time()
--> [298](file:///C:/Users/MXY/.conda/envs/myenv2/lib/site-packages/samap/mapping.py:298) smap.run(
    [299](file:///C:/Users/MXY/.conda/envs/myenv2/lib/site-packages/samap/mapping.py:299)     NUMITERS=NUMITERS,
    [300](file:///C:/Users/MXY/.conda/envs/myenv2/lib/site-packages/samap/mapping.py:300)     NHS=NHS,
    [301](file:///C:/Users/MXY/.conda/envs/myenv2/lib/site-packages/samap/mapping.py:301)     K=crossK,
    [302](file:///C:/Users/MXY/.conda/envs/myenv2/lib/site-packages/samap/mapping.py:302)     NCLUSTERS=N_GENE_CHUNKS,
    [303](file:///C:/Users/MXY/.conda/envs/myenv2/lib/site-packages/samap/mapping.py:303)     ncpus=ncpus,
    [304](file:///C:/Users/MXY/.conda/envs/myenv2/lib/site-packages/samap/mapping.py:304)     THR=hom_edge_thr,
    [305](file:///C:/Users/MXY/.conda/envs/myenv2/lib/site-packages/samap/mapping.py:305)     corr_mode=hom_edge_mode,
    [306](file:///C:/Users/MXY/.conda/envs/myenv2/lib/site-packages/samap/mapping.py:306)     scale_edges_by_corr = scale_edges_by_corr,
    [307](file:///C:/Users/MXY/.conda/envs/myenv2/lib/site-packages/samap/mapping.py:307)     neigh_from_keys=neigh_from_keys,
    [308](file:///C:/Users/MXY/.conda/envs/myenv2/lib/site-packages/samap/mapping.py:308)     pairwise=pairwise
    [309](file:///C:/Users/MXY/.conda/envs/myenv2/lib/site-packages/samap/mapping.py:309) )
    [310](file:///C:/Users/MXY/.conda/envs/myenv2/lib/site-packages/samap/mapping.py:310) samap = smap.final_sam
    [311](file:///C:/Users/MXY/.conda/envs/myenv2/lib/site-packages/samap/mapping.py:311) self.samap = samap

File [c:\Users\MXY\.conda\envs\myenv2\lib\site-packages\samap\mapping.py:695](file:///C:/Users/MXY/.conda/envs/myenv2/lib/site-packages/samap/mapping.py:695), in _Samap_Iter.run(self, NUMITERS, NHS, K, corr_mode, NCLUSTERS, scale_edges_by_corr, THR, neigh_from_keys, pairwise, ncpus)
...
    [364](file:///C:/Users/MXY/.conda/envs/myenv2/lib/site-packages/scipy/sparse/_coo.py:364) data = np.empty_like(self.data, dtype=self.dtype)
--> [366](file:///C:/Users/MXY/.conda/envs/myenv2/lib/site-packages/scipy/sparse/_coo.py:366) coo_tocsr(M, N, nnz, major, minor, self.data, indptr, indices, data)
    [367](file:///C:/Users/MXY/.conda/envs/myenv2/lib/site-packages/scipy/sparse/_coo.py:367) return indptr, indices, data, self.shape

ValueError: Output dtype not compatible with inputs.
Output is truncated. View as a [scrollable element](command:cellOutput.enableScrolling?e457b312-ff16-41cd-bf7b-c58a22b274ad) or open in a [text editor](command:workbench.action.openLargeOutput?e457b312-ff16-41cd-bf7b-c58a22b274ad). Adjust cell output [settings](command:workbench.action.openSettings?%5B%22%40tag%3AnotebookOutputLayout%22%5D)...

[jakubovciak]

Hi all, the same here:

ValueError                                Traceback (most recent call last)
Cell In[7], line 1
----> 1 sm.run()
      2 samap = sm.samap # SAM object with three species stitched together

File [~/miniconda3/envs/SAMap/lib/python3.9/site-packages/samap/mapping.py:296](http://localhost:8789/lab/tree/test_workdir/~/miniconda3/envs/SAMap/lib/python3.9/site-packages/samap/mapping.py#line=295), in SAMAP.run(self, NUMITERS, NHS, crossK, N_GENE_CHUNKS, umap, ncpus, hom_edge_thr, hom_edge_mode, scale_edges_by_corr, neigh_from_keys, pairwise)
    292         neigh_from_keys[sid] = False
    294 start_time = time.time()
--> 296 smap.run(
    297     NUMITERS=NUMITERS,
    298     NHS=NHS,
    299     K=crossK,
    300     NCLUSTERS=N_GENE_CHUNKS,
    301     ncpus=ncpus,
    302     THR=hom_edge_thr,
    303     corr_mode=hom_edge_mode,
    304     scale_edges_by_corr = scale_edges_by_corr,
    305     neigh_from_keys=neigh_from_keys,
    306     pairwise=pairwise
    307 )
    308 samap = smap.final_sam
    309 self.samap = samap

File [~/miniconda3/envs/SAMap/lib/python3.9/site-packages/samap/mapping.py:726](http://localhost:8789/lab/tree/test_workdir/~/miniconda3/envs/SAMap/lib/python3.9/site-packages/samap/mapping.py#line=725), in _Samap_Iter.run(self, NUMITERS, NHS, K, corr_mode, NCLUSTERS, scale_edges_by_corr, THR, neigh_from_keys, pairwise, ncpus)
    724 print("Calculating gene-gene correlations in the homology graph...")
    725 self.samap = sam4
--> 726 gnnmu = self.refine_homology_graph(ncpus = ncpus,  NCLUSTERS = NCLUSTERS,  THR=THR, corr_mode=corr_mode)
    728 self.GNNMS_corr.append(gnnmu)
    729 self.gnnmu = gnnmu

File [~/miniconda3/envs/SAMap/lib/python3.9/site-packages/samap/mapping.py:653](http://localhost:8789/lab/tree/test_workdir/~/miniconda3/envs/SAMap/lib/python3.9/site-packages/samap/mapping.py#line=652), in _Samap_Iter.refine_homology_graph(self, NCLUSTERS, ncpus, THR, corr_mode, wscale)
    650 keys = self.keys
    651 sam4 = self.samap
--> 653 gnnmu = _refine_corr(
    654     sams,
    655     sam4,
    656     gnnm,
    657     gns_dict,
    658     THR=THR,
    659     use_seq=False,
    660     T1=0,
    661     NCLUSTERS=NCLUSTERS,
    662     ncpus=ncpus,
    663     corr_mode=corr_mode,
    664     wscale=wscale
    665 )
    666 return gnnmu

File [~/miniconda3/envs/SAMap/lib/python3.9/site-packages/samap/mapping.py:1004](http://localhost:8789/lab/tree/test_workdir/~/miniconda3/envs/SAMap/lib/python3.9/site-packages/samap/mapping.py#line=1003), in _refine_corr(sams, st, gnnm, gns_dict, corr_mode, THR, use_seq, T1, NCLUSTERS, ncpus, wscale)
   1001     gn = gns_dict[sid]
   1002     gns_dict_sub[sid] = gn[np.in1d(gn,gnsub)]
-> 1004 gnnm2_sub = _refine_corr_parallel(
   1005     sams,
   1006     st,
   1007     gnnm_sub,
   1008     gns_dict_sub,
   1009     corr_mode=corr_mode,
   1010     THR=THR,
   1011     use_seq=use_seq,
   1012     T1=T1,
   1013     ncpus=ncpus,
   1014     wscale=wscale
   1015 )
   1016 GNNMSUBS.append(gnnm2_sub)
   1017 GNSUBS.append(gnsub)

File [~/miniconda3/envs/SAMap/lib/python3.9/site-packages/samap/mapping.py:1540](http://localhost:8789/lab/tree/test_workdir/~/miniconda3/envs/SAMap/lib/python3.9/site-packages/samap/mapping.py#line=1539), in _refine_corr_parallel(sams, st, gnnm, gns_dict, corr_mode, THR, use_seq, T1, ncpus, wscale)
   1536     nnms[-1] = nnms[-1].multiply(1 [/](http://localhost:8789/) s1)
   1538     xs.append(sams[sid].adata[:,gns_dictO[sid]].X.astype("float16"))
-> 1540 Xs = sp.sparse.block_diag(xs).tocsc()
   1541 nnms = sp.sparse.hstack(nnms).tocsr()
   1542 Xavg = nnms.dot(Xs).tocsc()

File [~/miniconda3/envs/SAMap/lib/python3.9/site-packages/scipy/sparse/_coo.py:311](http://localhost:8789/lab/tree/test_workdir/~/miniconda3/envs/SAMap/lib/python3.9/site-packages/scipy/sparse/_coo.py#line=310), in _coo_base.tocsc(self, copy)
    309 else:
    310     from ._csc import csc_array
--> 311     indptr, indices, data, shape = self._coo_to_compressed(csc_array._swap)
    313     x = self._csc_container((data, indices, indptr), shape=shape)
    314     if not self.has_canonical_format:

File [~/miniconda3/envs/SAMap/lib/python3.9/site-packages/scipy/sparse/_coo.py:366](http://localhost:8789/lab/tree/test_workdir/~/miniconda3/envs/SAMap/lib/python3.9/site-packages/scipy/sparse/_coo.py#line=365), in _coo_base._coo_to_compressed(self, swap)
    363 indices = np.empty_like(minor, dtype=idx_dtype)
    364 data = np.empty_like(self.data, dtype=self.dtype)
--> 366 coo_tocsr(M, N, nnz, major, minor, self.data, indptr, indices, data)
    367 return indptr, indices, data, self.shape

ValueError: Output dtype not compatible with inputs.

[GGboy-Zzz]

ok, I also met this problem, whereas I just complete its pipeline one day ago. I don't know what happend. @atarashansky

[jordan841220]

I have encountered the same issue while running sm.run(pairwise=True).
Any work around please?

[Zruni0710]

Hi all, I met the same question, but the same commands for my partner's SAMap was OK .
He installed SAMap several weeks earlier than me, I guess this error mitght be related to the SAMap installation.

[GGboy-Zzz]

I downgraded the 'scipy' package to '1.12.0', this problem could be solved.

[atarashansky]

Hi, thanks for flagging this. I'll update the requirements to pin the scipy package and then take a look to see what the breaking change is with the recent dependency upgrade to 1.13.0.

[dsb66]

I ran into a related error (ValueError: Output dtype not compatible with inputs.) in mapping.py at line 1538:
xs.append(sams[sid].adata[:,gns_dictO[sid]].X.astype("float16"))

Changing float16 to float32 fixed the problem, without much change in memory requirements. I think new versions of scipy do not take float16 with sparse matrices.

[atarashansky]

I ran into a related error (ValueError: Output dtype not compatible with inputs.) in mapping.py at line 1538:
xs.append(sams[sid].adata[:,gns_dictO[sid]].X.astype("float16"))

Thanks for reporting!

Is this only an issue for 1.13.0?

[dsb66]

I don't know, I have not tested. In any case I think that the restriction numpy=1.23.5 prevents pulling much newer versions of scipy.