Failure to connect cellxgene-VIP functionality

[DK-RBS]

With the default cellxgene binary provided with the package, gateway launches cellxgene successfully. However, I am trying to launch cellxgene with VIP plugin. To do this I provided the cellxgene binary from the cellxgene-VIP installation inside CELLXGENE_LOCATION variable. The homepage loads fine. But it results in an error when launching cellxgene from the filecrawler.html. Attached is the error message. On the webpage it shows as internal server error.

Any suggestions on how to enable VIP plugin from within the gateway. I am using python-3.8 for both gateway and VIP installations.

[HarryMWinters]

Fwiw it's worth I've been able to get this to work with CellxGene VIP. I'll post the dockerfile I'm using in the next couple days.

[HarryMWinters]

Here's the dockerfile. For longterm maintenance the versions really should be pinned but I didn't bother since I've got a working image.

FROM ubuntu:oracular-20241009

RUN apt-get update

# Install conda pre-requisites
RUN apt-get install -y \
    curl \
    git

# Download conda installer
RUN curl -O https://repo.anaconda.com/archive/Anaconda3-2024.10-1-Linux-x86_64.sh

# Run conda installer
RUN bash Anaconda3-2024.10-1-Linux-x86_64.sh -b && \
    rm Anaconda3-2024.10-1-Linux-x86_64.sh

# Get around bug in shell executable and conda
RUN rm /bin/sh && ln -s /bin/bash /bin/sh
RUN source /root/anaconda3/bin/activate \
    && conda init \
    && conda config --set channel_priority flexible

# Install CellXGene VIP Enviroment
RUN git clone https://github.com/interactivereport/cellxgene_VIP.git
WORKDIR /cellxgene_VIP

# Install R non-interactively
RUN DEBIAN_FRONTEND=noninteractive \
    TZ=Etc/UTC \
    apt-get install -y r-base

# Install CellXGene VIP Environment
RUN source /root/anaconda3/bin/activate \
    && conda env create -n cellxgene_vip -f VIP.yml

# Install CellXGene
RUN apt-get install -y cpio
RUN source /root/anaconda3/bin/activate \
    && conda activate cellxgene_vip && ./config.sh

# Install R packages c-libs
RUN apt-get install -y \
    build-essential \
    libcurl4-gnutls-dev \
    libxml2-dev \
    libssl-dev

# Install R Packages
COPY install_r_packages.sh /cellxgene_VIP
RUN  ./install_r_packages.sh

# Setup CellXGene Gateway
RUN source /root/anaconda3/bin/activate \
    && conda activate cellxgene_vip \
    && pip install cellxgene-gateway

# Run Server
COPY entrypoint.sh /cellxgene_VIP
ENTRYPOINT ["./entrypoint.sh"]

And the entrypoint script

#!/bin/bash
set -uex

# Remove first argument so it doesn't break conda
shift 1

source /root/anaconda3/bin/activate
conda activate cellxgene_vip

export CELLXGENE_DATA=/data
export CELLXGENE_LOCATION=`which cellxgene`

export GATEWAY_ENABLE_ANNOTATIONS=1
export GATEWAY_LOG_LEVEL=DEBUG

cellxgene-gateway

And the R package install script

export LIBARROW_MINIMAL=false
unset R_LIBS_USER
#  ensure that the right instance of R is used. e.g. system-wide: /bin/R or /usr/bin/R ; local R under conda: ~/.conda/envs/VIP_conda_R/bin/R
which R
#  The version of a Bioconductor package is controlled by BiocManager, whose version is provided

R -q -e 'if(!require(devtools)) install.packages("devtools",repos = "http://cran.us.r-project.org")'
R -q -e 'if(!require(Cairo)) devtools::install_version("Cairo",version="1.5-12",repos = "http://cran.us.r-project.org")'
R -q -e 'if(!require(foreign)) devtools::install_version("foreign",version="0.8-76",repos = "http://cran.us.r-project.org")'
R -q -e 'if(!require(ggpubr)) devtools::install_version("ggpubr",version="0.3.0",repos = "http://cran.us.r-project.org")'
R -q -e 'if(!require(ggrastr)) devtools::install_version("ggrastr",version="0.2.1",repos = "http://cran.us.r-project.org")'
R -q -e 'if(!require(arrow)) devtools::install_version("arrow",version="2.0.0",repos = "http://cran.us.r-project.org")'
R -q -e 'if(!require(Seurat)) devtools::install_version("Seurat",version="3.2.3",repos = "http://cran.us.r-project.org")'
R -q -e 'if(!require(rmarkdown)) devtools::install_version("rmarkdown",version="2.5",repos = "http://cran.us.r-project.org")'
R -q -e 'if(!require(tidyverse)) devtools::install_version("tidyverse",version="1.3.0",repos = "http://cran.us.r-project.org")'
R -q -e 'if(!require(viridis)) devtools::install_version("viridis",version="0.5.1",repos = "http://cran.us.r-project.org")'
R -q -e 'if(!require(hexbin)) devtools::install_version("hexbin",version="1.28.2",repos = "http://cran.us.r-project.org")'
R -q -e 'if(!require(ggforce)) devtools::install_version("ggforce",version="0.3.3",repos = "http://cran.us.r-project.org")'
R -q -e 'if(!require(RcppRoll)) devtools::install_version("RcppRoll",version="0.3.0",repos = "http://cran.r-project.org")'
R -q -e 'if(!require(fastmatch)) devtools::install_version("fastmatch",version="1.1-3",repos = "http://cran.r-project.org")'
R -q -e 'if(!require(BiocManager)) devtools::install_version("BiocManager",version="1.30.10",repos = "http://cran.us.r-project.org")'
R -q -e 'if(!require(fgsea)) BiocManager::install("fgsea")'
R -q -e 'if(!require(rtracklayer)) BiocManager::install("rtracklayer")'
R -q -e 'if(!require(rjson)) devtools::install_version("rjson",version="0.2.20",repos = "https://cran.us.r-project.org")'
R -q -e 'if(!require(ComplexHeatmap)) BiocManager::install("ComplexHeatmap")'

# Needed for Density Scatter plot
R -q -e 'if(!require(ggplot2)) devtools::install_version("ggplot2", repos = "http://cran.us.r-project.org")'

# These should be already installed as dependencies of above packages
R -q -e 'if(!require(dbplyr)) devtools::install_version("dbplyr",version="1.0.2",repos = "http://cran.us.r-project.org")'
R -q -e 'if(!require(RColorBrewer)) devtools::install_version("RColorBrewer",version="1.1-2",repos = "http://cran.us.r-project.org")'
R -q -e 'if(!require(glue)) devtools::install_version("glue",version="1.4.2",repos = "http://cran.us.r-project.org")'
R -q -e 'if(!require(gridExtra)) devtools::install_version("gridExtra",version="2.3",repos = "http://cran.us.r-project.org")'
R -q -e 'if(!require(ggrepel)) devtools::install_version("ggrepel",version="0.8.2",repos = "http://cran.us.r-project.org")'
R -q -e 'if(!require(MASS)) devtools::install_version("MASS",version="7.3-51.6",repos = "http://cran.us.r-project.org")'

# This one won't install
# R -q -e 'if(!require(data.table)) devtools::install_version("data.table",version="1.13.0",repos = "http://cran.us.r-project.org")'

And to build:

docker build \
    --tag cellxgene_gateway \
    --file cellxgene_gateway.dockerfile \
    .

And to run

docker run \
    -it \
    --rm \
    --publish "5005:5005" \
    --volume "/path/to/host/h5ad/file/directory":"/data" \
    cellxgene_gateway > cellxgene_gateway.log 2>&1

[eusebiu]

I had to use set -x as the container exited.

[eusebiu]

I also get when I try to use it:

405<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 Final//EN">
<title>405 Method Not Allowed</title>
<h1>Method Not Allowed</h1>
<p>The method is not allowed for the requested URL.</p>

[HarryMWinters]

What was the method, URL, and path to get that 405 response? Something like curl localhost:5005?

[eusebiu]

Sorry, I meant when using VIP (something like POST 127.0.0.1:8000). The cellxgene and gateway work fine; only VIP is showing that error.

[HarryMWinters]

Hey @eusebiu,

If the problem is only occurring when trying to access VIP functionality it likely makes sense to bring it up in the CellxGene VIP repo. If, however, everything is working fine until you add CellxGene Gateway maybe it makes sense to try and solve it here.

When you do it will be very helpful to include:

1. The actual network (HTTP) calls that are being made. You can get this with curl, postman, or most easily with the developer tools on a browser. To debug people will need to know exactly whats failing.
2. The logs from the server. These should be streaming into the terminal where you started the CellxGene program. If not there's likely a log file around. This might have redundant information about the HTTP calls but it will also have (hopefully) more information about what went wrong.
3. What environment and software versions your running, especially for CellxGene and CellxGene VIP.

Good luck!

[eusebiu]

Hey @HarryMWinters !
Thanks for your response! I am using the Dockerfile above and the .sh files as described (on a RHEL).

bash-4.4$ docker -v
Docker version 27.3.1, build ce12230

The error is - apparently the POST is not allowed (I remember that this was overridden by VIP but I don't understand why doesn't it work as the config.sh is executed in your Dockerfile):

The log is:

+ export CELLXGENE_DATA=/cellxgene-data
+ CELLXGENE_DATA=/cellxgene-data
++ which cellxgene
+ export CELLXGENE_LOCATION=/root/anaconda3/envs/cellxgene_vip/bin/cellxgene
+ CELLXGENE_LOCATION=/root/anaconda3/envs/cellxgene_vip/bin/cellxgene
+ export GATEWAY_ENABLE_ANNOTATIONS=1
+ GATEWAY_ENABLE_ANNOTATIONS=1
+ export GATEWAY_LOG_LEVEL=DEBUG
+ GATEWAY_LOG_LEVEL=DEBUG
+ cellxgene-gateway
2024-12-09 17:38:21,339:cellxgene_gateway:INFO:Got required env: {'CELLXGENE_LOCATION': '/root/anaconda3/envs/cellxgene_vip/bin/cellxgene'}
2024-12-09 17:38:21,339:cellxgene_gateway:INFO:Got optional env: {'EXTERNAL_HOST': 'localhost:5005', 'EXTERNAL_PROTOCOL': None, 'GATEWAY_IP': None, 'GATEWAY_PORT': 5005, 'GATEWAY_EXTRA_SCRIPTS': None, 'GATEWAY_EXPIRE_SECONDS': 3600, 'GATEWAY_ENABLE_ANNOTATIONS': True, 'GATEWAY_ENABLE_BACKED_MODE': False, 'GATEWAY_LOG_LEVEL': 10, 'CELLXGENE_ARGS': None, 'CELLXGENE_DATA': '/cellxgene-data', 'PROXY_FIX_FOR': 0, 'PROXY_FIX_PROTO': 0, 'PROXY_FIX_HOST': 0, 'PROXY_FIX_PORT': 0, 'PROXY_FIX_PREFIX': 0}
 * Serving Flask app "cellxgene_gateway.gateway" (lazy loading)
 * Environment: production
�[31m   WARNING: This is a development server. Do not use it in a production deployment.�[0m
�[2m   Use a production WSGI server instead.�[0m
 * Debug mode: off
2024-12-09 17:38:21,361:werkzeug:INFO: * Running on http://0.0.0.0:5005/ (Press CTRL+C to quit)
2024-12-09 17:39:21,400:cellxgene_gateway.prune_process_cache:DEBUG:Cutoff 1733762361.400127 = timestamp 1733765961.400127 - expire seconds 3600 , keeping [], pruning []
view path=pbmc3k.h5ad/, source_name=None, dataset=/cellxgene-data/pbmc3k.h5ad, annotation_file= None, key=pbmc3k.h5ad, source=local
2024-12-09 17:39:26,188:cellxgene_gateway.subprocess_backend:INFO:launching yes | /root/anaconda3/envs/cellxgene_vip/bin/cellxgene launch /cellxgene-data/pbmc3k.h5ad --port 8000 --host 127.0.0.1 --disable-annotations --disable-gene-sets-save
...
2024-12-09 17:40:27,859:werkzeug:INFO:172.22.0.1 - - [09/Dec/2024 17:40:27] "�[37mGET /view/pbmc3k.h5ad/static/main-af87f9c6698c8303d711.js.map HTTP/1.1�[0m" 200 -
2024-12-09 17:40:56,452:urllib3.connectionpool:DEBUG:Starting new HTTP connection (1): 127.0.0.1:8000
2024-12-09 17:40:56,454:urllib3.connectionpool:DEBUG:http://127.0.0.1:8000 "POST / HTTP/1.1" 405 178
2024-12-09 17:40:56,455:werkzeug:INFO:172.22.0.1 - - [09/Dec/2024 17:40:56] "�[31m�[1mPOST /view/pbmc3k.h5ad/ HTTP/1.1�[0m" 405 -
2024-12-09 17:41:21,468:cellxgene_gateway.prune_process_cache:DEBUG:Cutoff 1733762481.468447 = timestamp 1733766081.468447 - expire seconds 3600 , keeping [<cellxgene_gateway.cache_entry.CacheEntry object at 0x7fbde459dcd0>], pruning []
2024-12-09 17:42:21,509:cellxgene_gateway.prune_process_cache:DEBUG:Cutoff 1733762541.509547 = timestamp 1733766141.509547 - expire seconds 3600 , keeping [<cellxgene_gateway.cache_entry.CacheEntry object at 0x7fbde459dcd0>], pruning []

[HarryMWinters]

That's very helpful. I don't see an obvious reason why it would not be working.

A couple thoughts:

• Do any of the other VIP plots work?
• What is in the body of the post request?
• I noticed in your screen shot that your Selection 2 has zero cells. Have you tried it when all selections have at least one cell? It could be an edge-case bug in VIP.
• This is seeming more and more like a VIP bug rather then a CxG Gateway issue.

[eusebiu]

None of the plots work.

The payload is: