shed-tools Intermittent Connection Error

[dwightkuo]

I get an error when using shed-tools. This seems to occur intermittently and never repeats at the same place (when deleting the installed tools via the GUI and restarting the installation process via shed-tools).

shed-tools install -g http://127.0.0.1:8080/ -a 9da1b0b76e038a2a70e0649adb5205fa -t files/sc1galaxy-tools.yml --install_resolver_dependencies --latest --log_file logfile.txt
...
Traceback (most recent call last):
  File "/remote/home/kuop/galaxy/.venv/bin/shed-tools", line 11, in <module>
    sys.exit(main())
  File "/remote/home/kuop/galaxy/.venv/lib/python2.7/site-packages/ephemeris/shed_tools.py", line 937, in main
    install_tool_manager.install_repositories()
  File "/remote/home/kuop/galaxy/.venv/lib/python2.7/site-packages/ephemeris/shed_tools.py", line 716, in install_repositories
    install_repository_revision(repository, self.tsc)
  File "/remote/home/kuop/galaxy/.venv/lib/python2.7/site-packages/ephemeris/shed_tools.py", line 517, in install_repository_revision
    response = tool_shed_client.install_repository_revision(**repository)
  File "/remote/home/kuop/galaxy/.venv/lib/python2.7/site-packages/bioblend/galaxy/toolshed/__init__.py", line 146, in install_repository_revision
    return self._post(url=url, payload=payload)
  File "/remote/home/kuop/galaxy/.venv/lib/python2.7/site-packages/bioblend/galaxy/client.py", line 160, in _post
    files_attached=files_attached)
  File "/remote/home/kuop/galaxy/.venv/lib/python2.7/site-packages/bioblend/galaxyclient.py", line 137, in make_post_request
    timeout=self.timeout)
  File "/remote/home/kuop/galaxy/.venv/lib/python2.7/site-packages/requests/api.py", line 112, in post
    return request('post', url, data=data, json=json, **kwargs)
  File "/remote/home/kuop/galaxy/.venv/lib/python2.7/site-packages/requests/api.py", line 58, in request
    return session.request(method=method, url=url, **kwargs)
  File "/remote/home/kuop/galaxy/.venv/lib/python2.7/site-packages/requests/sessions.py", line 508, in request
    resp = self.send(prep, **send_kwargs)
  File "/remote/home/kuop/galaxy/.venv/lib/python2.7/site-packages/requests/sessions.py", line 618, in send
    r = adapter.send(request, **kwargs)
  File "/remote/home/kuop/galaxy/.venv/lib/python2.7/site-packages/requests/adapters.py", line 490, in send
    raise ConnectionError(err, request=request)
requests.exceptions.ConnectionError: ('Connection aborted.', BadStatusLine("''",))

[rhpvorderman]

This is a duplicate of #98 .
Thanks a lot for reporting the error. It is on our to do list.

[dwightkuo]

A side effect of this error is that the tool that was being installed when the error occurs can no longer be installed. When trying to reinstall the tool again via ephemeris it says that the tool is already installed but it is not listed in the Tools Menu. Additionally, when viewing the "Manage Tools" list in the Galaxy Admin menu, the tool is not listed and cannot be uninstalled either.

[rhpvorderman]

That might have something to do with galaxy's internal database. It also stores whether tools are installed or not, but I am no expert on this. Maybe @mvdbeek or @jmchilton knows more about this.

[mvdbeek]

Sorry, no idea. What do you see in galaxy's logfile if this happens ?

[dwightkuo]

I don't see anything really out of the ordinary in galaxy's logfile when this happens. I've pasted it below and masked the IP and key
X.X.X.X - - [23/Aug/2018:09:19:27 -0700] "GET /api/tools?in_panel=True&key=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX HTTP/1.1" 200 - "-" "python-requests/2.19.1"
[pid: 43003|app: 0|req: 36/136] X.X.X.X () {32 vars in 543 bytes} [Thu Aug 23 09:19:27 2018] GET /api/tools?in_panel=True&key=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX => generated 93512 bytes in 14 msecs (HTTP/1.1 200) 3 headers in 124 bytes (1 switches on core 2)
X.X.X.X - - [23/Aug/2018:09:19:27 -0700] "GET /api/tools?in_panel=False&key=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX HTTP/1.1" 200 - "-" "python-requests/2.19.1"
[pid: 43008|app: 0|req: 34/137] X.X.X.X () {32 vars in 545 bytes} [Thu Aug 23 09:19:27 2018] GET /api/tools?in_panel=False&key=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX => generated 110139 bytes in 9 msecs (HTTP/1.1 200) 3 headers in 124 bytes (1 switches on core 3)
X.X.X.X - - [23/Aug/2018:09:19:27 -0700] "GET /api/tool_shed_repositories?key=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX HTTP/1.1" 200 - "-" "python-requests/2.19.1"
[pid: 43014|app: 0|req: 33/138] X.X.X.X () {32 vars in 549 bytes} [Thu Aug 23 09:19:27 2018] GET /api/tool_shed_repositories?key=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX => generated 117646 bytes in 375 msecs (HTTP/1.1 200) 3 headers in 124 bytes (1 switches on core 0)
X.X.X.X - - [23/Aug/2018:09:19:28 -0700] "POST /api/tool_shed_repositories/new/install_repository_revision?key=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX HTTP/1.1" 200 - "-" "python-requests/2.19.1"
[pid: 43008|app: 0|req: 35/139] X.X.X.X () {36 vars in 667 bytes} [Thu Aug 23 09:19:28 2018] POST /api/tool_shed_repositories/new/install_repository_revision?key=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX => generated 131 bytes in 977 msecs (HTTP/1.1 200) 3 headers in 124 bytes (1 switches on core 0)