Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SCOOP crashes depending on network connection. #45

Open
erp12 opened this issue Jul 18, 2016 · 1 comment
Open

SCOOP crashes depending on network connection. #45

erp12 opened this issue Jul 18, 2016 · 1 comment

Comments

@erp12
Copy link

erp12 commented Jul 18, 2016

I have been using SCOOP with DEAP for while now, and I really enjoy how simple it is. I have run into one annoying problem that I am hoping is a simple fix.

When I run a script that uses SCOOP at home (or most other internet connections) it works fine. I was running into trouble when I tried to run the same scripts on my work's internet connection. When I switched to the public internet of the building next door, it began working again. I get the same SCOOP error when I am on my work's internet when I disable my internet connection all together.

I believe I read somewhere that SCOOP uses networking to communicate between workers. I also know that SCOOP has the capability of pinging out to other machines on the local network to see if it is being called as part of a cluster. I assume something in this realm is the source of the issue.

I am hoping there is a simple way to tell SCOOP to only use the cores on the local machine, and not to attempt to reach out across the network, as some connections (or no connections) make that impossible.

If such a workaround does or does not exist, I think it would be nice for the error reported to the user to explain more about what has gone wrong, and perhaps point to specific conditions which SCOOP must be run under.

Or perhaps I am completely wrong about what is happening here. Any help would be much appreciated.

Below is the error that I get when I run a script that uses SCOOP (python -m scoop my_file.py) at work or with my internet connection disabled:

Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/runpy.py", line 162, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/runpy.py", line 72, in _run_code
    exec code in run_globals
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/scoop/__main__.py", line 18, in <module>
    from .launcher import main
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/scoop/launcher.py", line 31, in <module>
    from scoop import utils
@soravux
Copy link
Owner

soravux commented Jul 20, 2016

Thanks for reaching out. I'm happy to see that SCOOP works for you at home.
I have a guess about what's happening when you disable your internet connection: SCOOP queries the networking module to get an identifier of the local system. When you disable every interface, this particular query fails (as it cannot get an identifier). By the way, I'm not sure I understand what you mean by your work's internet connection which is a disabled interface?

As to why it isn't reported more appropriately in a Python traceback, I am not sure. My best bet would be that most of SCOOP runs into a 'sandbox' generated by runpy. When the identity query call fails, if it does not fail at runtime but import time, that would explain what you are seeing (and runpy eats away the error message).

Now, as per the fix, I am of course welcoming any pull requests. These times, I am a bit overwhelmed with other projects, so I can't guarantee a quick fix on my part. If you are ready to mess a bit with the code, here's what I would try: on scoop/utils.py, try replacing the call to socket.getfqdn() to a static string of your choice. I believe it could be the culprit in your problem.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants