Select random port from a range (new implementation)

[rkdarst]

This implements the random_port selection from #102 in the method suggested by @mbmilligan in that issues.

Unfortunately I had to use testing machinery from #99, which is not pulled yet. So I am basing this on my dev branch, which has a lot of extra junk. The file diffs will not work, but just look at the last commit to see what changed.

• Closes: restricting port range for spawned jupyterhub singleuser servers #102.

[rkdarst]

This fails on JH 0.7.2.

[belfhi]

Hi,
any news on this merge request and/or the new batchspawner release?
I'm currently working on a Jupyterhub instance which needs open ports to the worker nodes.
If any help is needed I can assist if it's possible...

[rkdarst]

I still need to work on an integration branch. But really we should skip that step and work directly on this master, but I am not maintainer so I can't really do much useful...

Maybe I should just use my repo as current master?

[rkdarst]

With the introduction of "select port on remote side", #58 and #141, this no longer works as easily, because the port is chosen remotely where there is no guarantee of any type of config files. Remote port selection also isn't easily configurable. All of these can be fixed, but we need other stuff to be reviewed in order to have a base to develop against.

I see two options, which could be done together:

• Add a option to select on hub side, which has options "random_port", "remote side", or "port range"
• Be able to serialize the options, e.g. "--range=40000-50000" Pass the option to the remote side, where it is used.
• (add configuration to remote sides: I don't like this idea)

How important is this to you still? (cc @pontiggi)

Once someone accepts upcoming patches I'll be taking a look at this again.

[mbmilligan]

I imagine being able to configure a range will be important in fairly short order (HPC admins tend to like restrictive firewall rules), but I'm unclear on the level of expressiveness that would be desirable. My initial inclination is for the wrapper script to look for the range in an environment variable that can then be optionally set in the batch script. Merits further discussion though.

[belfhi]

@rkdarst For me this is a very important feature, it makes requests to the firewall people much easier to be able to give them a range (Jupyterhub and Batch nodes are not in the same subnet).
I still like the hub side option best, with a spawner option to specify the port range.

[mdehollander]

Just to let you know that we would be happy (and the IT admins as well) if a port range can be specified.

[Hoeze]

@rkdarst Is there any updates to this?
I found no possibility at all to select port ranges for the SingleUserNotebookApp s.t. the Jupyterhub can still connect to it.

[pelacables]

Hi, any update on this? this is key feature missing in JH.

[belfhi]

I agree with @pelacables that this should be part of the main batch spawner release!

[pelacables]

I wrote a small patch , everything in singleuser.py. I ran some tests and seem to work, but I'll leave it running for some days. In any case, if you want to test it :

$ diff singleuser.py.rnd singleuser.py
4a5
> import socket
12a14,17
>
> low_port = int(os.environ.get("JUPYTERHUB_LOW_PORT","28000"))
> high_port = int(os.environ.get("JUPYTERHUB_HIGH_PORT","28100"))
>
14a20,30
> def next_free_port( port=low_port, max_port=high_port):
>     sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
>     while port <= max_port:
>         try:
>             sock.bind(('', port))
>             sock.close()
>             return port
>         except OSError:
>             port += 1
>     raise IOError('no free ports')
>
17c33
<     port = random_port()
---
>     port = next_free_port()

and you have to define the OS env var JUPYTERHUB_LOW_PORT / JUPYTERHUB_HIGH_PORT (otherwise it will use 28000 and 28100 as default) and also add those vars to c.Spawner.env_keep in /etc/jupyterhub/jupyterhub_config.py .

batchspawner 1.2.0

[belfhi]

is there a way to use this script in the jupyterhub_config.py config file? As far as I understand the c.Spawner.port is an Int traitlet config.

edit: I now undestand why this needs to be set in singleuser.py since the port selection needs to happen on the singleuser server and not the hub itself.
We should try to get this merged into the main batchspawner, though.