[New Hub] NeuroHackademy 2022

[damianavila]

Hub Description

A general description of the hub requested lives here: #1300 (comment)

Community Representative(s)

@arokem

Important dates

• Target start date: July 15th
• Required start date: July 22th
• Any important dates for usage:
  • Event begin: July 25th
  • Event end: August 5th

Hub Authentication Type

GitHub Authentication (e.g., @MyGitHubHandle)

Hub logo information

• URL to Hub Image: https://user-images.githubusercontent.com/118582/177910655-a2403aeb-9c08-44fd-94a8-543b5070f6a9.png
• URL for Image Link: https://neurohackademy.org/ (todo: confirm this is it)

Hub user image

• Repository for user image: https://github.com/neurohackademy/hub-image-2022
• User image registry: quay.io
• User image tag and name: quay.io/arokem/nh-jhub-2022

Extra features you'd like to enable

• Specific cloud provider or datacenter: GCP or AWS is preferable.
• Dedicated Kubernetes cluster
• Scalable Dask Cluster

Other relevant information

More detailed information about the requirements:

• General needs + some tiny specifics: [EVENT] NeuroHackademy 2022 #1300 (comment)
• More specifics: [EVENT] NeuroHackademy 2022 #1300 (comment)
• Template customization: [EVENT] NeuroHackademy 2022 #1300 (comment)
• Zoom links under auth: Authenticated and optionally private static websites served from a hub #1500

Hub URL

neurohackademy.2i2c.cloud

Hub Type

basehub

Tasks to deploy the hub

• Engineer who will deploy the hub is assigned
• Deploy information filled in above
• Initial Hub deployment PR: Neurohackademy hub #1507
• Administrators able to log on
• Community Representative satisfied with hub environment
• Hub now in steady-state

[damianavila]

Event issue cross ref: #1300

[damianavila]

@arokem, do you have any preference for the name of the hub? Any logo image link that you want to share with us?

[damianavila]

@GeorgianaElena is going to lead on this deployment with @yuvipanda as a helper resource (in fact he is already helping/building new functionality for this one with #1502)!

[arokem]

Hey @damianavila - thanks for kicking this off. And hello @GeorgianaElena : excited to work with you on this! And always glad to have @yuvipanda on board!

If neurohackademy.2i2c.cloud works as a name for the hub (is that what you meant?) that would be great.

Here is the logo we use for our Slack/etc:

[GeorgianaElena]

Hey @arokem! I just did an initial deployment of the hub at https://neurohackademy.2i2c.cloud (PR here #1507) 🚀

Initial setup of the hub

Right now the neurohackademy:

• uses CILogon for authentication and only allows and shows GitHub identity provider as an option
• hub admins are 2i2c folks + @arokem
• user image is currently the default 2i2c one (defined at https://github.com/2i2c-org/2i2c-hubs-image)

Missing pieces

Below are the required features of the hub that are still missing and require more info from either @2i2c-org/tech-team or @arokem

1. Waiting for 2i2c

• Zoom links under auth: Authenticated and optionally private static websites served from a hub #1500

2. Waiting for @arokem

• Confirm https://neurohackademy.org is the official URL we should use on the landing login page

• Confirm which funding info we should link from the login page
  

• Decide which user image to use

  @arokem, I noticed you mentioning that this is the image you'd like to use https://github.com/neurohackademy/nh2021-jupyterhub/blob/main/deployments/hub-neurohackademy-org/image/Dockerfile, is that right?

  In order to configure the hub to use a specific image, we need the image name and tag. Are you building and pushing that docker image to any image registry already?

  If not, as @choldgraf mentioned, we usually recommend communities to build off the repo2docker user image template that's using jupyterhub/repo2docker-action to build and push the image defined in a repo.
  I suggest using the info in the README of that template repo and the docs at https://docs.2i2c.org/en/latest/admin/howto/environment/hub-user-image-template-guide.html to create a new repo for this hub's user-image and move over the Dockerfile there. What do you think?

• Template customization

  There is now a branch called 2i2c-neurohackademy at https://github.com/2i2c-org/default-hub-homepage/tree/2i2c-neurohackademy. Any change pushed to that branch should show up in your hub in max 5min. Feel free to check it out 🚀 and modify the templates there as you wish.
  Someone at 2i2c must probably merge the changes though as I don't think you have the right permissions to push to that repo, so probably a PR would be the way to go?
  Btw, how often do you intend to change the info in the login page?

Anything else missing?

@arokem, if I missed anything else here, please ping me ✨

[arokem]

Hello!

Right now the neurohackademy:

• uses CILogon for authentication and only allows and shows GitHub identity provider as an option
• hub admins are 2i2c folks + @arokem

Looks great! How do we add multiple new users to the hub? Can we manage a user-list somewhere on github? This could even be a public page -- something like this

Confirm https://neurohackademy.org/ is the official URL we should use on the landing login page

Confirmed!

Sorry about this, but could we change the logo image to this instead?

Confirm which funding info we should link from the login page

The National Institutes of Health grant 2R25MH112480-06

I suggest using the info in the README of that template repo and the docs at https://docs.2i2c.org/en/latest/admin/howto/environment/hub-user-image-template-guide.html to create a new repo for this hub's user-image and move over the Dockerfile there. What do you think?

That sounds good to me! I think that we could inherit from jupyter-datascience, adding some neuroimaging-specific software on top of that. I don't remember exactly why we didn't use repo2docker in the last few years, and it looks like we're inheriting from a pangeo image. Maybe @consideRatio (who worked with us on our hub last couple of years) remembers why we didn't use repo2docker directly?

Someone at 2i2c must probably merge the changes though as I don't think you have the right permissions to push to that repo, so probably a PR would be the way to go?
Btw, how often do you intend to change the info in the login page?

I don't think we'll change it much. Probably set it once before the beginning of the event and then leave it as-is.

[yuvipanda]

@GeorgianaElena may I suggest using GitHub authentication + a GitHub org for authentication rather than using GitHub via CILogon? That should make maintaining a list of allowed users easier. I think we should default to this for workshop and research hubs, reserving CILogon for educational hubs.

[consideRatio]

Maybe @consideRatio (who worked with us on our hub last couple of years) remembers why we didn't use repo2docker directly?

I don't remember, but I'll confabulate that it could have been because we wishes for it to be mostly like pange/pangeo-notebook was, and then just add a few things to that. I suspect I've also been a bit biased towards keeping a plain Dockerfile as use of repo2docker added complexity to understand things if additional debugging was needed.

If, but only if, we needed both sudo and custom installation logic (not just list a package name, but run arbitrary commands to install something), that would have been problematic with repo2docker as well. I don't recall if we needed that though? I recall needing that in some situation, since I recollect the limitation.

[GeorgianaElena]

@GeorgianaElena may I suggest using GitHub authentication + a GitHub org for authentication rather than using GitHub via CILogon?

@yuvipanda, actually @arokem's answer from a previous issue was why this is not using GitHub org based authentication -> "We'd actually rather not do that and use individual GitHub user-names instead, if that's OK."

Looks great! How do we add multiple new users to the hub? Can we manage a user-list somewhere on github? This could even be a public page -- something like this

@arokem, you can manually add users from the admin panel (multiple users at once too). There are docs about this here https://docs.2i2c.org/en/latest/admin/howto/manage-users.html#manage-users-from-the-administrator-panel

But given @yuvipanda's suggestion above, would you consider using a GitHub organization or GitHub teams membership to authorize users, instead of manually adding them through the hub admin panel? Does that seem more suitable? Just want to make sure if I should change or not the current implementation.

[GeorgianaElena]

I've updated the hub at https://neurohackademy.2i2c.cloud with latest logo image and funding info.

So I believe that the next steps need your input @arokem:

• 1. Decide whether or not to use GitHub organization or GitHub teams membership to authorize users
• 2. Create a new repository for this hub's user-image using the template repo and the guide at https://docs.2i2c.org/en/latest/admin/howto/environment/hub-user-image-template-guide.html to
• 3. Open a PR against the 2i2c-neurohackademy branch with the desired hub template changes
• 4. Create the private repository with static web content that should be served from within the hub (behind auth) (ref https://docs.2i2c.org/en/latest/admin/howto/content.html#create-your-static-html-files)

Once the steps above are completed, then they will all be hooked to the hub and deployed 🚀

[damianavila]

4. Create the private repository with static web content that should be served from within the hub (behind auth) (ref https://docs.2i2c.org/en/latest/admin/howto/content.html#create-your-static-html-files)

Using #1502 instead of what is currently described in the docs. I am sure you already know that @GeorgianaElena, just clarifying for other readers 😉.

[arokem]

1. Decide whether or not to use GitHub organization or GitHub teams membership to authorize users

Let's not. I can add them through the hub admin panel.

2. Create a new repository for this hub's user-image using the template repo and the guide at https://docs.2i2c.org/en/latest/admin/howto/environment/hub-user-image-template-guide.html to

Done! https://github.com/neurohackademy/hub-image-2022

3. Open a PR against the 2i2c-neurohackademy branch with the desired hub template changes

OK. I will if I spot anything, but it looks good for now.

4. Create the private repository with static web content that should be served from within the hub (behind auth) (ref https://docs.2i2c.org/en/latest/admin/howto/content.html#create-your-static-html-files)

I have created a repo at https://github.com/neurohackademy/zoom2022. The file to render is https://github.com/neurohackademy/zoom2022/blob/index/index.html

[GeorgianaElena]

Yay! Thanks for all this work @arokem ✨

There is one more small step to be taken for in order to host the zoom links repo, which is to grant access to the 2i2c github app that will read the content in the private repo that you just setup by installing the https://github.com/apps/2i2c-neurohackademyhub-static-pull app in the neurohackademy org.

Also, two small things to double-check:

• the branch of https://github.com/neurohackademy/zoom2022 to use is main, right?
• the zoom repo will be hosted at https://neurohackademy.2i2c.cloud/zoom/. Is this ok, or do you want to use a different path than /zoom?

[arokem]

There is one more small step to be taken for in order to host the zoom links repo, which is to grant access to the 2i2c github app that will read the content in the private repo that you just setup by installing the https://github.com/apps/2i2c-neurohackademyhub-static-pull app in the neurohackademy org.

I believe I just did that!

• the branch of https://github.com/neurohackademy/zoom2022 to use is main, right?

It's on master

• the zoom repo will be hosted at https://neurohackademy.2i2c.cloud/zoom/. Is this ok, or do you want to use a different path than /zoom?

Sounds good!

[GeorgianaElena]

Thanks @arokem! ✨ I just deployed the latest changes and the hub is running at https://neurohackademy.2i2c.cloud 🚀 Also, the zoom links from the private repo are available at https://neurohackademy.2i2c.cloud/zoom/

Please check it out and let me know if you can login ok and have access to the admin panel and if everything works as expected

[arokem]

Thank you! Yes: I can login, though it's still going through CILLogin and I have access to the admin panel.

Questions:

• How often does the static webpage at https://neurohackademy.2i2c.cloud/zoom/ update, or how soon after a push to the master branch on the designated repo? I pushed a change about 10 minutes ago and not seeing it propagate yet.
• Could you please set this to the "lab" view per default? I am currently getting the "tree" view.
• Is it ready for me to add users? I added one more admin for now.
• Could we please add another repo with notebooks/curriculum material to be pulled with nbgitpuller into user's home directories? This will be an open repo.
• In past events, we designated a folder in each user's home directory for shared data storage. This was a directory into only a few users ("instructors") were able to write (to upload shared datasets) and all users ("students") could read from. IIUC, @consideRatio had set this up so that the storage was centralized for this location, so we didn't have to store multiple copies of the same data. Would it be possible to set that up here as well?

Thanks again!

[consideRatio]

In past events, we designated a folder in each user's home directory for shared data storage. This was a directory into only a few users ("instructors") were able to write (to upload shared datasets) and all users ("students") could read from. IIUC, @consideRatio had set this up so that the storage was centralized for this location, so we didn't have to store multiple copies of the same data. Would it be possible to set that up here as well?

@arokem I think you can get this very conveniently out of the box in a 2i2c provided hub. There will be a shared folder available to everyone mounted in a readonly state, and then a shared-readwrite folder - the same folder as shared - that only jupyterhub admins will be able to see that they can use to also write data to.

(I did some work to optimize performance of that data, making it copied to each node etc to ensure many people reading it at the same time wouldn't slow each other down. I think probably such optimization isn't going to be needed - and the out of the box NSF storage will work fine.)

---

Could we please add another repo with notebooks/curriculum material to be pulled with nbgitpuller into user's home directories? This will be an open repo.

For reference, this was done with a lifecycle hook and a mounted bash script, that can now be provided via the JupyterHub Helm chart's singleuser.extraFiles configuration.

Long term: One could consider if there should be a lifecycleHook to try run misc content no matter what in the user pods, then the users could provide such scripts via the image if they want and like that control an action that will happen on startup without involving 2i2c tech team.

---

Could you please set this to the "lab" view per default? I am currently getting the "tree" view.

@arokem I think you can visit <hub url>/services/configurator/ of the hub in question, and choose you want to start jupyterlab by default.

---

How often does the static webpage at https://neurohackademy.2i2c.cloud/zoom/ update, or how soon after a push to the master branch on the designated repo? I pushed a change about 10 minutes ago and not seeing it propagate yet.

Seems like 5 minutes. So if you are sure you have made a relevant change, maybe there is a bug to fix.

[GeorgianaElena]

Seems like 5 minutes. So if you are sure you have made a relevant change, maybe there is a bug to fix.

I think there was a small bug with the sync. Should be fixed now. @arokem, can you confirm that you can now see the changes?

[arokem]

Yes - confirming that the sync now works. Thanks for fixing that quickly!

[arokem]

And thanks @consideRatio for pointing out the shared and shared-readwrite folders. That should do the job.

[arokem]

The configurator doesn't seem to be working for us. I did as instructed above, and I even see this when I return to the configurator page:

But I still see the "tree" view when returning to my server, even after logging in and out and restarting the server.

[consideRatio]

@arokem / @yuvipanda could this be a configurator-bug? Does it need the image set as well?

[arokem]

How long should it take between an update to the main branch of the hub image repo (https://github.com/neurohackademy/hub-image-2022) and until I can start using the new image specifications? I pushed a change more than 30 minutes ago, and I can see that the change was also pushed to quay, but still not seeing the software dependencies added in the hub (i.e., getting import errors for things like import numpy as np). This despite stopping and restarting the server just now.

[consideRatio]

@arokem there is no longer automation regarding updating the hub to use the new image - it was coupled to a redeployment of the helm chart that isn't done as part of building the image any more. So, after the image is built you need to visit /services/configurator/ again and write in your new image name:tag.

If you declare that the "latest" tag should be used, you can avoid this - the downside is that you will possibly run into version mismatches between user server image version and dask workers image version when using a latest tag.

(actually, I don't know if a latest tag is pushed, so maybe that won't work at all)

[arokem]

Looks like I broke it :-/

I tried to enter arokem/nh-jhub-2022:latest in there, which I think made it look for docker.io/arokem/nh-jhub-2022:latest, which doesn't exist and now the thing is stuck in Error: ImagePullBackOff.

I did try to change that to quay.io/arokem/nh-jhub-2022:latest (is that the correct format for this? And yes, quay.io does have a latest tag: https://quay.io/repository/arokem/nh-jhub-2022?tab=tags), but it still seems to be stuck unable to recover from the previous error.

[arokem]

Sorry! Looks like I spoke too soon. After finally timing out, I was given the option to relaunch the server and that seems to have done the trick. It found the right image/tag and launched successfully. So, I am back on track setting things up in terms of software/environment.

[GeorgianaElena]

@arokem / @yuvipanda could this be a configurator-bug? Does it need the image set as well?

I don't think it's configurator's fault. I can see in the logs that the correct interface is being set on the spawner (plus, I've made /lab the default anyway).

After investigating this morning and not finding anything obviously wrong, I remembered that we recently bumped z2jh and that required us to bump the jupyterhub version in the user images too otherwise, @yuvipanda noticed that the default URL doesn't take effect.

I believe we should try to update jupyterhub from 1.5.0 in the user image to 2.3.1, like we did for the 2i2c default user image. Hopefully that should fix the default url issue.

[arokem]

I am now getting a 403 error for the zoom webpage:

Did something change?

[GeorgianaElena]

@arokem, the 403 should be fixed now. I also noticed that the redirect to /lab also works now, yay 🚀

Is there anything else I could help with on the hub env side of things? Please let me know if I missed anything ✨

[arokem]

No - looks like smooth sailing from here on out! Thank you! I'll plan to start adding participants early next week, so they can start taking a look around.

[GeorgianaElena]

Yay! Then I'll close this issue and open up another one about pre-warming the hub before the event. We will ping you there if we need any extra info 🚀

In the meantime, if you notice anything that doesn't work as expected or something is missing, please reach out to us through support (https://docs.2i2c.org/en/latest/support.html) to get more visibility ✨