[blog] Quarter 2 update

[choldgraf]

Summary / idea for post

We just finished Quarter 2 of 2022! This is a short community update post to give others an idea of what we have been up to, major updates or accomplishments, and what we have learned.

Major updates to cover

We'll fill this in as ideas come in about things to cover

Tasks and updates

• Collect ideas for things to mention
• Post draft: {LINK}
• Blog location: {LINK}
• Publish to blog
• Send out tweets and LinkedIn
• Send to 2i2c listserv (optional)

[choldgraf]

Please provide ideas to include above

Can folks on the @2i2c-org/tech-team as well as @2i2c-org/partnerships-and-community-guidance provide any high-level ideas for topics that you'd like to cover?

This can be anything from technical improvements, to team and process improvements, to lessons we've learned, to highlights about impact that we've had. We'll leave this open for a day or two to collect ideas, and then generate a blog post out of what we've got. Anything in April, May, June of 2022.

cc @damianavila who might be particularly here, as the person who has been putting together useful summaries of our workflow over the past few weeks :-)

[colliand]

The experience with the Eddy Symposium could be part of a blog post. Some observations on that event:

• 2i2c responded quickly to the opportunity and deployed a hub in a little more than a week.
• 2i2c initially deployed the hub with a Pangeo-style software environment and then improved and switched over to an image designed by and for heliophysics research.
• 2i2c Co-Founder @fperez described living "la vida nube" in a talk at the start of the workshop. In response to requests from workshop participants, Fernando gave a demo on how to use the hub and an introduction to git.
• 2i2c Co-Founder @colliand gave a talk on governing the science commons. The idea that "intellectual generosity" should be the guiding virtue for open science resonated throughout the workshop.
• The hub and collaborative guidance led to productive afternoon hackathon sessions that continue today.

Another theme emerged over the week: 2i2c delivered a platform for collaborative computing that went beyond expectation, even among those familiar with the Jupyter ecoysystem. Some features that provoked oohs and aahs:

• Launch a linux desktop in the browser
• Authenticate to GitHub using tokens by executing a small notebook
• The shared drive as a method to deploy data and content across the entire workshop

The Eddy Symposium demonstrated that 2i2c has capacity that goes beyond the delivery of technical resources for data-intensive research. 2i2c's capacity to demonstrate the ways technology can be used for collaborative learning and research is incredibly valuable. The lessons learned from this experience will inform improvements to 2i2c's approach to catalyzing effective communities following the launch of their hubs.

[damianavila]

A list of the most remarkable (my subjective interpretation) things we have done from Apr to Jun (data collected from our backlog and sprint/cycle boards from iteration 8 to iteration 13):

Iteration 8

• Updating pricing and sole source guide docs#132
• Improve the hub health check in deployer infrastructure#586
• Document object storage and scratch bucket docs#135
• [LEAP Hub] Allow for profile list options based on GitHub team membership infrastructure#1146
• Protect cloudbank demo hub from crypto miners infrastructure#1216
• [EVENT] Openscapes NASA ECOSTRESS Workshop infrastructure#1173
• GPU option for LEAP and M2LInES Hubs infrastructure#1237
• [New Hub] M2LiNES Pangeo hub infrastructure#1168

Iteration 9

• Update daskhub's to use dask-gateway version 2022.4.0 infrastructure#1224
• [Incident] m2lines hub unavailable for a short period of time infrastructure#1248
• Request to install Tensorflow and Keras for Python and R utoronto-image#6
• Investigate using CILogon instead of Auth0 infrastructure#967
• Added Canada College to cloudbank infrastructure#1249

Iteration 10

• Move to using OAuthenticator from upstream infrastructure#1252
• Add gh-scoped-creds to images infrastructure#1220
• Support GPU on eksctl created AWS instances infrastructure#1314
• Add scratch bucket functionality for AWS infrastructure#1322

Iteration 11

• https://github.com/2i2c-org/leads/issues/56
• [New Hub] Jack Eddy Symposium infrastructure#1329
• Product and Community docs #427
• Shared responsibility model docs#144

Iteration 12

• https://github.com/2i2c-org/meta/issues/256
• Automate bumping of pangeo images for m2lines / leap hubs infrastructure#1338
• [New Hub] University of Washington - NASA SnowEx Hackweek 2022 infrastructure#1309
• Update service objectives page docs#143

Iteration 13

• [Monitoring] Enable GitHub OAuth2 authentication for Grafana infrastructure#1437
• Define support ticket urgency levels and practices for how to deal with them infrastructure#1154
  • Add incident commander role + more steps to support process #422
• Define some first-line and second-line support processes infrastructure#1068
• Create a BinderHub helm chart in the infrastructure repo infrastructure#1280
• Deployment of cert-manager and dask-gateway require kubectl apply -f <crd manifests> step when version bumped infrastructure#1381
• Metrics and Reporting for LEAP Hub infrastructure#1279
• Add Tuskegee University and Fresno City College hubs infrastructure#1398
• Added Glendale CC to pilot-hubs infrastructure#1494
• Added MiraCosta College to cluster infrastructure#1497

There are also some other issues such as #415 (and related 2i2c-org/team-roles-geekbot-sweep#43) which is essentially done but waiting to be deployed.
I guess we can talk about them after they are actually archived (next quarter). Another example would be the changes we have introduced from the PM perspective, it maybe makes sense to talk about them in the next quarter when we have a more complete picture.

Hopefully, this is helpful for you, @choldgraf!

[choldgraf]

Hey all - I dropped the ball on this one. I tried to quickly write down the content for a blog post, but found it difficult to pull out a couple high-level things from the list of merged PRs etc above. I'm going to close this issue because I don't think I have the time to write this blog post.

Here's a quick post-mortem on this issue in the hopes that it can help improve our team practices for this kind of thing in the future:

Synthesizing high-level impact from low-level actions is hard. I found it hard to define our high-level impact simply by parsing dozens of merged PRs and closed issues. It creates a lot of extra work to figure out the patterns there, and then to reverse-engineer our major accomplishments by looking at those patterns. Moreover, because my visibility is pretty high-level at 2i2c, I lack a lot of the context for each item that was completed so it's extra work to figure out the big picture.

There are two things I think we can do to more deliberately plan and communicate our actions:

1. Define our 3-4 most important goals ahead of time. I think we need to be more intentional about our goals for each quarter ahead of time. When we try to answer "what did we do this quarter?", the top 3-4 "big picture" items should already be there, and our goal should be to cross-reference our actual work against our intentions at the start. (maybe this happens more granularly than every quarter...but we need some kind of "planning cycle")
2. Ask domain areas to define their own impact. The people overseeing the work in various parts of 2i2c are the best ones to understand the impact that our actions were meant to have. In future iterations, we should have a (quarterly?) report in from engineering, community, product, and partnerships to define their own major impact items in one or two short paragraphs.

If others have suggestions, I would love to hear them. For now I'll close this and refer to the above points for future