Add historical roles

[lilyminium]

This PR makes a start at adding the historical leads of identified roles as laid out here: https://docs.google.com/spreadsheets/d/1PIi6xJfM_Yz3SgTt-DuVyxOsNj3IfYyFuzJB8Ki_aOM/edit#gid=2049764634

It also makes a start at listing responsibilities and tasks, which are largely from the spreadsheet above.

I tried following the Astropy model (https://www.astropy.org/team.html), although comments on the spreadsheet indicated that order should indicate proportion of role, so people have not been sorted alphabetically.

One other change I made is that I took the liberty of removing the "MDAKits registry" repo from the "non-core maintenance" and made it a separate entry, as it seemed to me there were significant additional duties on top of normal code repos.

Reviews and opinions would be very welcome here -- this should be a group effort and reflect our consensus opinion!

Once this is in, we can make progress on designating new leads and subgroups for each role, as well as possibly going futher back in history and assigning credit further back.

[lilyminium]

Apologies to everyone for the formatting, Markdown tables aren't very easy to work with. If we have time we can consider switching to however Astropy formats their very nice tables -- skimming the website repo indicates that they just create straight html files: https://github.com/astropy/astropy.github.com/blob/main/getteam.py

[lilyminium]

Preview of the render

[RMeli]

Few comments and suggestions. Please feel free to discard them as you see fit.

[lilyminium]

Thanks for the thoughtful review @RMeli -- updated!

[fiona-naughton]

lgtm - thanks Lily!

(There might be things we'll change about how we present this going forward but I think this is good as least an initial version so we can at least get something out!)

[lilyminium]

Thanks @fiona-naughton! If current reviewers are happy with the format, we should probably open this up to all @MDAnalysis/coredevs for comment? However it'll probably be slow to get feedback over the UGM.

[IAlibay]

Thanks @fiona-naughton! If current reviewers are happy with the format, we should probably open this up to all @MDAnalysis/coredevs for comment? However it'll probably be slow to get feedback over the UGM.

Maybe something to add on to the coredevs meeting tomorrow or one of the dinners? It might be a lot easier to have most folks look at it synchronously for a couple minutes.

[lilyminium]

Maybe something to add on to the coredevs meeting tomorrow or one of the dinners? It might be a lot easier to have most folks look at it synchronously for a couple minutes.

Yeah I think it's already on the agenda to discuss current/future leads, so probably an easy place to drop a review of the past!

[jbarnoud]

I haven't followed the whole discussion here, but I find the "Historical" header misleading. As it is now, it shows the current state, which is what you would expect (and actually need) from this table.

I suggest renaming the header to "Members" or "People" or something that conveys that the table shows the people doing the tasks now. And I would add a section about people who used to do these things toward the end of the document. Since there is no history about this organisation yet, I would put some text saying that this organisation in "teams" dates from fall 2023 and that many people contributed to similar tasks in the past (maybe a link to emeritus?), then when people move to other task or plainly leave some, a list would be updated in that section.

This way:

• there is always an easy-to-parse snapshot of who does what now;
• there is no need to try and recover who used to do what before these tasks were formalised (which would be very difficult; take me as an example: I touched many of these things over the years, but have I been responsible for any of them in the way that the table implies?);
• the history is kept (which is important to give credit to people) within the context of the formalised tasks.

[lilyminium]

Sorry for getting back to you so late on this @jbarnoud, everything got a bit hectic around the UGM.

I haven't followed the whole discussion here, but I find the "Historical" header misleading. As it is now, it shows the current state, which is what you would expect (and actually need) from this table.

You're totally right here -- a lot of people looking at this table are wanting to find the current leads and we do want to put that at the forefront when it's added, apologies I didn't mention that at the start!

Just to explain the heading, while the "Historical" column does currently reflect the current leads, a common theme in our discussions so far has been that some names pop up much more than others; one driver behind this effort to clarify roles and responsibilities has been to highlight where this happens so that we can re-distribute the workload more, reducing burnout and increasing our bus factor. That's all to say that if I'm representing the group view accurately, that as this goes forward we want the "Historical" column to reflect the situation from ~2020-2023 but not from late 2023 onwards.

However, you're certainly correct that this table does still currently reflect the situation so if everyone is happy for me to do so (cc @MDAnalysis/coredevs), updating "Historical" to "Current leads" would be more accurate.

I suggest renaming the header to "Members" or "People" or something that conveys that the table shows the people doing the tasks now.

I agree -- I think we generally want to move towards a table where the column is called "Members", and the lead is just bolded. This current one is already missing a lot of effort from the community and was really meant to be an intermediate starting point. The names here are just the "lead" positions -- more the people who "manage" the process (e.g. replying to issues, reviewing PRs).

This is part of an organisational process where the idea is that we start creating "manager" positions who are responsible for taking care of each Role. We would eventually like to make subgroups with multiple people who are managed or mentored/trained by the lead -- so the people in the subgroup may undertake the actual tasks, getting developer/maintenance experience and training while doing so, but the lead is responsible for making sure that things actually get done. When we assign future roles, we do intend to add or replace the lead column with a "Members" column that contains all these names (with the lead bolded).

And I would add a section about people who used to do these things toward the end of the document. Since there is no history about this organisation yet, I would put some text saying that this organisation in "teams" dates from fall 2023 and that many people contributed to similar tasks in the past (maybe a link to emeritus?), then when people move to other task or plainly leave some, a list would be updated in that section.

there is no need to try and recover who used to do what before these tasks were formalised (which would be very difficult; take me as an example: I touched many of these things over the years, but have I been responsible for any of them in the way that the table implies?);

I really like this suggestion for the history that's hard to recover! For more recent history (e.g. the current "Historical" column) there's been concern that people currently doing much of the work may have their contributions elided by this sort of representation. My personal opinion is also that we should err on the side of inclusivity and give as much credit where due as possible. Would turning the current column into a "Historical members" column instead, with past leads name bolded, be a solution for both of these concerns?

[IAlibay]

updating "Historical" to "Current leads" would be more accurate.

I don't believe so, the current status does not reflect what this table shows since we've included folks that used to drive things in the past.

See the issue and PR management for example.

edit: my point being that those are flipping around a lot - I'm not willing to engage with the discussion of "who should be here" though (because I don't know and I don't really want to be in a position to decide who should and shouldn't be included).

[jbarnoud]

my point being that those are flipping around a lot - I'm not willing to engage with the discussion of "who should be here" though (because I don't know and I don't really want to be in a position to decide who should and shouldn't be included).

This is difficult indeed. Keeping that page roughly up to date will, in itself, be quite some work.

I do still recommend in favour of a quick comment about when that table was introduced, not to erase the work of former contributors. In the meantime, I'll add myself to the table.

[IAlibay]

my point being that those are flipping around a lot - I'm not willing to engage with the discussion of "who should be here" though (because I don't know and I don't really want to be in a position to decide who should and shouldn't be included).

This is difficult indeed. Keeping that page roughly up to date will, in itself, be quite some work.

I do still recommend in favour of a quick comment about when that table was introduced, not to erase the work of former contributors. In the meantime, I'll add myself to the table.

Sorry I should have maybe explained myself better here @jbarnoud - my point wasn't that I was against making this "a snapshot of today" (indeed I agree with you that this is better). My point was more that just purely changing the title might not work because some of these categories were written with "the last 2-3 years in mind". For those I'd ask that folks try to work out if they still make sense.

[IAlibay]

(not to single anyone out, but if Oliver has been leading issue reviewing lately, then I shouldn't be on that list, etc..)

[IAlibay]

One option here is to define "current" as ~ 2021-2023 (so roughly ever since 2.0). Then I think that works? - sorry I know realise that's probably what you meant here @lilyminium, in which case I agree with you.

P.S. I'm not meant to be doing MDA coredev things atm, so take my opinion (or lack of ability to form an opinion) as that of a random passerby.

[orbeckst]

Sorry, busy at the moment. Have set aside time tomorrow (Thu) to look at it.

[jennaswa]

Following up on Friday's coredev meeting, getting this table up is of utmost priority for the project. From my understanding, we had reached consensus that the next steps were as follows:

1. Add a column for the Current Team that represents who has led roles for the last 2-3 years
2. Move relevant names to 'Historical' column to distinguish between current leads (i.e., since 2020) and those who were driving things in the past
3. Add people to the 'Historical' column such that it dates back to a particular milestone (e.g., 2016 paper). We should also add text to the paragraph under 'Roles' to explain how we have defined 'Historical'.

If we are in agreement with this plan, @lilyminium or @fiona-naughton, would you be able to take the lead in reformatting the table throughout such that it is easier for people to add names to the 'Current' or 'Historical' columns as necessary? We will then need people with historical knowledge of the project to help make sure we are giving credit where it is due in the 'Historical' column.

In order to keep things moving along, does it seem reasonable to everyone to have the above steps completed by EOD Sunday, Nov 5? Please comment if you will need more time to review.

[lilyminium]

Thanks @jennaswa for the suggestions and plan! @fiona-naughton and I met today and discussed -- given that people have signed up for new roles on the Tasks and Roles spreadsheet and the annual dev roll call has just gone out, what would you think of the following proposal:

1. Everyone in this current table (i.e. past 3 years) goes into "Historical", leads bolded
2. Everyone who has signed up for groups going forward goes into "Current", leads bolded
3. We invite everyone to back-fill "Historical" as far back as possible

Given that step 3 requires people to participate, Sunday EOD is probably too tight. Is there an external deadline we're working towards? If not, could we use the same deadline as the core dev call? (Nov 13).

I'll see if I can put together a script to make editing the table easier or if using HTML would be easier. Unfortunately I looked into this a bit when first opening the PR and Markdown tables are very simple and difficult to work with.

[jennaswa]

Thanks @fiona-naughton and @lilyminium for the quick follow-up!

1. Everyone in this current table (i.e. past 3 years) goes into "Historical", leads bolded

This seems to make sense. I think we would just need to be careful that the table is labeled correctly and has adequate text to indicate that we are bolding "leads from recent history (i.e., 2020-now)" to make sure not to mask the efforts of those in the larger project history. This is already partially accomplished from @jbarnoud's previous comment about being clear about when this table structure was started.

I think this leads to another question of how people can access past tables as references for jobs, etc. I apologize if I missed this somewhere, but have there already been discussions about how we archive past tables? For instance, would we be able to save previous tables somewhere (e.g., this initial table, then, if we decide on an annual update, 2024-2025 table, 2025-2026 table, and so on) in the pages on our website?

2. Everyone who has signed up for groups going forward goes into "Current", leads bolded

Sounds good.

3. We invite everyone to back-fill "Historical" as far back as possible

Given that step 3 requires people to participate, Sunday EOD is probably too tight. Is there an external deadline we're working towards? If not, could we use the same deadline as the core dev call? (Nov 13).

The next core dev call is not until the end of Nov (Nov 28), and I would argue we should get this published far sooner than that. Per @jbarnoud's comment from before that it would be hard to slot past contributors into these roles as they are defined now, I wonder if it makes sense to limit the 'Historical' column to the 2020-2023 period (or 2016 paper) that inspired this initiative and organization of roles, and (restating @jbarnoud's suggestion here) have a general comment at the end linking to emeriti core devs as not to diminish the contributions of people earlier in the project.

This could put us in a position to publish the table very soon (e.g., before end of next week), and we can always add people to the historical column as we go, or ask people to add themselves to the table based on which tasks they worked on most closely.

I think in summary, I would vote for structuring the table in a way that it acknowledges everyone and the tremendous amount of work they have put into running the project and has enough flexibility that it can be a working document that can be made more thorough over time. I think the most important thing is getting a working document up ASAP (I personally feel this should be possible before the end of next week) that we can update as we go. I worry a bit about the pursuit of perfection keeping us delayed in this process.

[lilyminium]

The next core dev call is not until the end of Nov (Nov 28)

Sorry, I should have been clearer, I meant the core dev roll-call that ends Nov 13. However if we are not going further back than 2020, I agree we can do it before end of next week. I don't personally have the institutional knowledge to assign the 2016 paper authors to roles so that would still require outside input.

I think this leads to another question of how people can access past tables as references for jobs, etc. I apologize if I missed this somewhere, but have there already been discussions about how we archive past tables? For instance, would we be able to save previous tables somewhere (e.g., this initial table, then, if we decide on an annual update, 2024-2025 table, 2025-2026 table, and so on) in the pages on our website?

The idea of this table being a formal document with years that could be used as references for jobs is new to me. I was under the impression that we would not try to place people by year or period, but rather simply list people who have held roles in the nebulous past under "Historical". This is already more than the much-referenced Astropy page does (they simply list major contributors, rather than breaking them down by role or time). Is this strongly desired / was it discussed at the last core dev meeting? If so, it'll affect the solution I pick for the table generation (long-form vs wide). Under the impression that we might be building up a lot of historical members, I'm currently building a long-form solution where editing yaml looks like the below snippet, and renders like the attached screenshot table.

team:
  roles:
    - name: Core library maintenance
      description: |
        The core library maintenance team is responsible for the
        maintenance of the MDAnalysis library. This includes
        reviewing and merging pull requests, maintaining the
        documentation, and releasing new versions of the library.
      tasks:
        - Initial triage and tagging of issues
        - Managing timely responses and resolving issues
        - Reviewing, shepherding, and merging pull requests
        - Standards compliange (e.g. managing metadata such as the author list)
        - Tracking new dependencies
        - Emergency fixes
        - Other emergency maintenance
      subroles:
        - name: Issue management
          current_leads:
            - Rocco Meli
          current_members:
            - All core devs
          historical_leads:
            - Oliver Beckstein
            - Irfan Alibay
            - Jonathan Barnoud
          historical_members: []

        - name: Pull request management
          current_leads:
            - Rocco Meli
            - Oliver Beckstein
          current_members:
            - All core devs
          historical_leads:
            - Oliver Beckstein
            - Richard Gowers
            - Irfan Alibay
            - Hugo MacDermott-Opeskin
            - Jonathan Barnoud
          historical_members: []
        
        - name: General maintenance
          current_leads: 
            - Irfan Alibay
          current_members:
            - All core devs
          historical_leads:
            - Irfan Alibay
            - Jonathan Barnoud
          historical_members: []

    - name: Continuous integration
      tasks:
        - Building and developing new CI infrastructure
        - Monitoring status
        - Maintenance and fixes
      subroles:
        - current_leads:
            - Irfan Alibay
            - Richard Gowers
          current_members:
            - Fiona Naughton
          historical_leads:
            - Irfan Alibay
          historical_members: []

[lilyminium]

If we wanted to archive tables by year it would probably be more sustainable to figure out how to do so with a CSV.

[fiona-naughton]

My two cents: it seems the trick is to balance the pre 2023 stuff we do have clear lead/dates on with the fact we don't have dates for a lot of the rest, without either "ignoring" the things we do know or making it seem like the stuff we don't know wasn't important.

I'd suggest we allow for adding years in the former case (e.g. remove all he bolding but have "Name (lead, 20xx-20xx)"; we have the cases Irfan has mentioned, and we can allow further self-reporting, but unfortunately I'd say it's out of scope to figure out any more), but make sure our disclaimer indicates this is exactly what's happening - e.g. expanding slightly on the current ti say something like "Past lead positions+dates have been listed where known, but not all such past contributions have been captured"

coredevs Can we please agree to halt or push back this arbitrary "end of week" deadline? There's a ton of stuff going on here and rushing is just going to end up hurting someone's feelings.

I'm wary that pushing back the deadline is a slippery slope to pushing this back indefinitely 😅 but yes, it does seem we have a few things we stil need to iron out. I would suggest maybe we pick say the rollcall (the 13th? perhaps too soon if we do want to make sure more discussion is had),.or the next coredev meeting (28th)

[jennaswa]

@MDAnalysis/coredevs Can we please agree to halt or push back this arbitrary "end of week" deadline? There's a ton of stuff going on here and rushing is just going to end up hurting someone's feelings.

My take here - nothing will change if this table gets published this week or next, it's already been a 6 month long process and it's not like a week is suddenly going to fix all our work imbalance problems.

Yes, I agree that many important points have come up this week that warrant holding off on publishing right away. I apologize if the "deadline" has created undue stress related to this process. I had never intended it to be a deadline, but rather it was meant to reignite this conversation and reflected my personal interpretation that (I thought) we were almost ready to go with at least a basic form of the table since there was no more engagement on this PR for roughly a month. I agree with Fiona that some sort of deadline/goal date is helpful though to ensure we continue to make progress and don't lose this momentum. Given the nature of the discussion points that have been raised, I think having everyone weigh in before the Nov 28 coredev meeting and having a discussion then to hash out final details seems sensible.

[IAlibay]

I suspect this will end up being up for further discussion - but here are the years I was active in all these roles. I think that better reflects things.

[orbeckst]

Apologies in advance if I missed important points in the discussion (here and on Discord). I know that I am making a kind of very cheap apology, given the incredible amount that so many of you have already put it into this effort (compared to me just having to read) — especially @lilyminium in developing and re-developing the framework, @fiona-naughton with thoughtful comments such as #324 (comment) and @IAlibay and @jennaswa in moving the discussion along as well as @RMeli and @jbarnoud with very constructive comments.

Thank you to @lilyminium for the summary #324 (comment).

My opinions on various points:

1. Give us more time if needed? Yes — unless everybody suddenly LGTMs the PR, we should give ourselves until at least the core dev meeting on Nov 28.
2. Historical column

• Make it as inclusive as possible; I think it's difficult in many cases to ascribe strong hierarchical responsibilities — that's just not how MDAnalysis as a volunteer open source project was running for most of its existence. So I would not distinguish leader/member or anything like that. If you were doing some of what the role says then I think you should be listed — in the spirit of a smallish open source project where anybody willing to help is doing what needs to be done.
• Adding year numbers is fine but I wouldn't want to insist on it, as it is again somewhat vague in the dim and distant past. I'd leave it to the individual to add them, especially if this would help in any form for external recognition.
• How to do historical pre-2020 contributions (related to Add historical contributors from 2016 onwards to table. #336 ): I strongly prefer the simplest solution: just keep adding people to the historical column, potentially add year numbers like (''first year they appear in AUTHORS''-''last commit year'') or just leaving out year numbers and just ordering in descending order so that past is last. A single table/set of data files is much easier to maintain than multiple copies. (If/how to tag and sort people seems to be a detail for Add historical contributors from 2016 onwards to table. #336 , though, I am just trusting that if we start with an imperfect solution that we eventually don't like for any particular reason then we are all willing to talk it out and reconsider and change if necessary.)

3. Do we need a "lead" for each role? Originally I said yes, but multiple comments (including @micaela-matta and @jennaswa on discord) convinced me that we should leave gaps if there's no one at the moment to take the role on. It will show us more clearly where gaps are and what we are actually able to do and what we need to do.
4. Who can be listed? Most of the people were CoreDevs when they did the job but not all of them, also, Ian is listed and he isn't a CoreDev. I don't remember reading or hearing a discussion but to state what should be clear: Names are not limited to CoreDevs — if you do a job for the project then you are being acknowledged. That's what we should be striving for. (And then there's the wider discussion about CoreDevs vs CoreContributors, maintainers of associated projects/MDAKits, etc — all of which points to being as open as possible here.)

I shouldn't be blocking anything.

[richardjgowers]

I'm not really comfortable with the current "historical" category. To me it feels like if I signed off on this it would be airbrushing out a lot of people that taught me a lot as I was learning on this project. There's an issue open about this, but I wouldn't be happy leaving this as-is, I'll try and find some time to correct this historical view before the end of the month.

Some of the sections seem over detailed, I'm not sure of the value of differentiating Issues, PRs and general maintenance.

I don't understand the distinction between Lead and Member

[richardjgowers]

Seb, and I think Manel too

[richardjgowers]

we've had plenty more people than this engaged here previously

[IAlibay]

@orbeckst - following on from yesterday's discussion, how do we feel about folks that aren't coredevs doing this?

[orbeckst]

Apologies, this is longer than anticipated — and sorry, you'll have to follow my typed out slow thought process:

Within our current definition of "core dev" as "project leadership" I think that's a bit problematic when viewing it from the perspective of the other side: If I were an outsider, looking to engage with the MDAnalysis Project, I'd eventually like to talk to someone in a position of obvious authority. The only way we currently obviously publicize "authority" is https://www.mdanalysis.org/about/#mdanalysis-core-developers and if I don't see the person I am negotiating with listed there then I might think that ultimately I will need to talk to someone else "who can make decisions".

From the internal side this seems like less of an issue to me. Within the current structure it would mean that the person leading this effort would not be able to vote on anything but could influence the process through discussion. If I were in this position I would probably feel not super-happy that I don't get a say in my own effort but that would depend on the person. We'd just need to be very clear up-front with them.

---

To come to something concrete:

I am considering

1. Decisions should be consistent, and in the absence of clear rules, precedent should generalize without breaking everything else.
2. Would we as a project be more successful in "external liaisons" if a CoreDev was doing it than if a non-CoreDev?

(2) is easy: Of course, the alternative may actually be "no-one does it or someone who cares does it" and then I am pretty sure the answer is that we'll do much better with someone who owns the role than someone whose name was put in to avoid a blank or who is stretched too thin. So there I think that someone who wants to do this and who has a track record of being able to work in such a role and work with the project is great. Ideally, we want both (for reasons above) but if we can't get both, then someone who wants to do it.

(1) It should be possible for "leadership" to delegate tasks. At the moment we are already doing this very successfully with our Project Manager. So I don't think that there's a problem per-se with someone leading a "role" who is not on the CoreDev list. However, the authority to do so must come from somewhere. Therefore, in these cases I'd say that "leadership" (CoreDevs) would need to vote to "appoint" and provide guidance on what the scope of authority is. This might feel mightily awkward.

In summary, I think we should be open to people who are not in the current Project Leadership (=CoreDev) to lead some of these roles and that ultimately they do this with delegated power from Leadership.

---

At the end of the day, it points more to the issue of rethinking our narrow governance structure.

[IAlibay]

en lieu of extra folks, there's a couple more packages I will continue to indirectly be involved with (mostly since they are upstream from the core library so it all falls under general "maintenance")

[orbeckst]

I don't understand the distinction between Lead and Member

My current understanding: The "Lead" part is there to acknowledge that there's at least one person who moves things along and takes on managerial roles if necessary, in addition to doing work. The person who is responsible for it, someone who "owns" the task. Members of the group do work, can also help with managerial tasks, and take part in discussions and decision making related to the group's purview.

[lilyminium]

As discussed at the last core-devs meeting, I've removed the historical column while we work out how best to acknowledge historical contributions, and also removed anything distinguishing leads. All teams are published in accordance to the "Current" tab in the Tasks and Roles spreadsheet. I think everyone by now has reviewed the "current team" column -- unless there are any objections, I'll look to merging this on the weekend?

(cc @richardjgowers @orbeckst @RMeli @jennaswa @IAlibay @fiona-naughton @micaela-matta @hmacdope)

The table current looks like the below:

[lilyminium]

Ok, I'm about to hit the big green button. Thank you everyone involved for pushing this through -- this was a huge group effort! If there's anything you'd like to see changed please raise an issue and we'll resolve it ASAP.

[orbeckst]

Thank you @lilyminium for your tireless and excellent work here – https://www.mdanalysis.org/pages/team/ looks very good!