Conventions for specifying versions in setup.cfg, conda-forge etc.

[mmcky]

Opening this as a general discussion for the EBP community given this feedback from conda-forge team.

conda-forge/myst-nb-feedstock#4

It appears the conda-forge recommendation is to specify version restrictions using >, <, and = operators.

myst-nb>=0.13.2,<0.14

We use ~ for specifying a single release within a major release family.

myst-nb~=0.13.2

---

It is hard to argue we shouldn't use ~= given it is in PEP 440 but I kind of sympathise with the consistency that > and < delivers when reviewing the requirements. It also requires no learning. I admit I was confused what ~= meant when I first saw it.

I personally don't feel strongly about this issue -- but perhaps if we use ~= we should add a comment at the top of every setup.cfg file that defines it?

[chrisjsewell]

Happy to improve documentation (just adding to COONTRIBUTING.md would be easiest) but for me, as mentioned, it's an official Python PEP convention, and so there is no reason to stop using it.

[choldgraf]

Sorry for the slow response - was off the last couple of days.

I don't have strong opinions about the convention itself, but at this point we've gotten enough pushback from potential contributors that I feel we are unnecessarily choosing to die on this hill. Since these two practices have no functional difference, it doesn't seem like something worth spending so much time debating.

In this case, I think we are feeling tension with others in the conda-forge community because we are choosing to draw a line in the sand about something that feels superficial. This makes us come across as inflexible, regardless of whether the policy is the "right one" or not.

If we were a static team then I'd say we just pick a convention and go with it, but since we wish to invite contributions from others in the community, I think it is important to follow practices that make it easier for others to contribute. In this case, I think we should distinguish between:

• Conventions that we require, and will block PRs if they don't follow. I think this should be done very sparingly and only for conventions that we know have a significant impact on maintainability, stability, etc. Things like "new features must have test coverage, documentation, and tests must pass."
• Conventions that we recommend and try to follow, but allow for wiggle room. I think that this is the majority of developer conventions in general.

I think that the burden of proof should be showing why a practice must be required across all of our repositories, rather than assuming that all of our conventions are required. We will pay a much higher social penalty by blocking things, and this will make it harder for us to build a healthy community dynamic.

So in sum, I suggest that we:

• Define a short list of the practices that we require across all repositories. Document these clearly and commit to following them, but choose these practices very sparingly.
• Treat everything else as convention, but not rule. Document conventions and try to show why it's useful to follow them, but do not force people to follow them.

That's my 2 cents - am happy to discuss this and iterate further. I think the broader topic is important to think about as the community grows.

[choldgraf]

I appreciate your concerns and I agree with some of them. I disagree with others. I'll just leave a few points below, and I will follow up in a separate thread with a proposal for what I think our project and community strategy should be as we move forward.

For transparency's sake, here are my goals:

• Create a community that is positive, energetic, inclusive, and that encourages creativity and open expression of ideas.
• With this community, build open source technology and standards that facilitate communicating with computational narratives, and that are high-quality and delightful to use.
• Strategically position the community and the ecosystem so that it has the best chance of competing against other products or services that are not aligned with our mission.
• Ensure that this community lives up to its values, and does not sacrifice them for the sake of productivity or expedience.

Regarding some of your points - I think that leadership is not ownership, and that we should build a culture of stewardship rather than laying claim to codebases. We should aspire to have a broad contributor base even if we anticipate that there will always be a few that contribute more code than others. I'd like us to adopt best-practices that other open source communities and organizations have pioneered, and to commit to professionalizing the way that we coordinate and communicate. I think that the best way to build impactful tools is to create a culture of positivity, mutual respect, and servant leadership, and to create a safe place for honest communication and open sharing of ideas. We should aspire to building this culture because it will let us live up to the project's values, build the most impactful and long-lasting technology, and create the most enjoyable community to participate in for many people.

I'll try to distill some of these ideas into some steps forward that I think we should consider.

[chrisjsewell]

For transparency's sake, here are my goals...

Yes I don't really disagree with any of these (some maybe feel a little contradictory), but mainly there is really nothing to disagree with, because they are very vague and IMO (to be a bit harsh), without actual documentation of how they manifest in reality just come across as hollow corporate buzzwords.

Now I know you personally have actually put thought in to this, so let's now get that on paper.
We are all scientists here, and I see no reason why we cannot try to apply the scientific method to things like community strategy and design (basically facts over feelings).
I would like to see the proposal loosely in the form of a scientific article, with actual references; to examples of where these strategies have been successful in other projects, and equally where projects have suffered because they have failed to implement such policies.
I'm honestly really interested in this and genuinely want you to prove to me that these goals are built on solid foundations.

I think that leadership is not ownership

But equally leadership is not actually doing the work.
Again, I feel this is a very "managerial" centric point of view; that you simply steward the codebase, and everyone will just be happy to do the work for you.

I have very clear agreement with taneli, that in bringing mdformat and mdurl under the EPB umbrella, myself or others in EBP would not start to dictate to him how he maintains he's own codebase. With this we have gained a long-term contributor, who has already begun to get involved in other aspects of EBP. It would be a large betrayal of trust if this began to be the case.
Equally, I would like to see about getting https://github.com/pradyunsg/sphinx-basic-ng and https://github.com/mcmtroffaes/sphinxcontrib-bibtex under the EBP umbrella, to use as core components of jupyter-book. I fear that having a small print that essentially states that you lose all rights to your own work once you transfer to EBP will put people off from considering this.

Then, coming back to this notion that standards are stopping people from contributing.
I'm still yet to have been shown any concrete proof of this, or anything less vague pointing to exactly what standards people are having issue with.
I think the answer here should be more focussed on improving documentation around the stack and practices, rather than just arbitrarily removing practices

What we should be doing is thinking about the routes to contribute and how we are advertising them, ranging from:

• Contributing to the gallery
• Upvoting an existing feature request
• Opening a bug report
• Opening a feature request
• Submitting a PR for a documentation fix
• Submitting a PR for a code change

In discussion with @mmcky, he rightly pointed out that some may be hesitant to put time into a coding a PR, if they feel it risks being rejected.

For this, I would suggest we implement en enhancement proposal "scheme", which I have started in jupyter-book/myst-enhancement-proposals#1. Perhaps your proposal should also be opened as an EEP

Coupled to this, @mmcky also mentioned about mentorship, which I think is a good idea; we could think about having a few slots open for offering mentorship for those that submit any particularly promising/complex EEPs. This could maybe also be linked to existing schemes like: https://summerofcode.withgoogle.com/

[choldgraf]

Sorry if I wasn't clearer above: I'm not planning to continue on a protracted debate in this thread. I'll follow up with some more specific, though high level, things that the team can discuss and iterate on in a separate dedicated issue.

[chrisjsewell]

That’s fine, I can copy over these discussion points when it’s opened

[chrisjsewell]

again I can copy over these to another issue when it is opened. I have started doing some research into the area, because I am genuinely interested, so just going to paste them here to iterate on:

Search for "examples of failed open source projects":

Scientific Papers:

• "Why Modern Open Source Projects Fail" (2017, paper) https://arxiv.org/pdf/1707.02327.pdf
  • a survey with the maintainers of 104 popular GitHub systems that have been deprecated.
• "Open Source Project Collapse – Sources and Patterns of Failure" (2017, paper) https://core.ac.uk/download/pdf/301371469.pdf
  • This study explores open source project failure by drawing on ten in-depth open source software case studies
• "On the abandonment and survival of open source projects: An empirical investigation" (2019, paper) https://arxiv.org/pdf/1906.08058.pdf
• "The Evolving Nature of Developers’ Contributions in Open Source Projects" (2020) https://dl.acm.org/doi/pdf/10.1145/3425269.3425284
• "Turnover in Open-Source Projects: The Case of Core Developers" (2020) https://dl.acm.org/doi/pdf/10.1145/3422392.3422433
• "An Empirical Study on the Structural Complexity Introduced by Core and Peripheral Developers in Free Software Projects" (2010) https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=5631694
  • We have found that core developers introduce less structural complexity than peripheral developers in general, and that in the case of complexity-reducing activities, core developers remove more structural complexity than peripheral developers. These results demonstrate the importance of having a stable and healthy core team to the sustainability of free software projects.
• "A Study of the Relationships between Source Code Metrics and Attractiveness in Free Software Projects" (2010) https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=5631691
• "Why Do Newcomers Abandon Open Source Software Projects?" (2013) https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=6614728
• "Let Me In: Guidelines for the Successful Onboarding of Newcomers to Open Source Projects" (2018) https://www.researchgate.net/publication/322414294_Let_Me_In_Guidelines_for_the_Successful_Onboarding_of_Newcomers_to_Open_Source_Projects

Other:

• "Why Opensource projects fail?" (2019, blog) https://filfreire.com/posts/os_projects_fail
• "Open source has a funding problem" (2021, article) https://stackoverflow.blog/2021/01/07/open-source-has-a-funding-problem/
• "What Happens When Developers Leave Their Open Source Projects?" (2020, blog) https://thenewstack.io/what-happens-when-developers-leave-their-open-source-projects/

[mmcky]

I think it would be great to move some of this thread to the discussion around sustainability and next steps for EBP. Thanks everyone for those thoughts.

---

Heading back to the original issue I have put together a preview of what it would look like if we used > and <:

mmcky/myst-nb-feedstock@63e28d7

The pro and con I see are:

Pro:

1. easy to understand (without reading the documentation I have currently added at the top) when using > and <
2. harmonises with other packages that we already use > and < for multi-version pins.

Con:

1. using ~= highlights packages with tighter version pinning requirements as it is restricted to a single major release family.