Versioning policy for graphql-python tools

[Cito]

I think it's time to discuss a versioning and naming policy for the graphql-python tools so we can move forward.

We currently have three "generations" of the underlying core library:

• graphql-core (supports Python 2+3, old API, based on an old version of GraphQL.js)
• graphql-core-next (supports only Python >= 3.6, new API, latest features from GraphQL.js)
• graphql-core/modern branch (trys to be compatible with core-next, but support Py 2 + 3)

All of these use the package name "graphql" (like GraphQL.js) which makes things difficult, and currently have version numbers that are not (chrono)logical (core-next has a lower version number than core).

We also have a large set of other libraries that use graphql-core as their foundation: Graphene, graphql-server-core, flask-graphql, graphene-sqlalchemy etc. etc.

We want to move things forward, but not break this whole carefully assembled house of cards or create a proliferation of different forks and subvariants. So we need to talk on how to proceed regarding the versioning and naming strategy.

Some questions that come to my mind:

• Do we really want to support the graphql-core/modern branch? Yes, it looks like a reasonable strategy and @syrusakbary already put a lot of work into this, but I'm not sure if it's realistic to keep it - to me it seems too much of an effort just for supporting Python 2.7 (end of life 2020-01) and Python 3.5 (end of life 2020-09), and I see nobody who feels responsible and will maintain it. Syrus dropped out, and I want to focus on core-next. And it makes the whole story even more complicated like juggling with three balls. But I'd like to hear the opinion of others, and maybe somebody really wants to maintain it?
• When the existing tools are rebased on the new API (graphql-core-next), should they be released under a new name (like graphql-server-core-next) or should they keep the name and use a new major version number? Should new repositories be created or just separate branches in the existing repositories?
• Should we use different names instead of just "graphql" for the base package so that graphql-core and core-next can be installed in parallel?
• Or should we use the same name, maybe even release them under the same well-known name ("graphql-core") on PyPI and distinguish them by version number ranges (e.g. core = 2.x, core/modern = 3.x, core-next = 4.x+)?
• Or maybe even choose a completely different distribution name? "GraphQL.py" or "graphql" would be great, but they are already in use.

I'd like to get your thoughts and feedback here or on the Slack channel.

[jplock]

I like the idea of the graphql-core-next project being the one and only python GraphQL implementation, retire the other two, and have it replace graphql-core (published with a major-version bump to 3.x or something) in PyPI.

Then update the dependent projects, where possible, around that new major version, bumping their major versions as well.

[Nabellaleen]

• I think all the core projects should be "unified" in the same repository and "differentiated" throught their version number
• I think graphql-core is a better name than graphql-core-next for the long term. Because next is a relative term. So it has sense now, because there is the "old" graphql-core, but in the future, this "old" version will disappear, and graphql-core-next will be a very bad name, creating confusion.
• I think graphql-core is not a bad name, so we could keep it
• I think it's important to maintain the 2.7 compatibility on the current graphql-core, to do not "force" 2.7 projects to migrate their python because of graphql-core repository
• But I think we should not continue to develop "new features" for 2.7/3.5 and move forward

Base on that, I think we should migrate modern and next as 3.x and 4.x like @Cito suggested. And we should stop to develop new "features" for the legacy and modern versions, to just provide a "long term security support".
And all the other projects in the organisation should follow this principle, migrating in a major version to the 4.x version, and just keeping a "long term security support" for the previous (the current today ;)) version

[ProjectCheshire]

Similar to above, I prefer a semver policy. Nb We also need to add a toggle on the docs so people can refer to previous versions and the relevant docs.(eg how would this even be done spread across packages? Also, more integration (or even a reference to) the core next docs from graphene)

It makes more sense to install and require based on versions / releases vs switching entire packages. Install @4.x or above or more than 2 less than 4. Features on 4.x only, maintenence (and bug fix if the contributor provides a compatible pr for that release) for the rest

[mvanlonden]

When the existing tools are rebased on the new API (graphql-core-next), should they be released under a new name (like graphql-server-core-next) or should they keep the name and use a new major version number?

Agree with @Nabellaleen. graphql-core is a better name. I think it would make sense to move graphql-core-next into graphql-core as a branch and use semver when we are ready to move the version forward.

[Cito]

I suggest to get the ball ralling, we call the Py3-only version 3, and use this version numbers also for the other projects - i.e. Graphene 3 should be Py3-only and use graphql-core-next. If somebody still wants to create and maintain the Py2+3-compatible branch, it can still be released as 2.5 then. I think that is more intuitive than using versions 3 and 4, particularly if 3 never happens.

[changeling]

I completely support the idea of graphql-core-next becoming graphql-core version 3.

I suggest there be a very, very clear point of communication outside the Issue discussions, pointedly available for all related and dependent packages illustrating the plan, the roadmap and intended changes to core packages. I'd suggest that at this point there may be some value in using wiki, duplicated in both packages, presenting what's actually going on. However, if allocating time to ensuring the wiki is current proves unlikely, a ROADMAP that is maintained might be more applicable.

Or, rather, I think at this point there's great value in establishing a single point of canonical information the userspace can look to for current status information and evolutionary planning, particularly addressing what happens when the merge hits. i.e. graph-core-next dependencies should be updated to graphql-core:^3 or equivalent versioning.

Which brings up a question. Does graph-core-next go away at that point? Are Issues, if relevant, preserved in some way? Commit history? Does anyone care about those things? Etcetera.

On a personal note, I'm trying to follow the Issue level conversations and finding it frustrating. This may be limiting adoption by others. I was on the verge of dismissing this as a viable long-term dependency, particularly after a brief scan of the opening post for graphql-python/graphene#884, which reads like a common death-knell for open sourced projects.

Upon further digging, I saw that there was renewed support for ongoing development, and decided to keep learning. And I suspect I'm not alone. I was encouraged by this only because I was diligently watching the Issues. Not all 'searchers' will be digging that deeply.

This rant, and yeah... I'm kinda ranting. Mea culpa... was in part inspired by graphql-python/graphql-core#21

But yes. By all means, core version 3. And please give a single-point source for what the... well.. f or h, your choice of profanity level, is going on and about to happen. ::cha-grin::

I so, so want this to rock the world. Or at least my use-case world. Thank you for re-invigorating this project.

[Cito]

Which brings up a question. Does graph-core-next go away at that point? Are Issues, if relevant, preserved in some way? Commit history? Does anyone care about those things? Etcetera.

I definitely care. My idea is to keep core and core-next as separate repositories. They would be released under the same graphql-core distribution name on PyPI, differentiated by their version number ranges. I'm still waiting for more feedback from the community and from Syrus, who also needs to give push access to the PyPI project. That's why for now I have still published the current release as graphql-core-next 1.0.3. Things were a lot easier when everything was managed by one person, who did an amazing job and was so incredibly productive. But in the long run I think the project will be better off when the responsibility is spread to different people - even if decisions, coordination and realization of ideas takes more time and can be frustratingly slow since all of us have day jobs and families and life often gets into the way of all our good plans and ambitions. Proper funding or a BDFL would help, but currently we need to do without these. I'm confident that everything will work out.

[eMerzh]

Yeah sounds good to me, let start with a version 3! And avoid wasting too much time on 2 project and let the project stale during this time....

Again thanks a lot(!!) For moving this forward.

[changeling]

My idea is to keep core and core-next as separate repositories. They would be released under the same graphql-core distribution name on PyPI, differentiated by their version number ranges.

That approach gets my vote!

...one person, who did an amazing job and was so incredibly productive.

It's honestly kinda stunning how far it came as a one-person project. Major kudos and thanks, @syrusakbary.

[eMerzh]

@Cito doesn't want to push you here,
but do you have an idea if your plan is settled? and if yes do you have an estimated timeframe for the publish of the version 3. ?

[Cito]

@eMerzh Still waiting to get the PyPI maintainer right for graphql-core in order to do that - will address this problem at our next meeting end of next week.

[mvanlonden]

@Cito if we migrate graphql-core-next into graphql-core and release graphql-core-next as v3 in graphql-core, we can still release without access to PyPI. Our meeting next week should illuminate the path forward but just wanted to write this out as a possibility.

[Cito]

I think the problem should be solved fundamentally.

First, I'm adamant that the maintainers of graphql-core need maintenance rights on PyPI and have the freedom to deploy to PyPI in any way they see fit. Syrus could still stay the project owner so I see no problem here.

And I'm not sure if merging the repositories would be good because people usually look only at the README of the master branch. With different repositories we can have different github home pages with READMEs appropriate for core-2 and core-3. I believe core-modern has been ignored and neglected because it did not have its own repository, but was hidden in a branch, and I fear the same will happen to core or core-next depending on which we will make master. Different repos will keep both projects visible and give us also the freedom to configure different webhooks and integrations - core-3 could use newer tools that do not support Python 2. Also, I think separate issue trackers are better.

[mvanlonden]

Yes I agree—the problem should be solved fundamentally. I've reached out to @syrusakbary to see if he's available to grant the maintainers PyPI permissions.

If we do end up releasing graphql-core-next as graphql-core v3 then I think it would make sense to change the repo name to graphql-core and the current graphql-core repo to graphql-core-legacy or something similar. The repo with the most recent version of graphql-core on PyPI should bear the same name.

[changeling]

Just my two cents, but if the path forward is graphql-core v3, it seems like the goal, at least, if not the immediate path, should be for graphql-core master to be v3, with a reference in the README to the v2 structure/branch. This would allow for PyPi distribution to be appropriately versioned, correct, rather than bifurcated?

This would allow strong focus going forward on the v3 branch, as master, avoiding the core-modern languishing issue, while allowing the ecosystem a transition path forward via dependency/PyPi versioning while encouraging the other projects towards updating to the v3 graphql-core. Am I understanding the forward path correctly?

On a personal note, one of the biggest frustrations I had getting my head around the ecosystem was trying to figure out which bits were current, and struggling through issues and StackOverflow and the various blogs and tutorials and sample repositories referencing various repos and resources. I almost abandoned it all before getting my head around it all. Noobs, in particular, face a bit of a cloudy path at the start, and clearing the fog, as it were, IMHO, would greatly benefit the entire ecosystem. Adding yet another repo, I'd suggest, will only deepen the murk.

A v2 README within the v3 structure, or simply a reference with link to the v2 branch, with requisite mention of the tools compatibility, would address the branch awareness. This could also be addressed via the wiki. This approach has already been used once, a la the v1.0 to v2.0 transition:

See 2.0 Upgrade Guide and this section:

2.0 Upgrade Guide

Please read UPGRADE-v2.0.md to learn how to upgrade.

of Graphene README.md

Regarding issues, updating New Issue templates to identify v2 or v3 issues would be a potential soultion to that, well, issue. It seems to me that going forward, reducing the visibility of v2 ultimately is a win, as we slowly approach python 2 EOL.

This approach also serves to incentivize focus and efforts to bring tools, webhooks and integrations more rapidly into sync with v3, should there be such that don't have an equivalence in the v2 ecosystem, while maintaining the v2 structure for those still in a python 2 context.

In short, it seems an approach like this will bring more rapid development and support throughout the ecosystem and within the core project itself from the larger community, while simplifying the adoption path for new users.

[Cito]

This question still needs to be decided. We have two options:

1. Rename graphql-core to graphql-core-legacy and graphql-core-next to graphql-core.

2. Merge the graphql-core-next master into graphql-core, after creating a v2 branch, and delete the graphql-core-next repository.

If we go with 2, we should at least keep the commit history, because the commits in core-next contain references to the corresponding commits in graphql-js, which is valuable information. But we would lose the issues and pull requests in the core-next repository, which also contain interesting discussions and are referenced in the changelog and commit messages. We would also need to add labels to the issues to clarify which version they belong to. That's why I still prefer 1.

[jkimbo]

@Cito I agree, option 1 makes the most sense.

[KingDarBoja]

I was going to suggest renaming the graphql-core as some carbon nanostructure like graphite so it matches the same family as graphene.

But glad to read the whole topic and see you guys went with the semantic versioning path as this is a widely accepted standard to release new versions instead of keeping them on separate projects.

As someone pointed out, the docs regarding each version can be loaded from the corresponding branch without problem, so everything related to the library is contained on the same repo.

Cheers!