-
Notifications
You must be signed in to change notification settings - Fork 555
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Bump to Python 3.9, Big typing updates #2963
Open
ashleysommer
wants to merge
16
commits into
main
Choose a base branch
from
py39_typing
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Update dependencies minimum package versions to post-3.9 versions. Upgrade latest pyparsing Update to latest MyPy Update to latest Black Re-issue new Poetry lockfile
…estictive, and fix the fallout
# Conflicts: # rdflib/namespace/__init__.py
…atically included in _subjectType, and we don't need a forward-definition for QuotedGraph in _subjectType alias.
…hat set in PyProject toml
ashleysommer
force-pushed
the
py39_typing
branch
from
November 1, 2024 06:09
b6a7794
to
10bac8d
Compare
Finally after a bunch of extra tweaks to satisfy tests, linters, and doc generation, this is ready to go! |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This is a very large PR.
This represents the first steps in the path to RDFLib 8.
There is a bunch to unpack in this one, this is the summary:
It is step number 7 that makes this PR so huge.
As an example, pre-3.9 typing annotations look like this:
and preferred type annotation style used in Python 3.9+ is like this:
The main differences being that importing basic type classes from the
typing
stdlib module is deprecated in Python 3.9+, for things likeList
,Set
,Dict
,Tuple
, Python 3.9+ supports using the native classes directly as type annotationslist
,set
,dict
,tuple
, etc. The other obvious change is the move away fro theUnion
helper fromtyping
module, because you can simply union types with the bar|
operator in Python 3.9+.Applying these annotation-pattern updates across the whole codebase caused hundreds of files to be modified in this PR.
Applying all these changes along with the bump to latest MyPy and asking MyPy type checker to operate in Python 3.9+ compatibly mode caused some fallout. There were over 250 failing type checks after applying the above changes, that each needed to be investigated and fixed.
The main culprit was RDFLib's existing excessive use of
typing: ignore[]
comments, that were either no longer needed (eg, the type was fixed, and the ignore was redundant which itself is a typing error), or causing their own typing problems. Mosttyping: ignore[]
comments were removed, and where there were still type errors the underlying cause was fixed instead of commented over.Another big contributor to the failures were some incompatibilities with RDFLib's fundamental base types, that were defined like this:
The problem is
Node
is the base classes for many different objects in RDFLib, some of which should not be used as subject or object in a Triple, and most of which cannot be used as the predicate in a triple.So this PR tightens up the typing of
_SubjectType
,_PredicateType
, and_ObjectType
to enforce correct usage of various python objects in triples, at the typing level. In various locations in the code baseinstead of simply usingNode
orIdentifier
annotations for objects in a graph (or serializer or parser), we now use the correct_SubjectType
,_PredicateType
,_ObjectType
where appropriate. This change caused a further 200+ type errors to be emitted from MyPy, all of which needed to be carefully checked and fixed. This exposed many instances where incorrect object types were being used, or edge-cases not considered and dealt with.The remaining portions of this PR are the fixes for those errors. The test suite was used continually throughout the process to ensure the actual behaviour of the executing RDFLib code did not change, and the test suite still passes.
All files were reformatted with latest Black, and Ruff linter with auto-fix used to ensure everything is compliant.