Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: allow non-Mapping Iterable parameters #3632

Closed

Conversation

rafalkrupinski
Copy link
Contributor

@rafalkrupinski rafalkrupinski commented Jul 16, 2024

Description

Allow Iterables other than str or Mappings to be used as Sequence parameters

Closes

Closes #3631

@rafalkrupinski rafalkrupinski requested review from a team as code owners July 16, 2024 13:21
@github-actions github-actions bot added area/kwargs area/private-api This PR involves changes to the privatized API size: small type/bug pr/external Triage Required 🏥 This requires triage labels Jul 16, 2024
Copy link

codecov bot commented Jul 16, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 98.29%. Comparing base (6d49fc2) to head (58da671).
Report is 32 commits behind head on main.

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #3632   +/-   ##
=======================================
  Coverage   98.29%   98.29%           
=======================================
  Files         330      330           
  Lines       15122    15135   +13     
  Branches     2401     2404    +3     
=======================================
+ Hits        14864    14877   +13     
  Misses        117      117           
  Partials      141      141           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Copy link

Documentation preview will be available shortly at https://litestar-org.github.io/litestar-docs-preview/3632

@rafalkrupinski
Copy link
Contributor Author

rafalkrupinski commented Jul 16, 2024

While we're at it, is_non_string_sequence seems buggy - it claims to return True for types that can be cast to Sequence, but it allows Sets.

It's also smelly - it tests against a whole bunch of types, while issubclass(..., Sequence) should suffice, unless I'm missing something.

Also, Collections.abc.Sequence type differs from its supertypes by adding reverse and __gettem__. Reverse is never used and index access is only used once in _extract_multipart. I'm not sure about the intention behind using is_non_string_sequence.

I suspect the problem is with python type system lacking a type suitable for sequential processing
Sequence doesn't include sets, and Iterable includes Mapping and str.

Perhaps we could introduce a new type, lets call it ArrayLike for now, as it can be represented as JSON arrays and list parameters in HTTP.

litestar/utils/predicates.py Outdated Show resolved Hide resolved
litestar/utils/predicates.py Show resolved Hide resolved
@rafalkrupinski
Copy link
Contributor Author

Can someone either accept it or drop it, before the changes go completely outdated? Thanks!

Copy link
Member

@provinzkraut provinzkraut left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Alright, I've taken another look at this and I think the change as such is good. The implementation can be simplified though to the following:

  • Add an is_iterable property to FieldDefinition
  • Set is_sequence=field_definition.is_sequence and not field_definition.is_mapping

The rest can be removed.

Also, this should include a test case that reproduces the actual bug it's fixing.

@rafalkrupinski
Copy link
Contributor Author

Alright, I've taken another look at this and I think the change as such is good. The implementation can be simplified though to the following:

* Add an `is_iterable` property to `FieldDefinition`

* Set `is_sequence=field_definition.is_sequence and not field_definition.is_mapping`

The rest can be removed.

Makes sense

Also, this should include a test case that reproduces the actual bug it's fixing.

I'd rather copy and adjust an existing test, not to break existing patterns, but I couldn't find anything similar.

@provinzkraut
Copy link
Member

I'd rather copy and adjust an existing test, not to break existing patterns, but I couldn't find anything similar.

You could basically just copy the example you gave in #3631, and create a parametrized test case that shows it works as expected with a bunch of iterable types that should be supported.

@rafalkrupinski
Copy link
Contributor Author

I wrote a nice matrix parametrized tests with Iterable, Sequece, List x str, int and it fails for Iterable
The patch as it is makes litestar generate correct openapi, but msgspec officially doesn't support Iterable.

I think the list of supported types could be clearer and link to https://jcristharif.com/msgspec/supported-types.html

@provinzkraut
Copy link
Member

I wrote a nice matrix parametrized tests with Iterable, Sequece, List x str, int and it fails for Iterable The patch as it is makes litestar generate correct openapi, but msgspec officially doesn't support Iterable.

I think the list of supported types could be clearer and link to https://jcristharif.com/msgspec/supported-types.html

Well that was kind of my point :)
It doesn't really make sense to decode into an Iterable, as you'll always have to decode into a concrete type, in this case a list. If we were to support encoding into Iterable, we'd have to override its actual type for the conversion to a list at some point.

IMO it's easier (and more correct) to just use a list to begin with.

Maybe your original problem can be addressed in a different way? Iiuc it's about setting defaults for mutable types. Maybe supporting a default_factory would help?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Enhancement: Allow parameter of type Iterable to be processed as list
2 participants