Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Custom LINQ Provider #87

Open
Turnerj opened this issue Jul 9, 2019 · 3 comments
Open

Custom LINQ Provider #87

Turnerj opened this issue Jul 9, 2019 · 3 comments
Labels
affects-querying Query-related issue enhancement Enhancements & features major-change Major changes required
Milestone

Comments

@Turnerj
Copy link
Member

Turnerj commented Jul 9, 2019

While #54 talks about replacing the entire driver, this is a more focused change on replacing the entire LINQ pipeline of the driver with a custom implementation.

The main benefit is extensibility of queries and potentially generation of more optimal queries.
This project is still a huge undertaking but is one independent step towards a better library.

Thoughts:

  • Have some type of mapping service between the IQueryable extension methods and the actual functions that will help generate that part of the stage.
  • Have this mapper be able to be updated at runtime by passing in the function you want mapped and the class capable of mapping the value to some intermediate type
  • Like the driver, certain commands that appear after each other (like two or more Where calls) need to be merged together.
  • Would entirely eliminate the reflection methods used to support the BsonDocument pre-stage stuff etc
  • Would allow more generic support for things like $geoNear etc
  • Optimising away the ResultTransformer, allowing better queries (eg. .Any()) to be translated more directly

The hardest thing in the whole build of this is all the sub-query bits, things like complex Where or Select statements.

@Turnerj Turnerj added enhancement Enhancements & features affects-querying Query-related issue major-change Major changes required labels Jul 9, 2019
@Turnerj
Copy link
Member Author

Turnerj commented Jul 9, 2019

It would also be really cool to add support for explain on the query. This could be taken even a step further suggesting the ideal way to query that data (commands, indexes etc).

@Turnerj Turnerj added this to the 1.0.0 milestone Jul 9, 2019
Turnerj added a commit that referenced this issue Nov 5, 2019
Kickstarts #87 though a ton more work is needed
@Turnerj
Copy link
Member Author

Turnerj commented Nov 5, 2019

As I suspected, it should be fairly straight forward to build the outer stages and having them configurable at runtime via a custom query converter. The harder bit is definitely parsing the internal lambda expressions.

Things it needs to consider:

  • Handling constants (effectively, they need to be written to the output BsonDocument)
  • Handling nested queries
  • Handling calls to certain property types (eg. MyDate.Date)
  • Handling nested method calls in general (there might be a nice way I can allow that to help build the query with the same general infrastructure)

Other changes required:

@Turnerj
Copy link
Member Author

Turnerj commented Sep 20, 2020

Basic benchmark of the current LINQ query generation. Some surprises but overall, not fantastic.

BenchmarkDotNet=v0.12.1, OS=Windows 10.0.18362.1082 (1903/May2019Update/19H1)
Intel Core i7-6700HQ CPU 2.60GHz (Skylake), 1 CPU, 8 logical and 4 physical cores
.NET Core SDK=5.0.100-rc.1.20452.10
  [Host]        : .NET Core 3.0.1 (CoreCLR 4.700.19.51502, CoreFX 4.700.19.51609), X64 RyuJIT
  .NET Core 3.0 : .NET Core 3.0.1 (CoreCLR 4.700.19.51502, CoreFX 4.700.19.51609), X64 RyuJIT

Job=.NET Core 3.0  Runtime=.NET Core 3.0
Method Mean Error StdDev Gen 0 Gen 1 Gen 2 Allocated
FirstOrDefault 775.0 us 5.83 us 5.46 us 19.5313 - - 62.21 KB
ToArray 784.4 us 14.69 us 16.32 us 15.6250 - - 49.95 KB
Count 1,021.1 us 13.37 us 12.51 us 23.4375 - - 72.99 KB
Any 1,010.0 us 14.41 us 12.78 us 23.4375 - - 72.82 KB

Turnerj added a commit that referenced this issue Dec 9, 2020
Kickstarts #87 though a ton more work is needed
Turnerj added a commit that referenced this issue Dec 17, 2020
Kickstarts #87 though a ton more work is needed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
affects-querying Query-related issue enhancement Enhancements & features major-change Major changes required
Projects
None yet
Development

No branches or pull requests

1 participant