Double percent encoding of forward slashes (%2F) in paths

[DanielFischer79]

Describe the bug

we use yarp in front of an nexus server which hosting node packages.
Node packages urls can contains %2f instead of slash as part of the path.
yarp encode the percent again to %25 so the resulting path was wrongly modifed.

To Reproduce

GET https://nexus.domain.com/repository/npm-proxy/@types%2fcors

YARP convert the url to, but its wrong:
https://nexus.domain.com/repository/npm-proxy/@types%252fcors

YARP should not modify the path by default. If there is a special use case for duing that, it should be done by a feature flag

Further technical details

We tested it with new version 1.0.0 / .net 5.0

We analysed the issue in detail and found the reason for this issue: #
By fixing the issue #1219 a major change was done with path encoding.
The char '%' was not longer a valid path character for yarp. (Class RequestUtilities, Method: IsValidPathChar)

We temporary fix the issue by creating a custom request transformer and set the uri like RequestUtilities did it before.

[Tratcher]

There's a fundamental issue in aspnetcore with %2F and %252F that makes it impossible for YARP to correctly round trip both values (dotnet/aspnetcore#30655). #1219 defaulted to the safer of the two misunderstandings since double decoding %252F could be dangerous. I don't think we'll be able to sort this out properly until .NET 7.

[karelz]

Triage: We depend on ASP.NET issue dotnet/aspnetcore#30655 to be fixed first. Then we may have to react. Moving to Backlog for now.

[mcrio]

Not sure if I may be doing something differently than @DanielFischer79 but I am observing the following:

• Original path: /path/value%2F123
• Yarp transforms path to: /path/value/123

In my case Yarp seems to be url-decoding the path.

Yarp 1.0.0 on Kestrel

EDIT: actually it seems like Daniel said it does encode it to /path/value%252F123. Then I assume the destination application which is an ASP core (Kestrel) application decodes that internally to /path/value/123 :/

[mcrio]

Conclusion for my case. It is the combination of YARP > DAPR > ASP CORE that converts /path/value%2F123 to /path/value/123

Steps:

• Original path: /path/value%2F123
• Yarp requests path as DAPR method invoke (PathPrefix transform): /v1.0/invoke/myservice/method/path/value%252F123
• Dapr seems to be doing some path magic...
• ASP core incoming path: /path/value/123

Dapr seems to be doing some magic as well because when I make a request to ASP Core with path /path/value%252F123 it resolves it to /path/value%2F123... while when making the request through Dapr it resolves to /path/value/123

[karelz]

Triage: Depends solely on ASP.NET changes, no YARP work -- see #1419 (comment), will be (hopefully) part of 7.0.