Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove br encoding #523

Merged
merged 2 commits into from
Apr 9, 2024
Merged

Remove br encoding #523

merged 2 commits into from
Apr 9, 2024

Conversation

dogancanbakir
Copy link
Member

@dogancanbakir dogancanbakir commented Mar 13, 2024

Proposed changes

Closes #519.

This PR removes br encoding from the request as the issue is not always reproducible.

Checklist

  • Pull request is created against the dev branch
  • All checks passed (lint, unit/integration/regression tests etc.) with my changes
  • I have added tests that prove my fix is effective or that my feature works
  • I have added necessary documentation (if appropriate)

Copy link
Member

@Mzack9999 Mzack9999 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

While this might be a viable solution, I think the proper one should be decoding the body it if the header is present, or even better detect br encoding via the initial magic bytes (even though this doesn't seem like the case for brotli). What do you think? <= implemented at projectdiscovery/utils#371

@Mzack9999 Mzack9999 self-requested a review March 15, 2024 10:53
Copy link
Member

@Mzack9999 Mzack9999 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm - not sure it's anymore necessary with projectdiscovery/utils#371

@dogancanbakir
Copy link
Member Author

Thanks for the review. Unfortunately, even with projectdiscovery/utils#371, the behavior was non-deterministic. So, that's why I raised this PR /shrug.

@tarunKoyalwar
Copy link
Member

how about using https://pkg.go.dev/net/http#DetectContentType to detect content type manually by magic bytes. If i am not wrong net/http already does this for every request or response

Copy link
Member

@tarunKoyalwar tarunKoyalwar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm !

@tarunKoyalwar tarunKoyalwar requested a review from ehsandeep April 9, 2024 15:05
@ehsandeep ehsandeep merged commit 1a55878 into dev Apr 9, 2024
8 checks passed
@ehsandeep ehsandeep deleted the remove_br_encoding branch April 9, 2024 15:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

proxify is missing some encodings the proxify not decrypt the body response
4 participants