Add option to forbid following HTTP redirects to private/reserved/special use networks

[mzpqnxow]

The option --follow-localhost-redirects is for those that might want to explicitly permit HTTP redirects to localhost (I can't imagine many people use that, seems like a giant security hole) but there is no option to permit or disallow redirects to private networks or other special use networks. I believe they're permitted by default, which while theoretically a bit risky, makes sense for a default behavior

I think that behavior should remain the same, but it would be nice to have a feature to disallow redirects to, e.g. 10/8, 172.16/12 192.168/16 and the dozen or so other IANA reserved or special blocks Something like --disallow-nonroutable-redirect or --disallow-private-redirect?

Would you accept a PR for this?

[mzpqnxow]

The option --follow-localhost-redirects is for those that might want to explicitly permit HTTP redirects to localhost (I can't imagine many people use that, seems like a giant security hole) but there is no option to permit or disallow redirects to private networks or other special use networks. I believe they're permitted by default, which while theoretically a bit risky, makes sense for a default behavior

I think that behavior should remain the same, but it would be nice to have a feature to disallow redirects to, e.g. 10/8, 172.16/12 192.168/16 and the dozen or so other IANA reserved or special blocks Something like --disallow-nonroutable-redirect or --disallow-private-redirect?

Would you accept a PR for this?

I'll answer my own question, I'm sure a PR would be accepted if it was an opt-in feature and implemented sanely

But... I clearly haven't had time to do this. Maybe someone else will pick it up, otherwise I'll close it out in a few months

[mzpqnxow]

@phillip-stephens any interest in taking this one on? I intended to do it but haven't had time, and I see you've been productive lately 😊

(If not, no worries)

[Seanstoppable]

Should we actually provide the functionality to specify a file, ala the zmap blocklist, as that is the 'default' of how zmap handles that
Then, actual added benefit that scanners could re-use the blocklist and not accidentally probe an ip because of a redirect?

[mzpqnxow]

Should we actually provide the functionality to specify a file, ala the zmap blocklist, as that is the 'default' of how zmap handles that

Then, actual added benefit that scanners could re-use the blocklist and not accidentally probe an ip because of a redirect?

Would certainly be more extensible that way

I imagine in practice, most people using it would end up specifying roughly the IANA networks I mentioned anyway, but at least with your idea they would have the option to add any that may have been forgotten, rather than be stuck with a hard-coded list

I think that's a long way of saying that I agree with your idea 😊