Adding more restriction on public instances

[ononoki1]

There are more than 90 online public instances now. I think we can add more restriction on them, since even with these restriction there will still be plenty of available instances.

Here are some examples:

• Instances using Cloudflare as reverse proxy must not be added
• Instances using tracking scripts must not be added
• Instances with TLS grade F must not be added

Note: TLS grade E is acceptable, since DH ciphers are safe to use according to Mozilla's advice.

These restriction should be fine, because most public instances do not violate them, especially recent joined ones.

[tiekoetter]

100% agree with your first two points. The TLS grade should also be strictly enforced in my opinion because if a client doesn't support the "modern" ciphers it is very likely that the client itself is unsafe.

[unixfox]

While I don't like cloudflare, I'm against removing the instances that are hosted on networks that do MITM. It's up to the user to not choose them and us to explain why it's not a good idea to use them. A maintainer may have good reasons to host it behind cloudflare:

• allow his instance to be available from the ipv4 world because the server only has IPv6
• some firewall rules that the wants to implement and are more simple on cloudflare than on iptables
• to do some loadbalancing for free without breaking the bank. cloudflare will randomly select a server if multiple record A are found
• and so on...

About the tracking scripts, I'm in favor of doing that, but there are hardly any instance doing it. There was serx.cf something back in the days, but the instance got shutted down: #49

And the TLS grade F, that could be a good idea. We must contact the existing owners though before removing their instance if the instance function well. If no reply after 1 month then they can be removed.

Other selectors that could be used to remove the instances would be:

• Remove instances that haven't been updated for like 2 or 3 months. That's technically only possible with SearXNG, not for SearX instances.
  At Invidious we remove instances that haven't been updated for 1 month: https://github.com/iv-org/documentation/blob/master/.github/ISSUE_TEMPLATE/new_instance.yaml
• About SearX instances, we have plans to remove the SearX instances from the list. That would also remove a big chunk of instances from the public list.
• Remove instances that have broken functionalities or broken resources (CSS or/and JS).
  Like instances that return no results by default, example: https://nibblehole.com/ (I'm removing it). Or instances with broken CSS and JS resources: https://suche.tromdienste.de/ and https://search.trom.tf/

[ononoki1]

While I don't like cloudflare, I'm against removing the instances that are hosted on networks that do MITM. It's up to the user to not choose them and us to explain why it's not a good idea to use them.

Most users do not know what Cloudflare does. They may never hear of it. It can be very hard to explain why Cloudflare should be avoided to network noobs.

Besides, there are only 3 instances using Cloudflare on searx.space. Removing them does not influence much.

And to your reasons that maintainers may use Cloudflare, there are always alternative solutions. For example, maintainers can simply use limiter plugin or modsecurity to defend against bots instead of depending on Cloudflare, using user's privacy as cost. If maintainers are not experts, they can just use searxng-docker.

When we display instances on searx.space, we have responsibility to let user's privacy and security not compromised. Of course we need to stand in the position of maintainers, but it takes some responsibilities to run a public service, it's not just install searx(ng) and that's it (you said it here). We must think for users first instead of instance maintainers.

[return42]

I agree with the criteria you both mentioned. 👍

About MITM networks I agree more with @ononoki1 ...

A maintainer may have good reasons to host it behind cloudflare:

Most users do not know what Cloudflare does.

further more ; I think the majority of the cloudflare users (SearXNG mainatiner) do not have good reasons, they even do not know what MITM means.

I also miss the balance of the decision to expect high TLS quality on one side and tolerate networks running MITM on the other.

[unixfox]

What I propose instead of banning MITM solutions is to incline the maintainer to not use it if we detect an instance behind a provider like cloudflare. If he still wants to use cloudflare then we ask them to give good reasons. We then validate if the reasons are worth to really use cloudflare, if that's the case then we allow the instance, if not we say that we won't add the instance.

The tough part in this case is that we will have to write down a list of valid reasons that allow the maintainer to use cloudflare and not just judging differently case by case if the maintainer can use cloudflare.

[hra42]

You know, I use Cloudflare not just for the reverse proxy stuff, but for hosting static files on R2, or use cloudflare workers for some custom application. The reverse proxy stuff and not having to worry about Let's encrypt is something I take as a bonus.

If you say Cloudflare is bad for privacy you are prob. right, but no ips are visible to the maintainer so I can see nothing besides some overall data about the country. When you use the strict full verify of the endpoint cloudflare uses SSL encryption to the endpoint. Cloudflare dosn't get more data then I would anyway.

Prob. you are right about the selection for this list, I don't care if my instance is on that list, if it would be, I'm fine, if not I'm fine too. Just wondered because I think its really stupid to not use Cloudflare for everything.

[unixfox]

@hra42 this github discussion is a bit outdated, please see this PR: #223