Downtime for https://opnxng.com

[extremelyonline]

Hello,

My instance is getting google ( [Suspended: too many requests] errors. I have used the mobile_ui.

It's likely that my instance is blocked by Google for the time being (?)

I tried enabling bing, duckduckgo, and yahoo at the meantime but they didn't play nice. Please consider my instance as down for now. I will report back,

Too many request
searx/search/processors/online.py:117
_send_http_request
response = req(params['url'], **request_args)

[unixfox]

It's technically not a downtime if it can still respond to the requests.

Don't worry we do not track these kind of uptime.

[extremelyonline]

Thank you. It has been more than 12 hours and the issue persists. I am afraid that it will take a while to debug and fix.

My instance is currently among the top 10-20 (fastest) instances on the list despite not being functional. If you think it is best, please remove the instance from the list.

[unixfox]

If you set other engines by default that works it's fine. I do host a public instance myself and sometimes google is blocked too unfortunately. That's not something you can easily solve.

I would like to point out though that your instance returns "too many requests" when someone do a request from chrome on android, especially on this page: https://opnxng.com/search?q=ok&category_general=1&language=all&time_range=&safesearch=0&theme=simple

[return42]

I would like to point out though that your instance returns "too many requests" when someone do a request from chrome on android,

Same with Firefox on desktop and other browsers.

My instance is getting google ( [Suspended: too many requests] errors. I have used the mobile_ui.

The engines of your instance have various problems, in #292 I see that you are using a branch from @mrpaulblack ...

• Do you have any other customizations?
• What happens if you use the master branch of SearXNG?
• Does your proxy filter or modify the outgoing requests?
• What is your proxy configuration?

[extremelyonline]

Thanks for checking.

I have some hostname_replace customization and disabled some engines. The instance is placed behind a Nginx proxy.

The set up was fine for a long time. I'm switching the the master branch with default settings to try to debug. My instance may be down in the meantime. Will report.

[extremelyonline]

Update:

• What happens if you use the master branch of SearXNG?

Tested using the master branch (docker) with use_default_settings: true. Google works for about 3 mins than gives google ( [Suspended: too many requests]. This seems to be unlike the usual, occasional errors that I've experienced.

• Does your proxy filter or modify the outgoing requests? What is your proxy configuration?

The instance is behind a Nginx proxy (docker). The requests are not modified.

Google image works (image proxy=true)

The set up was fine for a long time. I don't recall changing any settings except setting default_lang to "all" a while ago.

[unixfox]

I still get the "too many requests" error page. You should probably check why you are getting the error, people on android can't use your instance.

[extremelyonline]

[unixfox]

Reinstalling will do nothing. Sometimes google block the IP address for a long time.

You might also have a lot of bots using your instance and thus having google block your instance.

[extremelyonline]

The problem seems to be solved. It has to do with a Wireguard tunnel on the vps. I started a new vps without it and switched to Caddy. Everything seems fine now. Thanks.

[extremelyonline]

@unixfox, sorry to bother you again. Google search now works on my instance on browsers.

However, on searx.space, sometimes the instance is reported as having 429 errors. Is this normal or a problem with my instance? Thank you.

[unixfox]

See #321 (comment)

[extremelyonline]

Thank you. I've reinstalled and updated the settings to include yahoo.

The instance was taken off the list during this downtime. Would you consider adding it back (after a week/month)? Sorry for all the inconvenience caused.

[unixfox]

The instance was taken off the list during this downtime. Would you consider adding it back (after a week/month)? Sorry for all the inconvenience caused.

It's automatic.

[unixfox]

However, on searx.space, sometimes the instance is reported as having 429 errors. Is this normal or a problem with my instance? Thank you.

It's a problem with your instance.

[extremelyonline]

The problem is coming back again. Consistently getting errors relating to searx/search/processors/online.py:117 in Google, DDG, and Yahoo.

I've tried getting a new ip, changing the DNS server, switching to the main branch, switching to caddy, using default settings, disabling http2, changing the timeout (to more than 3 sec), changing the suspended time (to 0 and 1200), disabling a Wireguard tunnel, setting default_lang to "all"...

It's strange because the instance was running fine one or two weeks ago. Perhaps my network speed is too slow? I'm using the cheapest Vultr plan on a Singapore server. I suspect I need to find the optimal suspended_time settings. 1966, 2132.

[return42]

Not sure it is related to your issue, but a reverse lookup of your server's IP shows:

$ nslookup 139.180.212.73
73.212.180.139.in-addr.arpa   name = 139.180.212.73.vultrusercontent.com.

[extremelyonline]

I have upgraded the Vultr instance with more ram and bandwidth as well as setting the correct reverse look up.

Search has become more reliable now. I'm not sure what fixed it (or if it's just luck). But I'll keep monitoring. Thanks everyone!

[extremelyonline]

Problem persists. I've even added two outgoing proxies. Still consistently banned by google and yahoo.

[extremelyonline]

I'm very sorry to bother everyone. The situation is frustrating and I'm not technically equipped to troubleshot.

SInce today, I've been getting *** uWSGI listen queue of socket "0.0.0.0:8080" (fd: 4) full !!! (101/100) *** in the docker logs. Caddy logs shows many ips accessing my instance (with remote_port other than 443/80). I cannot even access my instance now... Would this indicate that my instance is being attacked by bots?

[mrpaulblack]

Hi,
I also had this problem on my instance in the past. If I understand correctly uwsgi has a queue with requests coming in an then service workers and server the requests... On my instance the proxies replies where to slow and therefore the service workers would be blocked while waiting on the proxy answer...

This is a bit hacky, but what I did on my instance is to increase the size of the workers: https://github.com/paulgoio/searxng/blob/ab4659c437c3cf0c17216596f04472ede5e0d6c8/src/run.sh#L22

If you set for example 4 workers with 2 threads you are going to have 8 threads in total (workers * threads); On my instance I set the values to 8 workers with 4 threads each: https://github.com/paulgoio/searxng/blob/ab4659c437c3cf0c17216596f04472ede5e0d6c8/docker-compose.yml#L39