Hanging onto sockets after error.

[burtyb]

I have multiple PicoW with INA219 running CircuitPython with Adafruit_CircuitPython_MiniMQTT subscribed to a topic (toggles a pin) and publishing INA219 volts/amps every 30 seconds. Some of them have poor wifi coverage (hidden behind computers, compressor tanks, etc.) so instead of resetting on error I've been trying to catch Exceptions and basically start again by reconnect to wifi and MQTT (trying to avoid a pin state changing that happens using microcontroller.reset() ).

Using wifi/mqtt code based on the example https://github.com/adafruit/Adafruit_CircuitPython_MiniMQTT/blob/main/examples/native_networking/minimqtt_adafruitio_native_networking.py ( https://github.com/8086net/usb-pwr-switch-pro-examples/blob/9e58f3e02d9033cacccc402575df6437b1ec731f/CircuitPython/wifi-mqtt-switch-prom/code.py ) I found after MQTT had been disconnected/reconnected 7 or 8 times I was eventually seeing "WARNING - Socket error when connecting: Out of sockets" which I couldn't figure out how to recover from.

[01:32:01.419] Waiting for next poll
[01:32:01.423] 748977.750: DEBUG - waiting for messages for 1 seconds
[01:32:02.429] 748978.750: DEBUG - Loop timed out after 1 seconds
[01:32:03.434] 748979.750: DEBUG - waiting for messages for 1 seconds
[01:32:13.439] Exception
[01:32:13.440] ('Unable to receive 1 bytes within 10 seconds.', None)

[01:32:34.467] Attempting to connect to brokerhostname
[01:32:34.471] 749010.750: DEBUG - Attempting to connect to MQTT broker (attempt #0)
[01:32:34.473] 749010.750: DEBUG - Attempting to establish MQTT connection...
[01:32:34.518] 749010.750: WARNING - Socket error when connecting: Out of sockets
[01:32:34.520] 749010.750: DEBUG - Resetting reconnect backoff
[01:32:34.523] 749010.750: DEBUG - Attempting to connect to MQTT broker (attempt #0)
[01:32:34.525] 749010.750: DEBUG - Attempting to establish MQTT connection...
[01:32:34.541] 749010.750: WARNING - Socket error when connecting: Out of sockets
[01:32:34.543] 749010.750: DEBUG - Resetting reconnect backoff
[01:32:34.546] 749010.750: DEBUG - Attempting to connect to MQTT broker (attempt #0)
[01:32:34.548] 749010.750: DEBUG - Attempting to establish MQTT connection...
[01:32:34.563] 749010.750: WARNING - Socket error when connecting: Out of sockets
[01:32:34.566] 749010.750: DEBUG - Resetting reconnect backoff
[01:32:34.568] 749010.750: DEBUG - Attempting to connect to MQTT broker (attempt #0)
[01:32:34.571] 749010.750: DEBUG - Attempting to establish MQTT connection...
[01:32:34.586] 749010.750: WARNING - Socket error when connecting: Out of sockets
[01:32:34.588] 749010.750: DEBUG - Resetting reconnect backoff
[01:32:34.591] 749010.750: DEBUG - Attempting to connect to MQTT broker (attempt #0)
[01:32:34.594] 749010.750: DEBUG - Attempting to establish MQTT connection...
[01:32:34.608] 749010.750: WARNING - Socket error when connecting: Out of sockets
[01:32:34.610] ('Repeated connect failures', None)
[01:32:34.610] Unable to connect MQTT

anecdata on Discord suggested using connection manager, so I switched the wifi/mqtt code to be more like https://github.com/adafruit/Adafruit_CircuitPython_MiniMQTT/blob/main/examples/minimqtt_simpletest.py ( https://github.com/8086net/usb-pwr-switch-pro-examples/blob/10263d5b2d9e43bff5ac38f8c1fc0633b7a1df6a/CircuitPython/wifi-mqtt-switch-prom/code.py - but without the close all on line 119) whilst this didn't resolve the problem it changed the error to "Socket error when connecting: An existing socket is already connected to mqtt://...".

[2025-01-06T20:05:07.952] Waiting for next poll
[2025-01-06T20:05:07.956] 259667.250: DEBUG - waiting for messages for 1 seconds
[2025-01-06T20:05:08.361] Exception
[2025-01-06T20:05:08.362] function takes 3 positional arguments but 2 were given

[2025-01-06T20:05:40.946] Attempting to connect to brokerhostname
[2025-01-06T20:05:40.950] 259700.250: DEBUG - Attempting to connect to MQTT broker (attempt #0)
[2025-01-06T20:05:40.952] 259700.250: DEBUG - Attempting to establish MQTT connection...
[2025-01-06T20:05:40.957] 259700.250: WARNING - Socket error when connecting: An existing socket is already connected to mqtt://brokerhostname:1883
[2025-01-06T20:05:40.960] 259700.250: DEBUG - Resetting reconnect backoff
[2025-01-06T20:05:40.963] 259700.250: DEBUG - Attempting to connect to MQTT broker (attempt #0)
[2025-01-06T20:05:40.966] 259700.250: DEBUG - Attempting to establish MQTT connection...
[2025-01-06T20:05:40.970] 259700.250: WARNING - Socket error when connecting: An existing socket is already connected to mqtt://brokerhostname:1883
[2025-01-06T20:05:40.973] 259700.250: DEBUG - Resetting reconnect backoff
[2025-01-06T20:05:40.976] 259700.250: DEBUG - Attempting to connect to MQTT broker (attempt #0)
[2025-01-06T20:05:40.978] 259700.250: DEBUG - Attempting to establish MQTT connection...
[2025-01-06T20:05:40.983] 259700.250: WARNING - Socket error when connecting: An existing socket is already connected to mqtt://brokerhostname:1883
[2025-01-06T20:05:40.985] 259700.250: DEBUG - Resetting reconnect backoff
[2025-01-06T20:05:40.988] 259700.250: DEBUG - Attempting to connect to MQTT broker (attempt #0)
[2025-01-06T20:05:40.990] 259700.250: DEBUG - Attempting to establish MQTT connection...
[2025-01-06T20:05:40.995] 259700.250: WARNING - Socket error when connecting: An existing socket is already connected to mqtt://brokerhostname:1883
[2025-01-06T20:05:40.997] 259700.250: DEBUG - Resetting reconnect backoff
[2025-01-06T20:05:41.001] 259700.250: DEBUG - Attempting to connect to MQTT broker (attempt #0)
[2025-01-06T20:05:41.003] 259700.250: DEBUG - Attempting to establish MQTT connection...
[2025-01-06T20:05:41.007] 259700.250: WARNING - Socket error when connecting: An existing socket is already connected to mqtt://brokerhostname:1883
[2025-01-06T20:05:41.008] ('Repeated connect failures', None)
[2025-01-06T20:05:41.009] Unable to connect MQTT

At the same time anecdata also suggested using "connection_manager_close_all()" if the change didn't help so I added "adafruit_connection_manager.connection_manager_close_all(release_references=True)" before the code tries to connect again (ending up with code like https://github.com/8086net/usb-pwr-switch-pro-examples/blob/10263d5b2d9e43bff5ac38f8c1fc0633b7a1df6a/CircuitPython/wifi-mqtt-switch-prom/code.py ).

After adding a close all the PicoW have been stable for 3 weeks (one having reconnected over 100 times).

So similar to #222 (which doesn't seem to have had the underlying issue resolved) it looks like sockets are being left behind somehow after an error occurs (or maybe I'm missing something obvious).

When I say "code like" above it basically means I didn't have the logging lines commented out and had lots of printing of gc.mem_free() surrounding gc.collect() as after looping through the "WARNING - Socket error when connecting: Out of sockets" message many times free memory went down from ~90k until "memory allocation failed, allocating 296 bytes" and got worse from there as you'd expect.