2024.9 WARNING - Detected blocking call...

[Petro31]

IMPORTANT: Please search the issues, including closed issues, and the FAQ before opening a new issue. The template is mandatory; failure to use it will result in issue closure.

Describe the bug

2024.9 causes this error to appear. Haven't investigated, just documenting for now. Causes entire integration to fail to load, however HA believes it loads.

To Reproduce

1. Install beta

Expected behavior

Integration loads.

Screenshots

System details

• Home Assistant version: 2024.9
• alexa_media version (from const.py or HA startup log): 4.12.11
• alexapy version (from pip show alexapy in homeasssistant container or HA startup log): 1.28.2
• Is Amazon 2FA/2SV enabled <!---We will not debug login issues if unanswered---> (y/n): y
• Amazon Domain: north america

Debug Logs (alexa_media & alexapy)
Please provide logs.

2024-08-28 16:17:20.339 WARNING (MainThread) [homeassistant.util.loop] Detected blocking call to load_verify_locations with args (<ssl.SSLContext object at 0x7fe134eecf50>, '/usr/local/lib/python3.12/site-packages/certifi/cacert.pem', None, None) inside the event loop by custom integration 'alexa_media' at custom_components/alexa_media/__init__.py, line 340: AlexaLogin( (offender: /usr/local/lib/python3.12/ssl.py, line 708: context.load_verify_locations(cafile, capath, cadata)), please create a bug report at https://github.com/alandtse/alexa_media_player/issues
For developers, please see https://developers.home-assistant.io/docs/asyncio_blocking_operations/#load_verify_locations
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/usr/src/homeassistant/homeassistant/__main__.py", line 223, in <module>
    sys.exit(main())
  File "/usr/src/homeassistant/homeassistant/__main__.py", line 209, in main
    exit_code = runner.run(runtime_conf)
  File "/usr/src/homeassistant/homeassistant/runner.py", line 189, in run
    return loop.run_until_complete(setup_and_run_hass(runtime_config))
  File "/usr/local/lib/python3.12/asyncio/base_events.py", line 674, in run_until_complete
    self.run_forever()
  File "/usr/local/lib/python3.12/asyncio/base_events.py", line 641, in run_forever
    self._run_once()
  File "/usr/local/lib/python3.12/asyncio/base_events.py", line 1990, in _run_once
    handle._run()
  File "/usr/local/lib/python3.12/asyncio/events.py", line 88, in _run
    self._context.run(self._callback, *self._args)
  File "/usr/src/homeassistant/homeassistant/setup.py", line 165, in async_setup_component
    result = await _async_setup_component(hass, domain, config)
  File "/usr/src/homeassistant/homeassistant/setup.py", line 461, in _async_setup_component
    await asyncio.gather(
  File "/usr/src/homeassistant/homeassistant/setup.py", line 463, in <genexpr>
    create_eager_task(
  File "/usr/src/homeassistant/homeassistant/util/async_.py", line 45, in create_eager_task
    return Task(coro, loop=loop, name=name, eager_start=True)
  File "/usr/src/homeassistant/homeassistant/config_entries.py", line 752, in async_setup_locked
    await self.async_setup(hass, integration=integration)
  File "/usr/src/homeassistant/homeassistant/config_entries.py", line 604, in async_setup
    result = await component.async_setup_entry(hass, self)
  File "/config/custom_components/alexa_media/__init__.py", line 340, in async_setup_entry
    AlexaLogin(

Additional context

[danielbrunt57]

No comment!
Other integrations are also triggering the same WARNING...

[jleinenbach]

ChatGPT says that there's no easy way to fix these warnings and gave me this instead:

Plan for Reducing Blocking Calls in Home Assistant

This plan outlines a careful approach to reducing blocking calls in the Home Assistant environment, particularly focusing on the AlexaLogin and HTTP2EchoClient classes. The steps include identifying and isolating blocking operations, refactoring initialization processes, and thorough testing to ensure stability.

1. Analyze and Identify Blocking Calls

• Objective: Pinpoint the exact blocking calls within the AlexaLogin and HTTP2EchoClient classes.
• Action:
  • Use tools like strace (for system calls) or debug logs to trace where blocking occurs.
  • Review the source code of these classes to identify operations that involve I/O, SSL initialization, or synchronous network requests.
  • Not all blocking operations can be safely moved to a different thread. For instance, objects that depend on the asyncio event loop (like aiohttp.ClientSession) must remain in the main thread.

2. Isolate Blocking Sections

• Objective: Break down the blocking calls into smaller, isolated sections that can be offloaded without moving the entire object initialization out of the main thread.
• Action:
  • Isolate specific blocking methods, such as load_verify_locations in SSL initialization, to run in a thread using asyncio.to_thread.
  • Keep the rest of the AlexaLogin or HTTP2EchoClient initialization in the main async context.
  • Ensure that the operations moved to a thread do not interact with the event loop. If an object makes async calls or depends on event loop state, it should stay in the main thread.

async def initialize_ssl_context():
    ssl_context = await asyncio.to_thread(create_ssl_context)
    return ssl_context

def create_ssl_context():
    context = ssl.create_default_context()
    context.load_verify_locations(cafile='/path/to/certfile')
    return context

3. Refactor the Initialization Process

• Objective: Ensure that the isolated blocking operations are correctly handled before proceeding with the rest of the initialization.
• Action:
  • Refactor the initialization methods in __init__.py to use async patterns effectively.
  • Chain the blocking call handling so that each step can be awaited asynchronously, ensuring the Event Loop isn’t blocked.
  • Ensure thread-safety when working with shared data structures. Shared resources need to be properly synchronized when parts of the initialization are moved to a separate thread.

async def async_setup_entry(hass, config_entry):
    ssl_context = await initialize_ssl_context()
    login_obj = await asyncio.to_thread(
        AlexaLogin,
        url=config_entry.data[CONF_URL],
        email=config_entry.data[CONF_EMAIL],
        password=config_entry.data[CONF_PASSWORD],
        ssl_context=ssl_context,
        ...
    )
    # Continue with the rest of the async setup

4. Test the Refactored Code

• Objective: Validate that the refactored code does not introduce deadlocks or runtime errors.
• Action:
  • Use extensive unit tests with mocked network responses and SSL contexts to simulate different scenarios.
  • Deploy the changes in a staging environment if possible, and monitor logs for any unexpected behavior.
  • Pay close attention to testing environments. Simulated async environments can sometimes behave differently than production, so consider running tests in an environment as close to production as possible.

5. Iterate and Optimize

• Objective: Continuously refine the approach based on feedback and performance.
• Action:
  • If some blocking operations cannot be safely isolated, reconsider the necessity of these operations at that stage in the initialization process.
  • Experiment with other async libraries or patterns if certain blocks cannot be handled with asyncio.to_thread.
  • Be cautious of hidden dependencies. Asynchronous programming can sometimes mask performance issues or introduce subtle bugs, so continue monitoring even after the changes seem stable.

Conclusion

This plan carefully balances the need to reduce blocking operations with the requirement to maintain system stability. By methodically isolating and handling blocking calls, while thoroughly testing and optimizing, we can achieve a more responsive and efficient Home Assistant environment.

[danielbrunt57]

2. Isolate Blocking Sections

• Objective: Break down the blocking calls into smaller, isolated sections that can be offloaded without moving the entire object initialization out of the main thread.

• Action:

  • Isolate specific blocking methods, such as load_verify_locations in SSL initialization, to run in a thread using asyncio.to_thread.
  • Keep the rest of the AlexaLogin or HTTP2EchoClient initialization in the main async context.
  • Ensure that the operations moved to a thread do not interact with the event loop. If an object makes async calls or depends on event loop state, it should stay in the main thread.

async def initialize_ssl_context():
    ssl_context = await asyncio.to_thread(create_ssl_context)
    return ssl_context

def create_ssl_context():
    context = ssl.create_default_context()
    context.load_verify_locations(cafile='/path/to/certfile')
    return context

I implemented the following to resolve the warning:

class AlexaLogin:
    # pylint: disable=too-many-instance-attributes
    """Class to handle login connection to Alexa. This class will not reconnect.

    Args:
    url (string): Localized Amazon domain (e.g., amazon.com)
    email (string): Amazon login account
    password (string): Password for Amazon login account
    outputpath (function): Local path with write access for storing files
    debug (boolean): Enable additional debugging including debug file creation
    otp_secret (string): TOTP Secret key for automatic 2FA filling
    uuid: (string): Unique 32 char hex to serve as app serial number for registration

    """

    def __init__(
        self,
        url: str,
        email: str,
        password: str,
        outputpath: Callable[[str], str],
        debug: bool = False,
        otp_secret: str = "",
        oauth: Optional[dict[Any, Any]] = None,
        uuid: Optional[str] = None,
        oauth_login: bool = True,
    ) -> None:
        # pylint: disable=too-many-arguments,import-outside-toplevel
        """Set up initial connection and log in."""
# Replace:
#        import ssl
# With:
        from homeassistant.util import ssl

        import certifi

        oauth = oauth or {}
        self._hass_domain: str = "alexa_media"
        self._prefix: str = "https://alexa."
        self._url: str = url
        self._email: str = email
        self._password: str = password
        self._session: Optional[aiohttp.ClientSession] = None
# Replace:
#        self._ssl = ssl.create_default_context(
#            purpose=ssl.Purpose.SERVER_AUTH, cafile=certifi.where()
#        )
# With:
        self._ssl = ssl.get_default_context()
        self._headers: dict[str, str] = {}
        self._data: Optional[dict[str, str]] = None
        self.status: Optional[dict[str, Union[str, bool]]] = {}
        self.stats: Optional[dict[str, Union[str, bool]]] = {
            "login_timestamp": datetime.datetime(1, 1, 1),
            "api_calls": 0,
        }

In your opinion, is this a viable alternative to your suggestion?

[jleinenbach]

Your suggestion is a good idea if alexapy is a library only for Home Assistant. By using Home Assistant’s native methods like homeassistant.util.ssl.get_default_context(), you ensure better integration and compatibility with Home Assistant. This approach likely prevents blocking operations and enhances maintainability and security. Of course, it’s important to test these changes to confirm that they effectively resolve the original problem.