Improve metered connection detection for macOS. By @jramnani (#1902)

* Add dependency for pyobjc-CoreWLAN on darwin

* Rename existing implementation with Android

The current implementation was tested with Android, but does not work
with iOS.

Move the existing implementation and include android in the name to make
room for adding a new iOS metered connection detection strategy.

* get_current_wifi works with objc

Switch from using command line tools to using the Objective-C Cocoa API
to get the Wi-Fi status information.

Cocoa has an API to specifically check whether a Wi-Fi connection is
using a Personal Hotspot on iOS.

I'm using a private method to get the Wi-Fi interface object in Cocoa.
The reason for this is that cleaning up mocks on PyObjC/ObjC objects is
much harder than mocking out methods on objects in our control. Using
test doubles also let's me check for different states the Wi-Fi network
could be in.

* get_known_wifis works on darwin

Use the networksetup command on macOS to get the list of the user's
Wi-Fi networks.

  networksetup -listpreferredwirelessnetworks bsd_device

It looks like this command and option has existed on macOS since at
least 2013.

Also add some type annotations around the PyObjC return values to help
the reader know what they're dealing with at each step.

* Add test for get_current_wifi when wifi is off

The user might have Wi-Fi turned off. Account for that use case.

* Add iOS Personal Hotspot support to is_network_metered

The DarwinNetworkManager can now determine if the user is connected to
a Personal Hotspot Wi-Fi network from iOS.

Account for whether the user has Wi-Fi turned on and off.

* Refactor to avoid deprecated API in Cocoa

According to Apple's developer documentation, creating CWInterface
objects directly are discouraged. Instead, they prefer to use
CWInterface objects created by CWWiFiClient.

This also happens to be more compliant with Apple's application sandbox.
Creating CWInterface objects directly accesses raw BSD sockets which is
not allowed in the sandbox.

More details here:
https://developer.apple.com/documentation/corewlan/cwinterface

* Add test case for blank Wi-Fi network name

I have one of these in my list of networks in Vorta. And this also
covers a missing branch in get_known_wifis.

* Move private method below public methods

This is to provide a little more clarity. Especially since this class is
subclassing another one.

* Account for when there is no wifi interface

When a Mac does not have a Wi-Fi interface, CWWiFiClient.interface() can
return None.

Update the type annotation to mark it as Optional, and account for the
null condition in the other methods.

* Fix type annotation error

The CI tests failed on python 3.8.

I used the wrong type annotation to describe a list of SystemWifiInfo's.

The tests now pass for me when I run 'make test-unit' using a python 3.8
interpreter.

* Fix linter issue with imports

setup.cfg

@@ -47,6 +47,7 @@ install_requires =
   pyobjc-core < 10; sys_platform == 'darwin'
   pyobjc-framework-Cocoa < 10; sys_platform == 'darwin'
   pyobjc-framework-LaunchServices < 10; sys_platform == 'darwin'
+  pyobjc-framework-CoreWLAN < 10; sys_platform == 'darwin'
 tests_require =
   pytest
   pytest-qt

src/vorta/network_status/darwin.py

@@ -1,53 +1,82 @@
 import subprocess
 from datetime import datetime as dt
-from typing import Iterator, Optional
+from typing import Iterator, List, Optional
+
+from CoreWLAN import CWInterface, CWNetwork, CWWiFiClient
 
 from vorta.log import logger
 from vorta.network_status.abc import NetworkStatusMonitor, SystemWifiInfo
 
 
 class DarwinNetworkStatus(NetworkStatusMonitor):
     def is_network_metered(self) -> bool:
-        return any(is_network_metered(d) for d in get_network_devices())
+        interface: CWInterface = self._get_wifi_interface()
+        network: Optional[CWNetwork] = interface.lastNetworkJoined()
+
+        if network:
+            is_ios_hotspot = network.isPersonalHotspot()
+        else:
+            is_ios_hotspot = False
+
+        return is_ios_hotspot or any(is_network_metered_with_android(d) for d in get_network_devices())
 
     def get_current_wifi(self) -> Optional[str]:
         """
-        Get current SSID or None if Wifi is off.
-
-        From https://gist.github.com/keithweaver/00edf356e8194b89ed8d3b7bbead000c
+        Get current SSID or None if Wi-Fi is off.
         """
-        cmd = [
-            '/System/Library/PrivateFrameworks/Apple80211.framework/Versions/Current/Resources/airport',
-            '-I',
-        ]
-        process = subprocess.Popen(cmd, stdout=subprocess.PIPE)
-        out, err = process.communicate()
-        process.wait()
-        for line in out.decode(errors='ignore').split('\n'):
-            split_line = line.strip().split(':')
-            if split_line[0] == 'SSID':
-                return split_line[1].strip()
-
-    def get_known_wifis(self):
+        interface: Optional[CWInterface] = self._get_wifi_interface()
+        if not interface:
+            return None
+
+        # If the user has Wi-Fi turned off lastNetworkJoined will return None.
+        network: Optional[CWNetwork] = interface.lastNetworkJoined()
+
+        if network:
+            network_name = network.ssid()
+            return network_name
+        else:
+            return None
+
+    def get_known_wifis(self) -> List[SystemWifiInfo]:
         """
-        Listing all known Wifi networks isn't possible any more from macOS 11. Instead we
-        just return the current Wifi.
+        Use the program, "networksetup", to get the list of know Wi-Fi networks.
         """
+
         wifis = []
-        current_wifi = self.get_current_wifi()
-        if current_wifi is not None:
-            wifis.append(SystemWifiInfo(ssid=current_wifi, last_connected=dt.now()))
+        interface: Optional[CWInterface] = self._get_wifi_interface()
+        if not interface:
+            return []
+
+        interface_name = interface.name()
+        output = call_networksetup_listpreferredwirelessnetworks(interface_name)
+
+        result = []
+        for line in output.strip().splitlines():
+            if line.strip().startswith("Preferred networks"):
+                continue
+            elif not line.strip():
+                continue
+            else:
+                result.append(line.strip())
+
+        for wifi_network_name in result:
+            wifis.append(SystemWifiInfo(ssid=wifi_network_name, last_connected=dt.now()))
 
         return wifis
 
+    def _get_wifi_interface(self) -> Optional[CWInterface]:
+        wifi_client: CWWiFiClient = CWWiFiClient.sharedWiFiClient()
+        interface: Optional[CWInterface] = wifi_client.interface()
+        return interface
+
 
 def get_network_devices() -> Iterator[str]:
     for line in call_networksetup_listallhardwareports().splitlines():
         if line.startswith(b'Device: '):
             yield line.split()[1].strip().decode('ascii')
 
 
-def is_network_metered(bsd_device) -> bool:
+def is_network_metered_with_android(bsd_device) -> bool:
     return b'ANDROID_METERED' in call_ipconfig_getpacket(bsd_device)
 
 
@@ -66,3 +95,11 @@ def call_networksetup_listallhardwareports():
         return subprocess.check_output(cmd)
     except subprocess.CalledProcessError:
         logger.debug("Command %s failed", ' '.join(cmd))
+
+
+def call_networksetup_listpreferredwirelessnetworks(interface) -> str:
+    command = ['/usr/sbin/networksetup', '-listpreferredwirelessnetworks', interface]
+    try:
+        return subprocess.check_output(command).decode(encoding='utf-8')
+    except subprocess.CalledProcessError:
+        logger.debug("Command %s failed", " ".join(command))

tests/network_manager/test_darwin.py

@@ -1,25 +1,118 @@
+from unittest.mock import MagicMock
+
 import pytest
 from vorta.network_status import darwin
 
 
+def test_get_current_wifi_when_wifi_is_on(mocker):
+    mock_interface = MagicMock()
+    mock_network = MagicMock()
+    mock_interface.lastNetworkJoined.return_value = mock_network
+    mock_network.ssid.return_value = "Coffee Shop Wifi"
+
+    instance = darwin.DarwinNetworkStatus()
+    mocker.patch.object(instance, "_get_wifi_interface", return_value=mock_interface)
+
+    result = instance.get_current_wifi()
+
+    assert result == "Coffee Shop Wifi"
+
+
+def test_get_current_wifi_when_wifi_is_off(mocker):
+    mock_interface = MagicMock()
+    mock_interface.lastNetworkJoined.return_value = None
+
+    instance = darwin.DarwinNetworkStatus()
+    mocker.patch.object(instance, "_get_wifi_interface", return_value=mock_interface)
+
+    result = instance.get_current_wifi()
+
+    assert result is None
+
+
+def test_get_current_wifi_when_no_wifi_interface(mocker):
+    instance = darwin.DarwinNetworkStatus()
+    mocker.patch.object(instance, "_get_wifi_interface", return_value=None)
+
+    result = instance.get_current_wifi()
+
+    assert result is None
+
+
+@pytest.mark.parametrize("is_hotspot_enabled", [True, False])
+def test_network_is_metered_with_ios(mocker, is_hotspot_enabled):
+    mock_interface = MagicMock()
+    mock_network = MagicMock()
+    mock_interface.lastNetworkJoined.return_value = mock_network
+    mock_network.isPersonalHotspot.return_value = is_hotspot_enabled
+
+    instance = darwin.DarwinNetworkStatus()
+    mocker.patch.object(instance, "_get_wifi_interface", return_value=mock_interface)
+
+    result = instance.is_network_metered()
+
+    assert result == is_hotspot_enabled
+
+
+def test_network_is_metered_when_wifi_is_off(mocker):
+    mock_interface = MagicMock()
+    mock_interface.lastNetworkJoined.return_value = None
+
+    instance = darwin.DarwinNetworkStatus()
+    mocker.patch.object(instance, "_get_wifi_interface", return_value=mock_interface)
+
+    result = instance.is_network_metered()
+
+    assert result is False
+
+
 @pytest.mark.parametrize(
     'getpacket_output_name, expected',
     [
         ('normal_router', False),
-        ('phone', True),
+        ('android_phone', True),
     ],
 )
-def test_is_network_metered(getpacket_output_name, expected, monkeypatch):
+def test_is_network_metered_with_android(getpacket_output_name, expected, monkeypatch):
     def mock_getpacket(device):
         assert device == 'en0'
         return GETPACKET_OUTPUTS[getpacket_output_name]
 
     monkeypatch.setattr(darwin, 'call_ipconfig_getpacket', mock_getpacket)
 
-    result = darwin.is_network_metered('en0')
+    result = darwin.is_network_metered_with_android('en0')
     assert result == expected
 
 
+def test_get_known_wifi_networks_when_wifi_interface_exists(monkeypatch):
+    networksetup_output = """
+Preferred networks on en0:
+    Home Network
+    Coffee Shop Wifi
+    iPhone
+
+    Office Wifi
+    """
+    monkeypatch.setattr(
+        darwin, "call_networksetup_listpreferredwirelessnetworks", lambda interface_name: networksetup_output
+    )
+
+    network_status = darwin.DarwinNetworkStatus()
+    result = network_status.get_known_wifis()
+
+    assert len(result) == 4
+    assert result[0].ssid == "Home Network"
+
+
+def test_get_known_wifi_networks_when_no_wifi_interface(mocker):
+    instance = darwin.DarwinNetworkStatus()
+    mocker.patch.object(instance, "_get_wifi_interface", return_value=None)
+
+    results = instance.get_known_wifis()
+
+    assert results == []
+
+
 def test_get_network_devices(monkeypatch):
     monkeypatch.setattr(darwin, 'call_networksetup_listallhardwareports', lambda: NETWORKSETUP_OUTPUT)
 
@@ -55,7 +148,7 @@ def test_get_network_devices(monkeypatch):
 server_identifier (ip): 172.16.12.1
 end (none):
 """,
-    'phone': b"""\
+    'android_phone': b"""\
 op = BOOTREPLY
 htype = 1
 flags = 0