Special sauce to dismiss captive portal assistant on Apple devices.

[derekgraham]

Apple devices try to connect to a few URLs looking for a specific response to determine internet access and if behind captive portal. There are newer/better ways to do this now, but this is consistent with the special cases in place for other platforms. There are probably some others that could be added, but need to set up testing (windows mostly.

[derekgraham]

You should try this out - but now it should just let you connect to the Maslow network on iPhone or Mac. It won't show the captive portal loading thing. It should just let you go out to safari and load 192.168.0.1. My experience on iOS was it would all load into the captive portal and if you tried to switch to safari it would kick you off the maslow network. I'd like to look at the whole captive portal thing and see what else can be done. Normally, you would get a form to enter wireless network and password. more sophisicated is show a list of available networks, etc.

[davidelang]

Let's add the other browsers as well

Firefox http://detectportal.firefox.com/ (returns 'success'
chrome http://clients3.google.com/generate_204 (status 204)

[derekgraham]

Hi David - Do you know examples of where an OS is using these browser captive portal checks? On iOS and MacOS, when you connect to a wifi network, like the one offered by Maslow, it hits their captive portal check to see if there is internet connectivity. Unless you fake it out by sending the response they are expecting, the captive portal assistant will display any web content from that wifi network / ap in a limited, modal browser window (no javascript, and if you dismiss it, it will disconnect from the network).

The Firefox one would only matter if the OS was using that address - do you know if that is the case? (none of this appears to be documented...)

Is the google one used by chrome books or something?

The example I'm basing this off of is lots of reading on iOS dev resources about this problem which I assumed was unique to Apple devices since they do things a bit different...

Here's the list of what 'works' from a relatively current project. The downside is all the examples these days use a slightly newer, asynchronous version of the web server for esp32 so the code needs a little massaging. There is also a great captive portal project (wifimanager) that will display available wifi networks, etc.

// Required
server.on("/connecttest.txt", [](AsyncWebServerRequest *request) { request->redirect("http://logout.net"); });	// windows 11 captive portal workaround
server.on("/wpad.dat", [](AsyncWebServerRequest *request) { request->send(404); });								// Honestly don't understand what this is but a 404 stops win 10 keep calling this repeatedly and panicking the esp32 :)

// Background responses: Probably not all are Required, but some are. Others might speed things up?
// A Tier (commonly used by modern systems)
server.on("/generate_204", [](AsyncWebServerRequest *request) { request->redirect(localIPURL); });		   // android captive portal redirect
server.on("/redirect", [](AsyncWebServerRequest *request) { request->redirect(localIPURL); });			   // microsoft redirect
server.on("/hotspot-detect.html", [](AsyncWebServerRequest *request) { request->redirect(localIPURL); });  // apple call home
server.on("/canonical.html", [](AsyncWebServerRequest *request) { request->redirect(localIPURL); });	   // firefox captive portal call home
server.on("/success.txt", [](AsyncWebServerRequest *request) { request->send(200); });					   // firefox captive portal call home
server.on("/ncsi.txt", [](AsyncWebServerRequest *request) { request->redirect(localIPURL); });			   // windows call home

// B Tier (uncommon)
//  server.on("/chrome-variations/seed",[](AsyncWebServerRequest *request){request->send(200);}); //chrome captive portal call home
//  server.on("/service/update2/json",[](AsyncWebServerRequest *request){request->send(200);}); //firefox?
//  server.on("/chat",[](AsyncWebServerRequest *request){request->send(404);}); //No stop asking Whatsapp, there is no internet connection
//  server.on("/startpage",[](AsyncWebServerRequest *request){request->redirect(localIPURL);});

I'm happy to put any/all of these in but would be useful to have some failure/test scenarios first. I'll try it soon on a windows 10 machine I have at my shop (if it has wifi) and see what happens on a kindle tablet maybe, but I don't have any android/others to try. I've assumed they do a 'better' jobs already since I haven't seen any complaints...

Anyway, longer term I think there is a new standard that involves DNS or DHCP and a JSON api that I'd like to look into, but above my pay grade at the moment. For me right now, this fixes so Maslow can be connected to by iPhone, then go to browser and open the FluicNC/Maslow web ui. I'm working on making the buttons fit next...

Long term I'd like to see a 'proper' captive portal for Maslow - Welcome, Logo and option to connect, or enter your wifi + password credentials and store those away and restart in client mode. Gravy would be pick your available networks from a list, etc.

Do you have any Windows 9x-10/11, Linux? / ChromeOS / etc. captive portal examples that are as blocking as iOS current behavior?

[davidelang]

Chrome and Firefox do this in the browser at startup, on any OS. It's not a matter of the OS doing the detection with chrome or firefox. the kindle tablet is android. I'm not thrilled with the browsers implementing this, but I guess it's better than my 100s of tabs all becoming the portal (possibly) David Lang

…

On Tue, 30 Apr 2024, Derek Graham wrote: Date: Tue, 30 Apr 2024 18:07:43 -0700 From: Derek Graham ***@***.***> Reply-To: BarbourSmith/FluidNC ***@***.***> To: BarbourSmith/FluidNC ***@***.***> Cc: David Lang ***@***.***>, Comment ***@***.***> Subject: Re: [BarbourSmith/FluidNC] Special sauce to dismiss captive portal assistant on Apple devices. (PR #109) Hi David - Do you know examples of where an OS is using these browser captive portal checks? On iOS and MacOS, when you connect to a wifi network, like the one offered by Maslow, it hits their captive portal check to see if there is internet connectivity. Unless you fake it out by sending the response they are expecting, the captive portal assistant will display any web content from that wifi network / ap in a limited, modal browser window (no javascript, and if you dismiss it, it will disconnect from the network). The Firefox one would only matter if the OS was using that address - do you know if that is the case? (none of this appears to be documented...) Is the google one used by chrome books or something? The example I'm basing this off of is lots of reading on iOS dev resources about this problem which I assumed was unique to Apple devices since they do things a bit different... Here's the list of what 'works' from a relatively current project. The downside is all the examples these days use a slightly newer, asynchronous version of the web server for esp32 so the code needs a little massaging. There is also a great captive portal project (wifimanager) that will display available wifi networks, etc. // Required server.on("/connecttest.txt", [](AsyncWebServerRequest *request) { request->redirect("http://logout.net"); }); // windows 11 captive portal workaround server.on("/wpad.dat", [](AsyncWebServerRequest *request) { request->send(404); }); // Honestly don't understand what this is but a 404 stops win 10 keep calling this repeatedly and panicking the esp32 :) // Background responses: Probably not all are Required, but some are. Others might speed things up? // A Tier (commonly used by modern systems) server.on("/generate_204", [](AsyncWebServerRequest *request) { request->redirect(localIPURL); }); // android captive portal redirect server.on("/redirect", [](AsyncWebServerRequest *request) { request->redirect(localIPURL); }); // microsoft redirect server.on("/hotspot-detect.html", [](AsyncWebServerRequest *request) { request->redirect(localIPURL); }); // apple call home server.on("/canonical.html", [](AsyncWebServerRequest *request) { request->redirect(localIPURL); }); // firefox captive portal call home server.on("/success.txt", [](AsyncWebServerRequest *request) { request->send(200); }); // firefox captive portal call home server.on("/ncsi.txt", [](AsyncWebServerRequest *request) { request->redirect(localIPURL); }); // windows call home // B Tier (uncommon) // server.on("/chrome-variations/seed",[](AsyncWebServerRequest *request){request->send(200);}); //chrome captive portal call home // server.on("/service/update2/json",[](AsyncWebServerRequest *request){request->send(200);}); //firefox? // server.on("/chat",[](AsyncWebServerRequest *request){request->send(404);}); //No stop asking Whatsapp, there is no internet connection // server.on("/startpage",[](AsyncWebServerRequest *request){request->redirect(localIPURL);}); I'm happy to put any/all of these in but would be useful to have some failure/test scenarios first. I'll try it soon on a windows 10 machine I have at my shop (if it has wifi) and see what happens on a kindle tablet maybe, but I don't have any android/others to try. I've assumed they do a 'better' jobs already since I haven't seen any complaints... Anyway, longer term I think there is a new standard that involves DNS or DHCP and a JSON api that I'd like to look into, but above my pay grade at the moment. For me right now, this fixes so Maslow can be connected to by iPhone, then go to browser and open the FluicNC/Maslow web ui. I'm working on making the buttons fit next... Long term I'd like to see a 'proper' captive portal for Maslow - Welcome, Logo and option to connect, or enter your wifi + password credentials and store those away and restart in client mode. Gravy would be pick your available networks from a list, etc. Do you have any Windows 9x-10/11, Linux? / ChromeOS / etc. captive portal examples that are as blocking as iOS current behavior?

[derekgraham]

Interesting. This is what I see on FireFox on my Mac. First I connect to the Maslow wifi, which now just connects (in wifi settings).

Then I open Firefox.
It displays a little notice bar across the top that I have to log in to the network to connect. If I click, I get the Maslow captive portal, then it connects and loads. The bar stays there with new text, but you can just close it.

If I type the 192.168.0.1 address, it just loads the Maslow page (also with the bar still showing).

Chrome just gives me a similar error bar telling my it's not my default browser, then pops up something different saying it can't update because its not connected to the internet. If I type 192.168.0.1, it just loads the Maslow controls.

Maybe chrome is using one of the other two things in the code already:
_webserver->on("/generate_204", HTTP_ANY, handle_root);
_webserver->on("/gconnectivitycheck.gstatic.com", HTTP_ANY, handle_root);
//do not forget the / at the end
_webserver->on("/fwlink/", HTTP_ANY, handle_root);

the generate_204 is referenced as an android thing, but it might be using that, or the gstatic is a google endpoint.
I wonder how many end points they check, legacy or not.

I'll play around with removing the existing ones to see which one is for Chrome.

I'll see what happens with old kindle tablet and windows 10 at least. All Janky AF. I don't even think the example above could work because it doesn't send "success", just redirects... Well, I wanted to dig into this anyway!

[davidelang]

I think this is a really good idea. I tend to go into my browser settings and disable captive portal detection.

I also would love to make the dhcp address that the maslow provides when it's an AP NOT include a default route (I think this is to not include option 3 for the router) and possibly also not include option 6 (dns) so that the user can have a machine connected to a wired network, and remain connected to the wired network while it's connected to the maslow as well.

not advertising a router should always work, not advertising dns may be something we want to switch on and off.

[derekgraham]

This is interesting, I assumed being on a Mac/iOS that it was more of an Apple issue based on them being dragged on in more of the comments I've been reading through. However, I was looking at WifiManager to see what they do, and figured out they don't, because they are a reasonable captive portal solution - i.e. it is displaying option to enter network/password and provide the list of available networks, etc. duh.

Then I ran across this reference in a pull request:
tzapu/WiFiManager#1637

Which basically says that Windows does so many different requests that it was crashing the device (such as the wpad.dat referred to in the code above that I hadn't seen elsewhere)... Fascinating.

So I'll take this down to the shop today and see if my windows machine has wifi, and add some of these in and at least have some behavior to double-check.

[BarbourSmith]

I don't know how I missed this PR! Somehow I wasn't getting notifications about it...weird! I'm sorry it took me so long to respond. I'll test it right now!

[BarbourSmith]

It looks great to me. I'm not getting the captive portal popup on iphone anymore which is a little bit of a bummer, but totally worth it to have the connection be that much more stable. Awesome!