Cellular change only: minor fixes for SARA-U201.

SARA-U201 is no longer sold by u-blox, however there are customers still using it and its descendants (SARA-U260).  It was not tested in the ubxlib test farm for a few years because the local cellular networks would reject a device that connected/disconnected frequently (something we do during testing); however, we now have SIM cards which work on local networks that do not throw a strop, and hence SARA-U201 can be tested once more.

This commit contains a few tweaks to the core code:
- move an AT timeout setting that was in the wrong place, around the sending of AT+COPS=0; it just happened to work with all other modules, not so SARA-U201,
- the "I've found no networks" response to AT+COPS=?, which SARA-U201 sometimes returns, is 13 characters long, not 12,
- there was a hole in the logic, specific to SARA-U201, where it would not coalesce the RATs back to single RAT mode when duplicate RATs appeared due to a uCellCfgSetRatRank().

There are also a few tweaks related to testing:
- add the ability to not run the security tests, since they use EC encryption which SARA-U201 does not support,
- allow the default APN, used when pApn is set to NULL, to be overridden by the test system to something other than "internet" in order that the examples (which set the APN to NULL) may pick up an APN that will work in the context of the test system; networks just seem to sort it out for LTE, not for 2G or 3G though.

Test: none

cell/api/u_cell_cfg.h

@@ -380,14 +380,14 @@ int32_t uCellCfgGetUdconf(uDeviceHandle_t cellHandle, int32_t param1,
  *  @param fsRestoreType the file system factory restore type. Valid options
  *                       are 0, 1 and 2.
  *                       0: no factory restore.
- *                       1: Check datasheet, if this option is
+ *                       1: check datasheet, if this option is
  *                          supported by your module.
  *                       2: all files stored in FS deleted.
  * @param nvmRestoreType the file system factory restore type. Valid options
  *                       are 0, 1 and 2.
  *                       0: no factory restore.
  *                       1: NVM flash sectors erased.
- *                       2: Check datasheet, if this option is
+ *                       2: check datasheet, if this option is
  *                          supported by your module.
  * @return               zero on success or negative error code on
  *                       failure.
@@ -406,6 +406,9 @@ int32_t uCellCfgFactoryReset(uDeviceHandle_t cellHandle, int32_t fsRestoreType,
  * with a call to uCellCfgSetAutoBaudOff() in the case of SARA-R5
  * and SARA-U201.
  *
+ * Note: DTR must be grounded in order for a greeting message to be
+ * emitted.
+ *
  * @param cellHandle   the handle of the cellular instance.
  * @param[in] pStr     the null-terminated greeting message; use NULL
  *                     to remove an existing greeting message.
@@ -423,6 +426,9 @@ int32_t uCellCfgSetGreeting(uDeviceHandle_t cellHandle, const char *pStr);
  * pin has been set using uCellPwrSetDtrPowerSavingPin()) then the
  * greeting message is NOT emitted by the module at a reboot.
  *
+ * Note: DTR must be grounded in order for a greeting message to be
+ * emitted.
+ *
  * Obviously for this to be useful it is important that the greeting
  * message is unique; you may consider using #U_CELL_CFG_GREETING.
  *

cell/api/u_cell_net.h

@@ -770,6 +770,9 @@ uCellNetStatus_t uCellNetGetNetworkStatus(uDeviceHandle_t cellHandle,
  * service then this function will likely return the reason for
  * that denial (e.g. 11 for "PLMN not allowed"), rather than zero.
  *
+ * Note: SARA-U201 always returns error 148, "SM activation error",
+ * even after a connection has succeeded.
+ *
  * @param cellHandle  the handle of the cellular instance.
  * @return            on success the last EMM cause from the network,
  *                    see appendix A.3 of the AT commands manual,

cell/src/u_cell_apn_db.h

@@ -42,6 +42,12 @@ extern "C" {
  * COMPILE-TIME MACROS
  * -------------------------------------------------------------- */
 
+#ifndef U_CELL_CFG_APN_DEFAULT
+/** A default APN to be applied when none is specified.
+ */
+# define U_CELL_CFG_APN_DEFAULT internet
+#endif
+
 /** Helper to generate the APN string.
  */
 //lint -emacro((786), _APN) Suppress use of null
@@ -71,7 +77,7 @@ typedef struct {
 
 /** Default APN settings used by many networks.
  */
-static const char *const pApnDefault = _APN("internet",,);
+static const char *const pApnDefault = _APN(U_PORT_STRINGIFY_QUOTED(U_CELL_CFG_APN_DEFAULT),,);
 
 /** List of special APNs for different network operators.
  *

cell/src/u_cell_cfg.c

@@ -481,7 +481,7 @@ static int32_t getRatRankSaraU2(uCellPrivateInstance_t *pInstance,
         errorCodeOrRank = (int32_t) U_CELL_ERROR_NOT_FOUND;
         // If the first mode is 1, dual mode, then there MUST be a second
         // number which indicates the preference
-        // If the RAT being asked for is 2G or 3G then if it in this
+        // If the RAT being asked for is 2G or 3G then if it is in this
         // second number it is at rank 0, else it must by implication
         // be at rank 1
         if ((rat == U_CELL_NET_RAT_GSM_GPRS_EGPRS) || (rat == U_CELL_NET_RAT_UTRAN)) {
@@ -583,30 +583,48 @@ static int32_t setRatRankSaraU2(uCellPrivateInstance_t *pInstance,
         if ((modes[0] >= 0) && (modes[1] >= 0)) {
             // ...and we already have dual mode...
             if (rank == 0) {
-                // ...and we are setting the first rank,
-                // then set the preference in the second number
-                modes[1] = cellRatToModuleRat(pInstance->pModule->moduleType, rat);
-                validOperation = true;
-            } else if (rank == 1) {
-                // ...otherwise if we are setting the second
-                // rank then we want to set the OPPOSITE of
-                // the desired RAT in the second number.
-                // In other words, to put 2G at rank 1, we
-                // need to set 3G as our preferred RAT.
-                if (rat == U_CELL_NET_RAT_GSM_GPRS_EGPRS) {
-                    modes[1] = cellRatToModuleRat(pInstance->pModule->moduleType,
-                                                  U_CELL_NET_RAT_UTRAN);
+                // ...and we are setting the top rank...
+                if (rat == uCellPrivateModuleRatToCellRat(pInstance->pModule->moduleType, modes[1])) {
+                    // ...then if we are setting the RAT to the
+                    // same as the current dual-mode preferred RAT,
+                    // switch to single-mode and that RAT.
+                    modes[0] = cellRatToModuleRat(pInstance->pModule->moduleType, rat);
+                    modes[1] = -1;
                     validOperation = true;
-                } else if (rat == U_CELL_NET_RAT_UTRAN) {
-                    modes[1] = cellRatToModuleRat(pInstance->pModule->moduleType,
-                                                  U_CELL_NET_RAT_GSM_GPRS_EGPRS);
+                } else {
+                    // ...else set the preference in the second number.
+                    modes[1] = cellRatToModuleRat(pInstance->pModule->moduleType, rat);
+                    validOperation = true;
+                }
+            } else if (rank == 1) {
+                // ...and we are setting the second rank...
+                if (rat == uCellPrivateModuleRatToCellRat(pInstance->pModule->moduleType, modes[1])) {
+                    // ...and the RAT we are setting is also the first rank,
+                    // then switch to single mode with that RAT.
+                    modes[0] = cellRatToModuleRat(pInstance->pModule->moduleType, rat);
+                    modes[1] = -1;
                     validOperation = true;
+                } else {
+                    // ...otherwise if we are setting the second
+                    // rank then we want to set the OPPOSITE of
+                    // the desired RAT in the preferred RAT position.
+                    // In other words, to put 2G in second rank, we
+                    // need to set 3G as our preferred RAT.
+                    if (rat == U_CELL_NET_RAT_GSM_GPRS_EGPRS) {
+                        modes[1] = cellRatToModuleRat(pInstance->pModule->moduleType,
+                                                      U_CELL_NET_RAT_UTRAN);
+                        validOperation = true;
+                    } else if (rat == U_CELL_NET_RAT_UTRAN) {
+                        modes[1] = cellRatToModuleRat(pInstance->pModule->moduleType,
+                                                      U_CELL_NET_RAT_GSM_GPRS_EGPRS);
+                        validOperation = true;
+                    }
                 }
             }
         } else if ((modes[0] >= 0) && (modes[1] < 0)) {
             // ...and we are in single mode...
             if (rank == 0) {
-                // ...then if we are setting rank 0 just set it
+                // ...then if we are setting rank 0 just set it.
                 modes[0] = cellRatToModuleRat(pInstance->pModule->moduleType, rat);
                 validOperation = true;
             } else if (rank == 1) {
@@ -642,7 +660,7 @@ static int32_t setRatRankSaraU2(uCellPrivateInstance_t *pInstance,
                 // If are removing the top-most rank
                 // then we set the single mode to be
                 // the opposite of the currently
-                // preferred RAT
+                // preferred RAT.
                 if (uCellPrivateModuleRatToCellRat(pInstance->pModule->moduleType, modes[1]) ==
                     U_CELL_NET_RAT_GSM_GPRS_EGPRS) {
                     modes[0] = cellRatToModuleRat(pInstance->pModule->moduleType,

cell/src/u_cell_net.c

@@ -875,9 +875,9 @@ static int32_t setAutomaticMode(const uCellPrivateInstance_t *pInstance)
         // has actually accepted the command,
         // despite what it says
         uAtClientLock(atHandle);
-        uAtClientTimeoutSet(atHandle, 1000);
         uAtClientCommandStart(atHandle, "AT+COPS=0");
         uAtClientCommandStop(atHandle);
+        uAtClientTimeoutSet(atHandle, 1000);
         x = -1;
         while ((x != 0) && keepGoingLocalCb(pInstance) &&
                (deviceError.type ==
@@ -935,7 +935,7 @@ static int32_t storeNextScanItem(uCellPrivateInstance_t *pInstance,
     // (<stat>,<long_name>,<short_name>,<numeric>[,<AcT>]
     // However, there can be gunk on the end of the AT+COPS=?
     // response string, for instance the "test" response:
-    // ,(0-6),(0-2)
+    // ,,(0-6),(0-2)
     // ...may appear there, so check for errors;
     // the <stat> and <numeric> fields must be present, the
     // rest could be absent or zero length strings.
@@ -2968,10 +2968,10 @@ int32_t uCellNetScanGetFirst(uDeviceHandle_t cellHandle,
                 // "test" response to the AT+COPS=? command,
                 // i.e.: +COPS: ,,(0-6),(0-2)
                 // If we get the "test" response instead
-                // readBytes will be 12 whereas for the
+                // readBytes will be 13 whereas for the
                 // intended response of:
                 // (<stat>,<long_name>,<short_name>,<numeric>[,<AcT>])
-                // it will be at longer than that hence we set
+                // it will be longer than that hence we set
                 // a threshold for readBytes of > 12 characters.
                 pInstance->startTimeMs = uPortGetTickTimeMs();
                 for (size_t x = U_CELL_NET_SCAN_RETRIES + 1;
@@ -3026,7 +3026,7 @@ int32_t uCellNetScanGetFirst(uDeviceHandle_t cellHandle,
                         // "test" response or a device error
                         gotAnswer = true;
                     }
-                    if (bytesRead > 12) {
+                    if (bytesRead > 13) {
                         // Got a real answer: process it in
                         // chunks delimited by ")"
                         for (pStr = strtok_r(pBuffer, ")", &pSaved);

cell/src/u_cell_ppp.c

@@ -312,6 +312,9 @@ static void closePpp(uCellPrivateInstance_t *pInstance,
                 // and it was us who started it
                 uCellMuxPrivateDisable(pInstance);
             }
+            // Some modules (e.g. SARA-U201) need a little rest just here
+            // or the next AT command may stall
+            uPortTaskBlock(100);
             // Re-enable UART sleep if we had switched it off
             if (pContext->uartSleepWakeOnDataWasEnabled) {
                 uCellPwrPrivateEnableUartSleep(pInstance);

cell/src/u_cell_private.c

@@ -113,12 +113,11 @@ const uCellPrivateModule_t gUCellPrivateModuleList[] = {
          (1ULL << (int32_t) U_CELL_PRIVATE_FEATURE_CTS_CONTROL)                   |
          (1ULL << (int32_t) U_CELL_PRIVATE_FEATURE_SOCK_SET_LOCAL_PORT)           |
          (1ULL << (int32_t) U_CELL_PRIVATE_FEATURE_UART_POWER_SAVING)             |
-         // CMUX is supported here but we do not test it hence it is not marked as supported
+         (1ULL << (int32_t) U_CELL_PRIVATE_FEATURE_CMUX)                          |
          (1ULL << (int32_t) U_CELL_PRIVATE_FEATURE_UCGED)                         |
          (1ULL << (int32_t) U_CELL_PRIVATE_FEATURE_AUTHENTICATION_MODE_AUTOMATIC) |
-         (1ULL << (int32_t) U_CELL_PRIVATE_FEATURE_HTTP) /* features */
-         // PPP is supported by the module but we do not test its integration with
-         // ubxlib and hence it is not marked as supported
+         (1ULL << (int32_t) U_CELL_PRIVATE_FEATURE_HTTP)                          |
+         (1ULL << (int32_t) U_CELL_PRIVATE_FEATURE_PPP) /* features */
         ),
         6, /* Default CMUX channel for GNSS */
         15 /* AT+CFUN reboot command */

cell/src/u_cell_pwr.c

@@ -1063,12 +1063,15 @@ static int32_t moduleConfigure(uCellPrivateInstance_t *pInstance,
                                        UUPSMR_urc, pInstance);
             }
         }
-        // Update the sleep parameters; note that we ask for the
-        // requested 3GPP power saving state here, rather than the
-        // assigned, since it might not be assigned by the network
-        // at this point but can come along later
-        uCellPwrPrivateGet3gppPowerSaving(pInstance, false, NULL, NULL, NULL);
-        uCellPrivateSetDeepSleepState(pInstance);
+        if (U_CELL_PRIVATE_HAS(pInstance->pModule,
+                               U_CELL_PRIVATE_FEATURE_3GPP_POWER_SAVING)) {
+            // Update the sleep parameters; note that we ask for the
+            // requested 3GPP power saving state here, rather than the
+            // assigned, since it might not be assigned by the network
+            // at this point but can come along later
+            uCellPwrPrivateGet3gppPowerSaving(pInstance, false, NULL, NULL, NULL);
+            uCellPrivateSetDeepSleepState(pInstance);
+        }
         if (success &&
             U_CELL_PRIVATE_MODULE_IS_SARA_R4(pInstance->pModule->moduleType)) {
             // For SARA-R4, whether the E-DRX URC is on or not does not

common/security/test/u_security_tls_test.c

@@ -22,9 +22,11 @@
 
 /** @file
  * @brief Test for the u-blox TLS security API: these should pass on all
- * platforms.
+ * platforms that support transport security.
  */
 
+#ifndef U_CFG_TEST_TRANSPORT_SECURITY_DISABLE
+
 #ifdef U_CFG_OVERRIDE
 # include "u_cfg_override.h" // For a customer's configuration override
 #endif
@@ -695,4 +697,6 @@ U_PORT_TEST_FUNCTION("[securityTls]", "securityTlsCleanUp")
     uTestUtilResourceCheck(U_TEST_PREFIX, NULL, true);
 }
 
+#endif // #ifndef U_CFG_TEST_TRANSPORT_SECURITY_DISABLE
+
 // End of file

example/sockets/main_dtls.c

@@ -82,7 +82,7 @@
 // for the module is likely different to that of the MCU: check
 // the data sheet for the module to determine the mapping.
 
-#ifdef U_CFG_TEST_CELL_MODULE_TYPE
+#if defined(U_CFG_TEST_CELL_MODULE_TYPE) && !defined(U_CFG_TEST_TRANSPORT_SECURITY_DISABLE)
 // DEVICE i.e. module/chip configuration: in this case a cellular
 // module connected via UART
 static const uDeviceCfg_t gDeviceCfg = {
@@ -422,7 +422,7 @@ U_PORT_TEST_FUNCTION("[example]", "exampleSocketsDtls")
 
     uPortLog("Done.\n");
 
-#ifdef U_CFG_TEST_CELL_MODULE_TYPE
+#if defined(U_CFG_TEST_CELL_MODULE_TYPE) && !defined(U_CFG_TEST_TRANSPORT_SECURITY_DISABLE)
     // For u-blox internal testing only
     EXAMPLE_FINAL_STATE((txSize == 0) && (rxSize == sizeof(message)));
 #endif

example/sockets/main_tls.c

@@ -82,7 +82,7 @@
 // for the module is likely different to that of the MCU: check
 // the data sheet for the module to determine the mapping.
 
-#ifdef U_CFG_TEST_CELL_MODULE_TYPE
+#if defined(U_CFG_TEST_CELL_MODULE_TYPE) && !defined(U_CFG_TEST_TRANSPORT_SECURITY_DISABLE)
 // DEVICE i.e. module/chip configuration: in this case a cellular
 // module connected via UART
 static const uDeviceCfg_t gDeviceCfg = {
@@ -422,7 +422,7 @@ U_PORT_TEST_FUNCTION("[example]", "exampleSocketsTls")
 
     uPortLog("Done.\n");
 
-#ifdef U_CFG_TEST_CELL_MODULE_TYPE
+#if defined(U_CFG_TEST_CELL_MODULE_TYPE) && !defined(U_CFG_TEST_TRANSPORT_SECURITY_DISABLE)
     // For u-blox internal testing only
     EXAMPLE_FINAL_STATE((txSize == 0) && (rxSize == sizeof(message)));
 #endif

port/platform/common/automation/SETUP.md

@@ -628,7 +628,7 @@ If the server has no DNS entry on the public internet (required for some tests)
 ### Cellular And Short Range Network Test Peers
 A Nutaq cellular network box is required for cellular Cat-M1 coverage; set up of this is out of scope of this document: provided an RF link gets to the relevant test instances and the Nutaq has public internet access, that is all that is required.  To be clear, the Nutaq box does _not_ have to be on the same network as the `ubxlib` test system (though it can be if desired).
 
-Some cellular test instances (e.g. instance 25 and 29) may use the live network for their cellular test peer, rather than the Nutaq box (e.g. because the RAT that they use is not supported by the Nutaq box, which is the case for 2G, 3G, LTE non-Cat-M1 and LTE non-NB1).  Some live networks (e.g. O2/Telefonica in the UK) apply extremely annoying attach/detach rate limitations, and a UE may be banned for a attaching/detaching too often, certainly the case for one running our test regime.  To stop this buggering everything up, you should initially control the module in question directly, do a manual selection of a network which does not apply attach/detach rate limitations (e.g. Vodafone or 3 in the UK, where `AT+COPS=1,2,"23415` would, for example, select Vodafone in the UK), then switch back to automatic mode (`AT+COPS=0`), and maybe also add the same network to the preferred list (e.g. `AT+CPOL=1,2,"23415",1,1,1,1` would add Vodafone UK to the top of the list): this should hopefully stick the module to that network.
+Some cellular test instances (e.g. instance 25 and 29) may use the live network for their cellular test peer, rather than the Nutaq box (e.g. because the RAT that they use is not supported by the Nutaq box, which is the case for 2G, 3G, LTE non-Cat-M1 and LTE non-NB1).  Some live networks (e.g. O2/Telefonica in the UK) apply extremely annoying attach/detach rate limitations, and a UE may be banned for a attaching/detaching too often, certainly the case for one running our test regime.  To stop this buggering everything up, you should initially control the module in question directly, do a manual selection of a network which does not apply attach/detach rate limitations (e.g. Vodafone or 3 in the UK, where `AT+COPS=1,2,"23415"` would, for example, select Vodafone in the UK), then switch back to automatic mode (`AT+COPS=0`), and maybe also add the same network to the preferred list (e.g. `AT+CPOL=1,2,"23415",1,1,1,1` would add Vodafone UK to the top of the list): this should hopefully stick the module to that network.
 
 Some short-range test instances require BLE test peers; these just need to be [configured](https://wiki.u-blox.com/bin/view/ShortRange/NewPlatforms), MAC addressses in [DATABASE.md](DATABASE.md) and then plugged into power from the shared resource Ethernet-based relay boxes so that they are powered up at the start of testing and powered down again afterwards.