Add ability to specify add-on store metadata URL from within NVDA (#1…

…7099)

Closes #14974

Summary of the issue:
The add-on store metadata URL is currently hard-coded in NVDA. This presents problems for users who are unable to use the default URL for any reason. This particularly affects users in the People's Republic of China, many of whom are unable to access the add-on store at an acceptable speed. A community add-on exists to work around this, but a means of specifying a mirror directly within NVDA is considered a better approach.

Description of user facing changes
Added a text box in Add-on store settings that allows users to specify the URL to use for the add-on store. Added slightly more helpful wording to add-on store errors that encourages users to check the metadata URL if there are issues accessing the add-on store and a custom URL is in use.

Description of development approach
Added a config key, `addonStore.baseURL`, to hold the URL of the server to contact for add-on store data. Added a control to the settings dialog that is populated by and populates this config item. Removed `addonStore.BASE_URL` in favour of a private constant which holds the default base URL. Added a private function, `addonStore.network._getBaseURL`, which gets the custom URL if one is set, or the default otherwise. Modified `addonStore.dataManager`, refactoring the `DisplayableError` code into a helper function and adding some troubleshooting steps to the messages that NVDA shows when showing errors.

Testing strategy:
Tested accessing the add-on store with no mirror set, with an invalid mirror set, and with no internet access.

Known issues with pull request:
The settings dialog makes a best effort at normalizing the URL. However, URLs with incompatible protocols or other issues that will cause them to fail with the Add-on Store are not checked for. Users will see a message asking them to check the mirror URL if contacting the store fails and a mirror is in use.

requirements.txt

@@ -7,6 +7,7 @@ pyserial==3.5
 wxPython @ https://github.com/nvaccess/nvda-misc-deps/raw/51ae7db821d1d5166ab0c030fe20ec72dd7a2ad9/python/wxPython-4.2.2a1-cp311-cp311-win32.whl
 configobj @ git+https://github.com/DiffSK/configobj@e2ba4457c4651fa54f8d59d8dcdd3da950e956b8#egg=configobj
 requests==2.32.0
+url-normalize==1.4.3
 schedule==1.2.1
 -c constraints.txt
 

source/addonStore/dataManager.py

@@ -207,6 +207,18 @@ def _getCachedAddonData(self, cacheFilePath: str) -> Optional[CachedAddonsModel]
 
 	# Translators: A title of the dialog shown when fetching add-on data from the store fails
 	_updateFailureMessage = pgettext("addonStore", "Add-on data update failure")
+	_updateFailureMirrorSuggestion = pgettext(
+		"addonStore",
+		# Translators: A suggestion of what to do when fetching add-on data from the store fails and a metadata mirror is being used.
+		# {url} will be replaced with the mirror URL.
+		"Make sure you are connected to the internet, and the Add-on Store mirror URL is valid.\n"
+		"Mirror URL: {url}",
+	)
+	_updateFailureDefaultSuggestion = pgettext(
+		"addonStore",
+		# Translators: A suggestion of what to do when fetching add-on data from the store fails and the default metadata URL is being used.
+		"Make sure you are connected to the internet and try again.",
+	)
 
 	def getLatestCompatibleAddons(
 		self,
@@ -234,15 +246,12 @@ def getLatestCompatibleAddons(
 					cachedLanguage=self._lang,
 					nvdaAPIVersion=addonAPIVersion.CURRENT,
 				)
-			elif onDisplayableError is not None:
-				from gui.message import DisplayableError
-
-				displayableError = DisplayableError(
+			else:
+				self._do_displayError(
+					onDisplayableError,
 					# Translators: A message shown when fetching add-on data from the store fails
 					pgettext("addonStore", "Unable to fetch latest add-on data for compatible add-ons."),
-					self._updateFailureMessage,
 				)
-				callLater(delay=0, callable=onDisplayableError.notify, displayableError=displayableError)
 
 		if self._compatibleAddonCache is None:
 			return _createAddonGUICollection()
@@ -273,20 +282,44 @@ def getLatestAddons(
 					cachedLanguage=self._lang,
 					nvdaAPIVersion=_LATEST_API_VER,
 				)
-			elif onDisplayableError is not None:
-				from gui.message import DisplayableError
-
-				displayableError = DisplayableError(
+			else:
+				self._do_displayError(
+					onDisplayableError,
 					# Translators: A message shown when fetching add-on data from the store fails
 					pgettext("addonStore", "Unable to fetch latest add-on data for incompatible add-ons."),
-					self._updateFailureMessage,
 				)
-				callLater(delay=0, callable=onDisplayableError.notify, displayableError=displayableError)
 
 		if self._latestAddonCache is None:
 			return _createAddonGUICollection()
 		return deepcopy(self._latestAddonCache.cachedAddonData)
 
+	def _do_displayError(
+		self,
+		onDisplayableError: "DisplayableError.OnDisplayableErrorT | None",
+		displayMessage: str,
+		titleMessage: str | None = None,
+	):
+		"""Display a DisplayableMessage if an OnDisplayableError action is given.
+
+		See gui.message.DisplayableError for further information.
+
+		:param onDisplayableError: The displayable error action.
+		:param displayMessage: Body of the displayable error.
+		:param titleMessage: Title of the displayable error. If None, _updateFailureMessage will be used. Defaults to None.
+		"""
+		if onDisplayableError is None:
+			return
+		from gui.message import DisplayableError
+
+		tip = (
+			self._updateFailureMirrorSuggestion.format(url=url)
+			if (url := config.conf["addonStore"]["baseServerURL"])
+			else self._updateFailureDefaultSuggestion
+		)
+		displayMessage = f"{displayMessage}\n{tip}"
+		displayableError = DisplayableError(displayMessage, titleMessage or self._updateFailureMessage)
+		callLater(delay=0, callable=onDisplayableError.notify, displayableError=displayableError)
+
 	def _deleteCacheInstalledAddon(self, addonId: str):
 		addonCachePath = os.path.join(self._installedAddonDataCacheDir, f"{addonId}.json")
 		if pathlib.Path(addonCachePath).exists():

source/addonStore/network.py

@@ -27,6 +27,7 @@
 import NVDAState
 from NVDAState import WritePaths
 from utils.security import sha256_checksum
+from config import conf
 
 from .models.addon import (
 	_AddonGUIModel,
@@ -40,25 +41,31 @@
 	from gui.addonStoreGui.viewModels.addonList import AddonListItemVM
 
 
-BASE_URL = "https://nvaccess.org/addonStore"
+_DEFAULT_BASE_URL = "https://nvaccess.org/addonStore"
 _LATEST_API_VER = "latest"
 """
 A string value used in the add-on store to fetch the latest version of all add-ons,
 i.e include older incompatible versions.
 """
 
 
+def _getBaseURL() -> str:
+	if url := conf["addonStore"]["baseServerURL"]:
+		return url
+	return _DEFAULT_BASE_URL
+
+
 def _getCurrentApiVersionForURL() -> str:
 	year, major, minor = addonAPIVersion.CURRENT
 	return f"{year}.{major}.{minor}"
 
 
 def _getAddonStoreURL(channel: Channel, lang: str, nvdaApiVersion: str) -> str:
-	return f"{BASE_URL}/{lang}/{channel.value}/{nvdaApiVersion}.json"
+	return f"{_getBaseURL()}/{lang}/{channel.value}/{nvdaApiVersion}.json"
 
 
 def _getCacheHashURL() -> str:
-	return f"{BASE_URL}/cacheHash.json"
+	return f"{_getBaseURL()}/cacheHash.json"
 
 
 class AddonFileDownloader:

source/config/configSpec.py

@@ -338,6 +338,7 @@
 [addonStore]
 	showWarning = boolean(default=true)
 	automaticUpdates = option("notify", "disabled", default="notify")
+	baseServerURL = string(default="")
 """
 
 #: The configuration specification

source/gui/settingsDialogs.py

@@ -62,6 +62,7 @@
 	Set,
 	cast,
 )
+from url_normalize import url_normalize
 import core
 import keyboardHandler
 import characterProcessing
@@ -3184,10 +3185,27 @@ def makeSettings(self, settingsSizer: wx.BoxSizer) -> None:
 		index = [x.value for x in AddonsAutomaticUpdate].index(config.conf["addonStore"]["automaticUpdates"])
 		self.automaticUpdatesComboBox.SetSelection(index)
 
+		# Translators: This is the label for a text box in the add-on store settings dialog.
+		self.addonMetadataMirrorLabelText = _("Server &mirror URL")
+		self.addonMetadataMirrorTextbox = sHelper.addLabeledControl(
+			self.addonMetadataMirrorLabelText,
+			wx.TextCtrl,
+		)
+		self.addonMetadataMirrorTextbox.SetValue(config.conf["addonStore"]["baseServerURL"])
+		self.bindHelpEvent("AddonStoreMetadataMirror", self.addonMetadataMirrorTextbox)
+
+	def isValid(self) -> bool:
+		self.addonMetadataMirrorTextbox.SetValue(
+			url_normalize(self.addonMetadataMirrorTextbox.GetValue().strip()).rstrip("/"),
+		)
+		return True
+
 	def onSave(self):
 		index = self.automaticUpdatesComboBox.GetSelection()
 		config.conf["addonStore"]["automaticUpdates"] = [x.value for x in AddonsAutomaticUpdate][index]
 
+		config.conf["addonStore"]["baseServerURL"] = self.addonMetadataMirrorTextbox.Value.strip().rstrip("/")
+
 
 class TouchInteractionPanel(SettingsPanel):
 	# Translators: This is the label for the touch interaction settings panel.

user_docs/en/changes.md

@@ -11,6 +11,7 @@ In order to use this feature, the application volume adjuster needs to be enable
 
 * When editing in Microsoft PowerPoint text boxes, you can now move per sentence with `alt+upArrow`/`alt+downArrow`. (#17015, @LeonarddeR)
 * In Mozilla Firefox, NVDA will report the highlighted text when a URL containing a text fragment is visited. (#16910, @jcsteh)
+* It is now possible to specify a mirror URL to use for the Add-on Store. (#14974)
 
 ### Changes
 
@@ -40,6 +41,8 @@ Add-ons will need to be re-tested and have their manifest updated.
 These are breaking API changes.
 Please open a GitHub issue if your add-on has an issue with updating to the new API.
 
+* The `addonStore.network.BASE_URL` constant has been removed.
+As the Add-on Store base URL is now configurable directly within NVDA, no replacement is planned. (#17099)
 * `NVDAObjects.UIA.winConsoleUIA.WinTerminalUIA` has been removed with no public replacement. (#14047, #16820, @codeofdusk)
 * `NVDAObjects.IAccessible.ia2TextMozilla.FakeEmbeddingTextInfo` has been removed. (#16768, @jcsteh)
 

user_docs/en/userGuide.md

@@ -3073,6 +3073,13 @@ For example, for installed beta add-ons, you will only be notified of updates wi
 |Notify |Notify when updates are available to add-ons within the same channel |
 |Disabled |Do not automatically check for updates to add-ons |
 
+##### Server mirror URL {#AddonStoreMetadataMirror}
+
+This option allows you to specify an alternative URL to download Add-on Store data from.
+This may be of use in locations where access to the NV Access Add-on Store server is slow or unavailable.
+
+Leave this blank to use the default NV Access Add-on Store server.
+
 #### Windows OCR Settings {#Win10OcrSettings}
 
 The settings in this category allow you to configure [Windows OCR](#Win10Ocr).