Doc: Describing optional PII redaction functionality.

README.md

@@ -7,6 +7,9 @@ Provides additional configuration, styling and components for the Drupal Webform
 * LocalGov Forms Date - A date input field based on the [GDS Date Input pattern](https://design-system.service.gov.uk/components/date-input/)
 * LocalGov address lookup - Webform element with a configurable address lookup backend.  Geocoder plugins act as backends.
 
+## Plugins
+- Personally Identifiable Information (PII) redactor from Webform submissions: At the moment, a plugin manager `plugin.manager.pii_redactor` and a sample plugin are provided.
+
 ## Dependencies
 The geocoder-php/nominatim-provider package is necessary to run automated tests:
 ```
@@ -23,4 +26,3 @@ To avoid the configuration being removed by deployments, install the [Config ign
 webform.webform.*
 webform.webform_options.*
 ```
-

modules/localgov_forms_lts/README.md

@@ -1,12 +1,13 @@
-## Long term storage for Webform submission
+## Long term storage for Webform submissions
+This module copies Webform submissions to a separate database for Long Term Storage (LTS).  The LTS database can then be used for data warehousing needs such as reporting and analysis.  Optionally, Personally Identifiable Information (PII) can be redacted from Webform submissions while they are copied to the LTS database.
 
 ### Setup process
-- Create a database which will serve as the Long term storage.
+- Create a database which will serve as the LTS database.
 - Declare it in Drupal's settings.php using the **localgov_forms_lts** key.  Example:
   ```
   $databases['localgov_forms_lts']['default'] = [
-    'database'  => 'our_longer_term_storage_database',
-    'username'  => 'database_username_goes_here',
+    'database'  => 'our-long-term-storage-database',
+    'username'  => 'database-username-goes-here',
     'password'  => 'database-password-goes-here',
     'host'      => 'database-hostname-goes-here',
     'port'      => '3306',
@@ -16,23 +17,23 @@
   ```
 - Install the localgov_forms_lts submodule.
 - Check the module requirement report from Drupal's status page at `admin/reports/status`.  This should be under the **LocalGov Forms LTS** key.
-- If all looks good in the previous step, run `drush deploy:hook` which will copy existing Webform submissions into the Long term storage.  If you are using `drush deploy`, this will be taken care of as part of it and there would be no need for `drush deploy:hook`.
-- Ensure cron is running periodically.  This will copy any new Webform submissions or changes to existing Webform submissions since deployment or the last cron run.
+- [Optional] If all looks good in the previous step, run `drush localgov-forms-lts:copy --force` which will copy existing Webform submissions into the LTS database.
+- By default, periodic Webform submissions copying to the LTS database is disabled.  Activate it from `/admin/structure/webform/config/submissions-lts`.
+- Ensure cron is running periodically.  This will copy any new Webform submissions or changes to existing Webform submissions since the last cron run or the last `drush localgov-forms-lts:copy` run.
 - [Optional] Tell individual Webforms to purge submissions older than a chosen period.  This is configured for each Webform from its `Settings > Submissions > Submission purge settings` configuration section.
 
 ### Inspection
-To inspect Webform submissions kept in Long term storage, look for the "LTS" tab in the Webform submissions listing page.  This is usually at /admin/structure/webform/submissions/manage.
+To inspect Webform submissions kept in Long term storage, look for the **LTS** tab in the Webform submissions listing page.  This is usually at `/admin/structure/webform/submissions/manage`.
 
 ### Good to know
-- Each cron run copies 50 Webform submissions.  If your site is getting more than that many Webform submissions between subsequent cron runs, not all Webform submissions will get copied to Long term storage during a certain period.  If that happens, adjust cron run frequency.
-- Files attached to Webform submissions are *not* moved to Long term storage.
-- Elements with Personally Identifiable Information (PII) are redacted.  At the moment, this includes all name, email, telephone, number, and various address type elements.  Additionally, any text or radio or checkbox element whose machine name (AKA Key) contains the following also gets redacted: name, mail, phone, contact_number, date_of_birth, dob_, personal_, title, nino, passport, postcode, address, serial_number, reg_number, pcn_, and driver_.
+- Each cron run copies 50 Webform submissions.  If your site is getting more than that many Webform submissions between subsequent cron runs, not all Webform submissions will get copied to LTS during a certain period.  If that happens, adjust cron run frequency.
+- Files attached to Webform submissions are *not* moved to LTS.
+- You can choose to redact elements with Personally Identifiable Information (PII) while they are copied to the LTS database.  For that, select *Best effort PII redactor* (or another redactor) from the `PII redactor plugin` dropdown in the LTS config page at `/admin/structure/webform/config/submissions-lts`.  At the moment, this plugin redacts all name, email, telephone, number, and various address type elements.  Additionally, any text or radio or checkbox element whose machine name (AKA Key) contains the following also gets redacted: name, mail, phone, contact_number, date_of_birth, dob_, personal_, title, nino, passport, postcode, address, serial_number, reg_number, pcn_, and driver_.
+- If you are using this module in multiple instances of the same site (e.g. dev/stage/live), ensure that the database settings array points to *different* databases.  Alternatively, disable copying for the non-live environments from `/admin/structure/webform/config/submissions-lts`.  The relevant settings `localgov_forms_lts.settings:is_copying_enabled` can be [overridden](https://www.drupal.org/docs/drupal-apis/configuration-api/configuration-override-system#s-global-overrides) from settings.php.  The [config_split](https://www.drupal.org/project/config_split) module can be handy as well.
 - This module is currently in experimental stage.
-- If you are using this module in multiple instances of the same site (e.g. dev/stage/live), ensure that the database settings array points to *different* databases.
 
 ### Todo
-- Removal of Webform submissions from Long term storage after a predefined period e.g. 5 years.
-- Machine names which are indicative of PII are hardcoded within the Drupal\localgov_forms_lts\PIIRedactor class at the moment.  This needs a configuration UI.
+- Removal of Webform submissions from LTS after a predefined period e.g. 5 years.
 
 ### Testing in DDEV