#4040 Map Enterprise Attributes during Trellis Events import

[trackleft]

Description

This pull request adds updates existing configuration to account for config drift and a new migrate process plugin to process enterprise attributes called AZEnterpriseAttributesArrayFlatten.php

This plugin takes API data in the form of

  "az_enterprise_attributes" => array:1 [
    0 => array:3 [
      "Audiences" => array:1 [
        0 => "Community Member"
      ]
      "Topic" => array:1 [
        0 => "Science & Research"
      ]
      "Category" => array:1 [
        0 => "Lectures, Workshops & Panels"
      ]
    ]
  ]

And transforms it to

  "pseudo_az_enterprise_attributes" => array:1 [
    0 => array:7 [

        0 => "Audiences"
        1 => "Topic"
        2 => "Category"
        3 => "Community Member"
        4 => "Science & Research"
        5 => "Lectures"
        6 => "Workshops & Panels"
  ]
]

This then needs to be flattened again, in order to be used.

  "pseudo_az_enterprise_attributes" => array:7 [
    0 => "Audiences"
    1 => "Topic"
    2 => "Category"
    3 => "Community Member"
    4 => "Science & Research"
    5 => "Lectures"
    6 => "Workshops & Panels"
  ]

In the migration for field_az_enterprise_attributes on events, we use entity_lookup (not entity_generate) to find existing enterprise attributes and map them up. These can be entered manually, or imported using the import module. (Note: this is not a migration lookup)

  field_az_enterprise_attributes:
    -
      plugin: entity_lookup
      entity_type: taxonomy_term
      bundle_key: vid
      bundle: az_enterprise_attributes
      value_key: name
      ignore_case: true
      source: '@pseudo_az_enterprise_attributes'

Release notes

For automated enterprise attribute management in events, enable and configure
- Quickstart Enterprise Attributes Import
- Quickstart Event - Trellis Event Importer
- Quickstart Http

Related issues

Close #4040

How to test

Enable:

• Quickstart Enterprise Attributes Import
• Quickstart Event - Trellis Event Importer
• Quickstart Http

Go to /admin/structure/taxonomy/manage/az_enterprise_attributes/overview to ensure enterprise attributes have been enabled.

Go to /admin/trellis-event-importer and import some events.

Now you can check that the imported events have the appropriate enterprise attributes.

Types of changes

Arizona Quickstart (install profile, custom modules, custom theme)

• Patch release changes
  • Bug fix
  • Accessibility, performance, or security improvement
  • Critical institutional link or brand change
  • Adding experimental module
  • Update experimental module
• Minor release changes
  • New feature
  • Breaking or visual change to existing behavior
  • Upgrade experimental module to stable
  • Enable existing module by default or database update
  • Non-critical brand change
  • New internal API or API improvement with backwards compatibility
  • Risky or disruptive cleanup to comply with coding standards
  • High-risk or disruptive change (requires upgrade path, risks regression, etc.)
• Other or unknown
  • Other or unknown

Drupal core

• Patch release changes
  • Security update
  • Patch level release (non-security bug-fix release)
  • Patch removal that's no longer necessary
• Minor release changes
  • Major or minor level update
• Other or unknown
  • Other or unknown

Drupal contrib projects

• Patch release changes
  • Security update
  • Patch or minor level update
  • Add new module
  • Patch removal that's no longer necessary
• Minor release changes
  • Major level update
• Other or unknown
  • Other or unknown

Checklist

• My code follows the code style of this project.
• My change requires a change to the documentation.
• I have updated the documentation accordingly.
• I have read the CONTRIBUTING document.
• I have added tests to cover my changes.
• All new and existing tests passed.
• My change requires release notes.

[trackleft]

Here's a up to date copy of the output from migrate_devel

1^ "---------------------------------------------------------------------"
2^ "|                             $Source                               |"
3^ "---------------------------------------------------------------------"
4^ array:14 [
  "id" => "a3U6R000003C2r2UAC"
  "name" => "WSIP Workshop: 'AI and Academic Writing'"
  "url" => "https://events.trellis.arizona.edu/f44lNu67/5a3U6R3C2r2"
  "summary" => "We'll address the growing popularity of Artificial Intelligence tools available among academic writers and some common benefits and dangers with their use."
  "timezone" => "America/Phoenix"
  "start_date_time" => "2025-02-10T19:00:00.000Z"
  "end_date_time" => "2025-02-10T19:50:00.000Z"
  "publish" => true
  "az_enterprise_attributes" => array:1 [
    0 => array:3 [
      "Audiences" => array:1 [
        0 => "Staff, Community Member, Undergraduate Student, Graduate Student, Research & Teaching Faculty/Postdoc"
      ]
      "Topic" => array:1 [
        0 => "Student Success"
      ]
      "Category" => array:1 [
        0 => "Lectures, Workshops & Panels"
      ]
    ]
  ]
  "plugin" => "az_trellis_events_api"
  "trellis_ids" => []
  "track_changes" => true
  "fields" => array:10 [
    0 => array:3 [
      "name" => "id"
      "label" => "ID"
      "selector" => "Id"
    ]
    1 => array:3 [
      "name" => "name"
      "label" => "Name"
      "selector" => "Name"
    ]
    2 => array:3 [
      "name" => "url"
      "label" => "URL"
      "selector" => "Branded_Event_URL"
    ]
    3 => array:3 [
      "name" => "summary"
      "label" => "Summary"
      "selector" => "Summary"
    ]
    4 => array:3 [
      "name" => "owner"
      "label" => "Owner NetID"
      "selector" => "Owner_NetID"
    ]
    5 => array:3 [
      "name" => "timezone"
      "label" => "Time Zone"
      "selector" => "tz_name"
    ]
    6 => array:3 [
      "name" => "start_date_time"
      "label" => "Start Date and Time"
      "selector" => "Start_DateTime"
    ]
    7 => array:3 [
      "name" => "end_date_time"
      "label" => "End Date and Time"
      "selector" => "End_DateTime"
    ]
    8 => array:3 [
      "name" => "publish"
      "label" => "Publish"
      "selector" => "Publish_Formula"
    ]
    9 => array:3 [
      "name" => "az_enterprise_attributes"
      "label" => "Enterprise Attributes"
      "selector" => "Attributes"
    ]
  ]
  "ids" => array:1 [
    "id" => array:1 [
      "type" => "string"
    ]
  ]
]
5^ "---------------------------------------------------------------------"
6^ "|                           $Destination                            |"
7^ "---------------------------------------------------------------------"
8^ array:8 [
  "pseudo_az_enterprise_attributes" => array:11 [
    0 => "Audiences"
    1 => "Topic"
    2 => "Category"
    3 => "Staff"
    4 => "Community Member"
    5 => "Undergraduate Student"
    6 => "Graduate Student"
    7 => "Research & Teaching Faculty/Postdoc"
    8 => "Student Success"
    9 => "Lectures"
    10 => "Workshops & Panels"
  ]
  "field_az_enterprise_attributes" => array:9 [
    0 => "72"
    1 => "2"
    2 => "42"
    3 => "71"
    4 => "70"
    5 => "65"
    6 => "61"
    7 => "4"
    8 => "5"
  ]
  "title" => "WSIP Workshop: 'AI and Academic Writing'"
  "field_az_trellis_id" => "a3U6R000003C2r2UAC"
  "field_az_link" => array:2 [
    "uri" => "https://events.trellis.arizona.edu/f44lNu67/5a3U6R3C2r2"
    "title" => "Event details"
  ]
  "field_az_summary" => "We'll address the growing popularity of Artificial Intelligence tools available among academic writers and some common benefits and dangers with their use."
  "status" => true
  "field_az_event_date" => array:2 [
    "value" => "1739214000"
    "end_value" => "1739217000"
  ]
]
9^ "---------------------------------------------------------------------"
10^ "|                       $DestinationIdValues                        |"
11^ "---------------------------------------------------------------------"
12^ array:1 [
  0 => "516"
]
1^ "---------------------------------------------------------------------"
2^ "|                             $Source                               |"
3^ "---------------------------------------------------------------------"
4^ array:14 [
  "id" => "a3U6R000003C2rWUAS"
  "name" => "Drop-in Crafting: Make Your Own Victorian Valentine"
  "url" => "https://events.trellis.arizona.edu/f44lNu67/5a3U6R3C2rW"
  "summary" => "Make a card or two at CATalyst Studios."
  "timezone" => "America/Phoenix"
  "start_date_time" => "2025-02-13T22:30:00.000Z"
  "end_date_time" => "2025-02-14T00:00:00.000Z"
  "publish" => true
  "az_enterprise_attributes" => array:1 [
    0 => array:3 [
      "Audiences" => array:1 [
        0 => "Staff, Community Member, Undergraduate Student, Graduate Student, Research & Teaching Faculty/Postdoc, Designated Campus Colleague (DCC)"
      ]
      "Topic" => array:1 [
        0 => "Arts & Culture"
      ]
      "Category" => array:1 [
        0 => "Lectures, Workshops & Panels"
      ]
    ]
  ]
  "plugin" => "az_trellis_events_api"
  "trellis_ids" => []
  "track_changes" => true
  "fields" => array:10 [
    0 => array:3 [
      "name" => "id"
      "label" => "ID"
      "selector" => "Id"
    ]
    1 => array:3 [
      "name" => "name"
      "label" => "Name"
      "selector" => "Name"
    ]
    2 => array:3 [
      "name" => "url"
      "label" => "URL"
      "selector" => "Branded_Event_URL"
    ]
    3 => array:3 [
      "name" => "summary"
      "label" => "Summary"
      "selector" => "Summary"
    ]
    4 => array:3 [
      "name" => "owner"
      "label" => "Owner NetID"
      "selector" => "Owner_NetID"
    ]
    5 => array:3 [
      "name" => "timezone"
      "label" => "Time Zone"
      "selector" => "tz_name"
    ]
    6 => array:3 [
      "name" => "start_date_time"
      "label" => "Start Date and Time"
      "selector" => "Start_DateTime"
    ]
    7 => array:3 [
      "name" => "end_date_time"
      "label" => "End Date and Time"
      "selector" => "End_DateTime"
    ]
    8 => array:3 [
      "name" => "publish"
      "label" => "Publish"
      "selector" => "Publish_Formula"
    ]
    9 => array:3 [
      "name" => "az_enterprise_attributes"
      "label" => "Enterprise Attributes"
      "selector" => "Attributes"
    ]
  ]
  "ids" => array:1 [
    "id" => array:1 [
      "type" => "string"
    ]
  ]
]
5^ "---------------------------------------------------------------------"
6^ "|                           $Destination                            |"
7^ "---------------------------------------------------------------------"
8^ array:8 [
  "pseudo_az_enterprise_attributes" => array:12 [
    0 => "Audiences"
    1 => "Topic"
    2 => "Category"
    3 => "Staff"
    4 => "Community Member"
    5 => "Undergraduate Student"
    6 => "Graduate Student"
    7 => "Research & Teaching Faculty/Postdoc"
    8 => "Designated Campus Colleague (DCC)"
    9 => "Arts & Culture"
    10 => "Lectures"
    11 => "Workshops & Panels"
  ]
  "field_az_enterprise_attributes" => array:10 [
    0 => "72"
    1 => "2"
    2 => "71"
    3 => "70"
    4 => "65"
    5 => "62"
    6 => "61"
    7 => "4"
    8 => "5"
    9 => "31"
  ]
  "title" => "Drop-in Crafting: Make Your Own Victorian Valentine"
  "field_az_trellis_id" => "a3U6R000003C2rWUAS"
  "field_az_link" => array:2 [
    "uri" => "https://events.trellis.arizona.edu/f44lNu67/5a3U6R3C2rW"
    "title" => "Event details"
  ]
  "field_az_summary" => "Make a card or two at CATalyst Studios."
  "status" => true
  "field_az_event_date" => array:2 [
    "value" => "1739485800"
    "end_value" => "1739491200"
  ]
]
9^ "---------------------------------------------------------------------"
10^ "|                       $DestinationIdValues                        |"
11^ "---------------------------------------------------------------------"
12^ array:1 [
  0 => "517"
]
1^ "---------------------------------------------------------------------"
2^ "|                             $Source                               |"
3^ "---------------------------------------------------------------------"
4^ array:14 [
  "id" => "a3U6R000003C2sFUAS"
  "name" => "Doctoral Oral Examination – Second Language Acquisition and Teaching"
  "url" => "https://events.trellis.arizona.edu/f44lNu67/5a3U6R3C2sF"
  "summary" => """
    Hanyu Jia\n
    "Exploring the Affordances of Immersive Virtual Reality for Chinese L2 Pragmatics Learning"
    """
  "timezone" => "America/Phoenix"
  "start_date_time" => "2025-02-26T17:00:00.000Z"
  "end_date_time" => "2025-02-26T20:00:00.000Z"
  "publish" => true
  "az_enterprise_attributes" => array:1 [
    0 => array:3 [
      "Audiences" => array:1 [
        0 => "Graduate Student, Research & Teaching Faculty/Postdoc"
      ]
      "Topic" => array:1 [
        0 => "Social Sciences & Education"
      ]
      "Category" => array:1 [
        0 => "Thesis/Doctoral Performances & Defenses"
      ]
    ]
  ]
  "plugin" => "az_trellis_events_api"
  "trellis_ids" => []
  "track_changes" => true
  "fields" => array:10 [
    0 => array:3 [
      "name" => "id"
      "label" => "ID"
      "selector" => "Id"
    ]
    1 => array:3 [
      "name" => "name"
      "label" => "Name"
      "selector" => "Name"
    ]
    2 => array:3 [
      "name" => "url"
      "label" => "URL"
      "selector" => "Branded_Event_URL"
    ]
    3 => array:3 [
      "name" => "summary"
      "label" => "Summary"
      "selector" => "Summary"
    ]
    4 => array:3 [
      "name" => "owner"
      "label" => "Owner NetID"
      "selector" => "Owner_NetID"
    ]
    5 => array:3 [
      "name" => "timezone"
      "label" => "Time Zone"
      "selector" => "tz_name"
    ]
    6 => array:3 [
      "name" => "start_date_time"
      "label" => "Start Date and Time"
      "selector" => "Start_DateTime"
    ]
    7 => array:3 [
      "name" => "end_date_time"
      "label" => "End Date and Time"
      "selector" => "End_DateTime"
    ]
    8 => array:3 [
      "name" => "publish"
      "label" => "Publish"
      "selector" => "Publish_Formula"
    ]
    9 => array:3 [
      "name" => "az_enterprise_attributes"
      "label" => "Enterprise Attributes"
      "selector" => "Attributes"
    ]
  ]
  "ids" => array:1 [
    "id" => array:1 [
      "type" => "string"
    ]
  ]
]
5^ "---------------------------------------------------------------------"
6^ "|                           $Destination                            |"
7^ "---------------------------------------------------------------------"
8^ array:8 [
  "pseudo_az_enterprise_attributes" => array:7 [
    0 => "Audiences"
    1 => "Topic"
    2 => "Category"
    3 => "Graduate Student"
    4 => "Research & Teaching Faculty/Postdoc"
    5 => "Social Sciences & Education"
    6 => "Thesis/Doctoral Performances & Defenses"
  ]
  "field_az_enterprise_attributes" => array:7 [
    0 => "2"
    1 => "58"
    2 => "41"
    3 => "70"
    4 => "65"
    5 => "4"
    6 => "5"
  ]
  "title" => "Doctoral Oral Examination – Second Language Acquisition and Teaching"
  "field_az_trellis_id" => "a3U6R000003C2sFUAS"
  "field_az_link" => array:2 [
    "uri" => "https://events.trellis.arizona.edu/f44lNu67/5a3U6R3C2sF"
    "title" => "Event details"
  ]
  "field_az_summary" => """
    Hanyu Jia\n
    "Exploring the Affordances of Immersive Virtual Reality for Chinese L2 Pragmatics Learning"
    """
  "status" => true
  "field_az_event_date" => array:2 [
    "value" => "1740589200"
    "end_value" => "1740600000"
  ]
]
9^ "---------------------------------------------------------------------"
10^ "|                       $DestinationIdValues                        |"
11^ "---------------------------------------------------------------------"
12^ array:1 [
  0 => "518"
]
 [notice] Processed 518 items (0 created, 518 updated, 0 failed, 0 ignored) - done with 'az_trellis_events'

[tadean]

One oddity I notice about the Event search API is that the Format attribute is available, but it is as a top level field and not in the attributes array:

"Format" : "In-Person",

[tadean]

In testing this mapping, I think I found two separate issues. The import mapping adds references for the parent terms like Audience, Topic, etc - but the Attribute field widget does not allow these terms to be selected, only the child terms. I think the import should only set the child terms, the parent terms are enforced by the widget config.

On the matching news config, parent terms don't get referenced, only the child terms.

The second is that after saving the event again, the attributes are empty and are lost. This definitely appears to be related to the field being disabled for trellis events - I'm uncertain why this happens because other widgets besides this one don't lose their value when disabled. Maybe it's a product of the transformations that happen to the data on saving with the Attributes widget.

Example field data before the additional save (the first few ids are the parent terms that probably should be tagged):

MariaDB [drupal10]> select * from node__field_az_enterprise_attributes where entity_id=1 order by field_az_enterprise_attributes_target_id;
+----------+---------+-----------+-------------+----------+-------+------------------------------------------+
| bundle   | deleted | entity_id | revision_id | langcode | delta | field_az_enterprise_attributes_target_id |
+----------+---------+-----------+-------------+----------+-------+------------------------------------------+
| az_event |       0 |         1 |           1 | en       |     2 |                                        2 |
| az_event |       0 |         1 |           1 | en       |     6 |                                        3 |
| az_event |       0 |         1 |           1 | en       |     8 |                                        4 |
| az_event |       0 |         1 |           1 | en       |     9 |                                        5 |
| az_event |       0 |         1 |           1 | en       |    10 |                                       30 |
| az_event |       0 |         1 |           1 | en       |     7 |                                       36 |
| az_event |       0 |         1 |           1 | en       |     3 |                                       42 |
| az_event |       0 |         1 |           1 | en       |     0 |                                       44 |
| az_event |       0 |         1 |           1 | en       |     4 |                                       45 |
| az_event |       0 |         1 |           1 | en       |     5 |                                       65 |
| az_event |       0 |         1 |           1 | en       |     1 |                                       72 |
+----------+---------+-----------+-------------+----------+-------+------------------------------------------+
11 rows in set (0.001 sec)

And after the additional save:

MariaDB [drupal10]> select * from node__field_az_enterprise_attributes where entity_id=1 order by field_az_enterprise_attributes_target_id;
+----------+---------+-----------+-------------+----------+-------+------------------------------------------+
| bundle   | deleted | entity_id | revision_id | langcode | delta | field_az_enterprise_attributes_target_id |
+----------+---------+-----------+-------------+----------+-------+------------------------------------------+
| az_event |       0 |         1 |           4 | en       |     0 |                                        0 |
| az_event |       0 |         1 |           4 | en       |     1 |                                        1 |
| az_event |       0 |         1 |           4 | en       |     2 |                                        2 |
| az_event |       0 |         1 |           4 | en       |     3 |                                        3 |
+----------+---------+-----------+-------------+----------+-------+------------------------------------------+

Note that it seems like the target ids have been replaced by the delta or something...0 and 1 were not referenced originally. I'm guessing that if the widget is disabled, something in the Widget's massage form values fails.