local variable 'coll' referenced before assignment syncing EPEL 8 modular Pulp->Pulp

[ianballou]

Version

  "versions": [
    {
      "component": "core",
      "version": "3.28.10",
      "package": "pulpcore",
      "domain_compatible": true
    },
    {
      "component": "rpm",
      "version": "3.22.3",
      "package": "pulp-rpm",
      "domain_compatible": true
    },
    {
      "component": "python",
      "version": "3.10.0",
      "package": "pulp-python",
      "domain_compatible": false
    },
    {
      "component": "ostree",
      "version": "2.1.1",
      "package": "pulp-ostree",
      "domain_compatible": false
    },
    {
      "component": "file",
      "version": "1.14.3",
      "package": "pulp-file",
      "domain_compatible": true
    },
    {
      "component": "deb",
      "version": "3.0.0",
      "package": "pulp_deb",
      "domain_compatible": false
    },
    {
      "component": "container",
      "version": "2.15.2",
      "package": "pulp-container",
      "domain_compatible": false
    },
    {
      "component": "certguard",
      "version": "1.6.5",
      "package": "pulp-certguard",
      "domain_compatible": true
    },
    {
      "component": "ansible",
      "version": "0.18.1",
      "package": "pulp-ansible",
      "domain_compatible": false
    }
  ],

Katello 4.11.master

Describe the bug
Katello server has EPEL 8 modular (https://dl.fedoraproject.org/pub/epel/8/Modular/x86_64/) synced using "On Demand" download policy and "Additive" mirroring policy. A smart proxy server running the same versions as Katello and as above is failing to sync the EPEL 8 modular repo from Katello. Here's the error task info:

---
smart_proxy_history_id: 8
pulp_tasks:
- pulp_href: "/pulp/api/v3/tasks/018b1a1d-d5fd-7bb5-89fc-31e76a6cd6e8/"
  pulp_created: '2023-10-10T15:04:04.865+00:00'
  state: failed
  name: pulp_rpm.app.tasks.synchronizing.synchronize
  logging_cid: 3fc51064-7375-4199-8ad4-bef945a9ce1c
  created_by: "/pulp/api/v3/users/1/"
  started_at: '2023-10-10T15:04:05.812+00:00'
  finished_at: '2023-10-10T15:04:07.119+00:00'
  error:
    traceback: |2
        File "/usr/lib/python3.9/site-packages/pulpcore/tasking/tasks.py", line 66, in _execute_task
          result = func(*args, **kwargs)
        File "/usr/lib/python3.9/site-packages/pulp_rpm/app/tasks/synchronizing.py", line 569, in synchronize
          repo_version = dv.create() or repo.latest_version()
        File "/usr/lib/python3.9/site-packages/pulpcore/plugin/stages/declarative_version.py", line 161, in create
          loop.run_until_complete(pipeline)
        File "/usr/lib64/python3.9/asyncio/base_events.py", line 647, in run_until_complete
          return future.result()
        File "/usr/lib/python3.9/site-packages/pulpcore/plugin/stages/api.py", line 220, in create_pipeline
          await asyncio.gather(*futures)
        File "/usr/lib/python3.9/site-packages/pulpcore/plugin/stages/api.py", line 41, in __call__
          await self.run()
        File "/usr/lib/python3.9/site-packages/pulp_rpm/app/tasks/synchronizing.py", line 860, in run
          await self.parse_repository_metadata(repomd, repomd_files)
        File "/usr/lib/python3.9/site-packages/pulp_rpm/app/tasks/synchronizing.py", line 933, in parse_repository_metadata
          await self.parse_advisories(updateinfo_result)
        File "/usr/lib/python3.9/site-packages/pulp_rpm/app/tasks/synchronizing.py", line 1370, in parse_advisories
          coll.pulp_domain = get_domain()
    description: local variable 'coll' referenced before assignment
  worker: "/pulp/api/v3/workers/018b1569-6327-74fa-8ee4-dcb9eb0351bb/"
  child_tasks: []
  progress_reports:
  - message: Downloading Metadata Files
    code: sync.downloading.metadata
    state: completed
    done: 6
  - message: Downloading Artifacts
    code: sync.downloading.artifacts
    state: canceled
    done: 0
  - message: Associating Content
    code: associating.content
    state: canceled
    done: 0
  - message: Parsed Modulemd
    code: sync.parsing.modulemds
    state: completed
    total: 24
    done: 24
  - message: Parsed Modulemd-defaults
    code: sync.parsing.modulemd_defaults
    state: completed
    total: 5
    done: 5
  - message: Parsed Modulemd Obsolete
    code: sync.parsing.modulemd_obsoletes
    state: completed
    total: 0
    done: 0
  - message: Skipping Packages
    code: sync.skipped.packages
    state: completed
    total: 0
    done: 0
  - message: Parsed Packages
    code: sync.parsing.packages
    state: completed
    total: 290
    done: 290
  - message: Parsed Advisories
    code: sync.parsing.advisories
    state: failed
    total: 14
    done: 0
  created_resources: []
  reserved_resources_record:
  - "/pulp/api/v3/repositories/rpm/rpm/018b1a1d-5e02-7fc0-b75d-b457d98fc8e5/"
  - shared:/pulp/api/v3/remotes/rpm/rpm/018b1a1d-7606-7988-a458-713cdc966fcc/
task_groups: []

To Reproduce
Sync https://dl.fedoraproject.org/pub/epel/8/Modular/x86_64/ on Katello with "On Demand" download policy and "Additive" mirroring policy. Katello may matter here because the error seems to be related to domains, which Katello does not use.

Expected behavior
Sync succeeds

[ianballou]

To workaround the issue I ran a complete sync on the repository after switching to metadata complete mirroring. The Pulp to Pulp sync worked after that.

[ggainey]

I don't see how that line has ever worked. If you want to try a quick hack-test for me, remove 1370 from /usr/lib/python3.9/site-packages/pulp_rpm/app/tasks/synchronizing.py and retry with your original parameters.

UpdateReferences don't even have a pulp_domain...

[dralley]

I mentioned on Slack, but this seems likely to be a copy/paste error from a few lines above, where this line is present in a context where it actually makes sense.

However, it doesn't "really" make sense, since as you said these models don't actually have a pulp_domain by virtue of not actually being content themselves. So it's just setting a value on the object, not setting a field on the model to be saved. In other words it's not doing anything.

[ianballou]

I fixed the line in question and reset my repo to use the additive mirroring policy. Looks like it's working now.