Handle server side generated archives with prefix _service:* via archive argument without the prefix

[macedogm]

TL;DR: Artifacts generated inside OBS have a _service: prefix which is currently not automatically identified/expanded in the archive parameter.

When we use the tar_scm and recompress services, for example, to download a Git repo before the build phase in OBS, it will create an artifact of the repo named as _service:recompress:tar_scm:file.tar.gz.

For the go_modules service to properly identify the repo archive that will be vendored, the archive file name must match the _service prefix like:

  <service name="go_modules">
    <param name="archive">_service:recompress:tar_scm:file.tar.gz</param>
  </service>

Specifying the full name works, but looks unpleasant and the same _service file can't be used locally and in OBS without modifications, because locally the file will be generated and named as file.tar.gz.

It would be nice if we could write the _service file as exemplified below and then go_modules services tries to match possible file names as file.tar.gz or _service:*:file.tar.gz by doing a simple and controlled regex, for example.

  <service name="go_modules">
    <param name="archive">file.tar.gz</param>
  </service>

[jfkw]

Thanks for documenting the issue. Yes, I'd like to ensure support for this is working.

There related match groups in the basename_from_archive_name() regex:

def basename_from_archive_name(archive_name):
    basename = re.sub(
        r"^(?P<service_prefix>_service:[^:]+:)?(?P<basename>.*)\.(?P<extension>obscpio|tar\.[^\.]+)$",
        r"\g<basename>",
        archive_name,
    )
    if basename:
        log.info(f"Detected basename {basename} form archive name")
    return basename

I'll investigate improvements here to allow regex matching without writing out the prefix in the archive parameter.

[jfkw]

Should the implementation raise an error on _service:* prefix spelled out in the archive parameter?

Doing so including documentation of course would guide toward normalized usage that works on local and server side modes.

Is the number of : delinted sections in the prefix reliably two in all server side service modes? i.e. _service:servicename: Wondered if there were any configurations that might encode additional metadata such as timestamp or version into the server side generated filename.

[macedogm]

In case those questions are for me, my opinion is:

Should the implementation raise an error on _service:* prefix spelled out in the archive parameter?
Doing so including documentation of course would guide toward normalized usage that works on local and server side modes.

Yes.

Is the number of : delinted sections in the prefix reliably two in all server side service modes? i.e. _service:servicename: Wondered if there were any configurations that might encode additional metadata such as timestamp or version into the server side generated filename.

That's a good question and I don't know.

[jfkw]

Based on OBS Docs > Using Source Services > Modes of Services there are no additional configurations that encode extra information e.g. timestamp or version info in the server side generated filenames.

We'll plan to implement the following:

1. Raise an error if the parameter "archive" is spelled with a prefix _service:*:. Don't try to silently fix up the filename to determine the archive basename.
2. Provide documentation and a helpful error message as to the reason. Users should specify an archive name as it would be run in local modes.
3. The regex matching in basename_from_archive_name() is intended to match _service: prefixed archives. Determine whether it does so with the new behavior or in what way it needs to be adjusted.

Packages in OBS that use obs-service-go_modules and specify parameter "archive" spelled with a prefix _service:*: would break. There are currently no packages in openSUSE:Factory that would be affected by this change.