Add support to registry-support for sample checkoutFrom.revision

[thepetk]

Which area this feature is related to?

/area registry

Which functionality do you think we should add?

As mentioned in issue #1112 it could be a good solution to check inside the extraDevfileEntries.yaml for any branches mentioned for the git repo that each sample has.

Why is this needed? Is your feature request related to a problem?

This feature will open the way to other registry features needed, like supporting multiple sample parent stacks versions while having a single repo to maintain.

Detailed description:
The idea of this feature was brought up inside the issue #1112 where we need to intoruce different versions of parent stacks per sample version. For example, a proposed update to a sample like go-basic would be:

- name: go-basic
  displayName: Basic Go
  description: Go 1.16 application
  icon: https://go.dev/blog/go-brand/Go-Logo/SVG/Go-Logo_Blue.svg
  tags:
    - Go
  projectType: Go
  language: Go
  provider: Red Hat
  versions:
    - version: 1.0.2
      schemaVersion: 2.1.0
      git:
        remotes:
          origin: https://github.com/devfile-samples/devfile-sample-go-basic.git
    - version: 2.1.0
      schemaVersion: 2.2.0
      default: true
      git:
        remotes:
          origin: https://github.com/thepetk/devfile-sample-go-basic.git

As you can see, we will have 2 different versions pointing to different parent stack version. Right now, with the current version of registry-support we are not able to use the same repo with different branches.

Describe the solution you'd like

Ideally we could add an additional check to the cache_samples file in order to use a specific branch when caching all samples.

So after this feature is created we can have:

 versions:
  - version: 2.1.0
    schemaVersion: 2.2.0
    default: true
    git:
      checkoutFrom:
        revision: 2.2.x
      remotes:
        origin: https://github.com/devfile-samples/devfile-sample-go-basic.git
  - version: 1.0.2
    schemaVersion: 2.1.0
    git:
      checkoutFrom:
        revision: main
      remotes:
        origin: https://github.com/devfile-samples/devfile-sample-go-basic.git

As you can see, we can use the same repo but different branches in order to support different versions of parent stacks.

Describe alternatives you've considered

Other alternatives can be found here

Acceptance Criteria

• The index generator is able to detect checkoutFrom and use a different branch per version of sample.
• All related documentation must be updated

Target date: 22 Aug 2023

[thepetk]

Blocker for #1112

[thepetk]

A PR is created and currently is under review

[thepetk]

After the merge of the PR for the support of branches for multiple devfile versions an update to the documentation of the Registry contributing guide was added with this PR

[maysunfaisal]

@thepetk is there an issue for registry-support library to support this?

[thepetk]

I'm working on the library part of this issue in order to be able to provide a function which will be able to return all newer (2.2.x) samples.

[thepetk]

@maysunfaisal I think the best way to use the new feature is to:

• Fetch all the samples from registry GET https://registry.devfile.io/v2index/sample.

• For each sample version with schemaVersion 2.2.x, we can try to fetch it by using the library and the URL registry.devfile.io/devfiles//

I was wondering if we need to create a func to wrap all this logic somewhere. If yes, where do you think it should be the best place to implement that?

[maysunfaisal]

@thepetk

• In Go implementation, ODC gets it using the registry-library, so it would be good to have some filter here https://github.com/openshift/console/blob/master/pkg/devfile/sample.go#L19
• In TS implementation, HAC gets it directly though the API URL https://github.com/openshift/hac-dev/blob/11820ae5430662d96e54329f81f6d3a255ecf3da/src/utils/devfile-utils.ts#L51

[thepetk]

@maysunfaisal
I've tried to find any existing workaround for filtering the new versions of samples (sample versions using schemaVersion >= 2.2.x). We have already a filtering mechanism inside the GetRegistryIndex func. So if I run the code:

package main

import (
	"fmt"

	indexSchema "github.com/devfile/registry-support/index/generator/schema"
	registryLibrary "github.com/devfile/registry-support/registry-library/library"
)

const registryURL = "https://registry.devfile.io"

func main() {
	// devfiles matching debugging
	fmt.Printf("** Checking registryLibrary.GetRegistryIndex() ** \n")
	// indexSchema.StackDevfileType will return no results as there are no versions using schemaVersion 2.2.x
	// use indexSchema.StackDevfileType to get more results
	registryIndex, err := registryLibrary.GetRegistryIndex(registryURL, registryLibrary.RegistryOptions{
		Filter: registryLibrary.RegistryFilter{MinSchemaVersion: "2.2.0"}, Telemetry: registryLibrary.TelemetryData{}, NewIndexSchema: true,
	}, indexSchema.StackDevfileType)
	if err != nil {
		fmt.Printf("error:: %s", err)
	} else {
		for _, devfile := range registryIndex {
			fmt.Printf("Devfile %s found\n", devfile.Name)
			for _, version := range devfile.Versions {
				fmt.Printf("Version %s - using schemaVersion %s\n", version.Version, version.SchemaVersion)
			}
		}
	}
}

This way I take only the samples using schemaVersion >= 2.2.0 which can be the case for our issue. WDYT?

[thepetk]

The issue remains under review (regarding PR which updates the contributing guide of registry). If the PR is approved, it can be merged and if the proposed solution for usage of registry support in order to filter samples is ok, we can close the issue.

[maysunfaisal]

@thepetk thanks, let me try it out to see if it meets my needs! I shall get back to you shortly!