Update Devfile Samples to reference v2.2.0 Parent stacks #1112

kim-tsao · 2023-05-02T15:27:56Z

Which area this feature is related to?

/area registry

Which functionality do you think we should add?

The current samples have a parent reference to the default stack version in the registry. e.g. Devfile sample go basic points to the default go stack version in the community registry. With this set up, parsing the samples will be successful because there are no conflicts. If the sample were changed to point to the latest version (v2.2.0 schema), parsing will fail because the parent and sample devfiles both contain the same command ids. We should create a new version of the devfile sample that is compatible with the parent 2.2.0 schema.

Why is this needed? Is your feature request related to a problem?

Detailed description:

Describe the solution you'd like

Describe alternatives you've considered

Additional context

thepetk · 2023-05-15T14:57:51Z

@kim-tsao I'm not 100% about the workaround for this issue. In my mind I have two options:

Create a dir inside the sample repo and have different versions of devfiles (like we do in the registry)
Create a new branch that will host the new devfile which will point to the latest version of go stack)

kim-tsao · 2023-05-17T17:16:07Z

@kim-tsao I'm not 100% about the workaround for this issue. In my mind I have two options:

1. Create a dir inside the sample repo and have different versions of devfiles (like we do in the registry)

2. Create a new branch that will host the new devfile which will point to the latest version of go stack)

@thepetk I had to verify if the extraDevfileEntries.yaml has multi-version support and it looks like it does: https://github.com/devfile/registry-support/blob/main/index/generator/tests/registry/extraDevfileEntries.yaml#L11-L23

We can set up the samples similar to the test registry nodejs sample , so option 1 is preferred but we need to see if the viewer can process samples like stacks where default versions can be specified like this example here: https://github.com/devfile/registry/blob/main/stacks/nodejs/stack.yaml

thepetk · 2023-05-19T13:59:29Z

Moving to next sprint (239) as we need to put it on hold for now.

thepetk · 2023-06-26T13:46:15Z

Resumed working on the issue.

thepetk · 2023-06-28T11:52:28Z

Working locally in order to check if the registry viewer supports sample versioning

thepetk · 2023-06-28T19:26:44Z

I've tried to check if the viewer supports multiple versions for samples:

Created a new image locally for registry having the nodejs sample with two versions:

versions:
  - version: 1.0.0
    schemaVersion: 2.0.0
    description: nodejs with devfile v2.0.0
    git:
      remotes:
        origin: https://github.com/redhat-developer/devfile-sample
  - version: 1.0.1
    schemaVersion: 2.2.1
    default: true
    description: nodejs with devfile v2.2.0
    git:
      remotes:
        origin: https://github.com/nodeshift-starters/devfile-sample

I've built the image and deploy it on local minikube cluster but I wasn't able to see the 2 different versions of the sample in the UI.

Should we consider adding a feature in the viewer or am I missing something?

thepetk · 2023-06-29T16:42:38Z

We have confirmed that the viewer does show up an updated image of the registry with multi-version samples.
Additionally it is confirmed that the index generator is able to handle multi-version samples. The investigation will continue on index server side (in order to be sure that the API is ok before started investigating on viewer side)

thepetk · 2023-07-03T14:24:46Z

After an investigation on the registry-support I have found that the multiple versions of a sample are supported. I think most probably we should investigate the viewer.

More detailed looking into v2index/sample I can see all updates from the image and both versions look ok.

{
  "name": "nodejs-basic",
  "displayName": "Basic Node.js",
  "description": "Node.js 16 application using Express 4.18.x",
  "type": "sample",
  "tags": [
    "Node.js",
    "Express"
  ],
  "icon": "https://nodejs.org/static/images/logos/nodejs-new-pantone-black.svg",
  "projectType": "Node.js",
  "language": "JavaScript",
  "provider": "Red Hat",
  "versions": [
    {
      "version": "1.0.0",
      "schemaVersion": "2.0.0",
      "git": {
        "remotes": {
          "origin": "https://github.com/redhat-developer/devfile-sample"
        }
      },
      "description": "nodejs with devfile v2.0.0"
    },
    {
      "version": "1.0.1",
      "schemaVersion": "2.2.1",
      "default": true,
      "git": {
        "remotes": {
          "origin": "https://github.com/nodeshift-starters/devfile-sample"
        }
      },
      "description": "nodejs with devfile v2.2.0"
    }
  ]
}

thepetk · 2023-07-07T13:34:00Z

@michael-valdron I was investigating on the viewer side and found this https://github.com/devfile/devfile-web/blob/main/libs/core/src/functions/fetch-devfiles/fetch-devfiles.tsx#L38 which I think means that by default on the viewer we are not accepting samples having multiple versions. WDYT?

michael-valdron · 2023-07-07T20:48:37Z

@michael-valdron I was investigating on the viewer side and found this https://github.com/devfile/devfile-web/blob/main/libs/core/src/functions/fetch-devfiles/fetch-devfiles.tsx#L38 which I think means that by default on the viewer we are not accepting samples having multiple versions. WDYT?

@thepetk Yeah that looks to be the culprit. I checked the /v2index again and I can see both versions for code-with-quarkus so it looks like a fix we need for the registry viewer.

thepetk · 2023-07-10T09:23:55Z

@michael-valdron do you think we should create a separate issue for the viewer or we should included it as part of this item?

michael-valdron · 2023-07-10T16:36:11Z

@michael-valdron do you think we should create a separate issue for the viewer or we should included it as part of this item?

@thepetk I think this should be its own item, as a user story under the area/registry-viewer tag with a reference to this thread.

thepetk · 2023-07-11T11:10:38Z

Issue created here: #1184 , not sure if we should assign it the blocker label.

Temporarily, I'm removing the target date for this issue.

thepetk · 2023-07-28T11:00:11Z

I'm resuming working on the issue. As discussed, we can introduce references to v.2.2.0 Parent stacks even if the viewer is not supporting them.

New target date: [EDIT] Removed

thepetk · 2023-08-02T12:53:48Z

I've tested different things while looking for the best workaround. The approach I'm most interested in is:

Multiple sample versions inside extraDevfileEntries

As suggested above a good way to start is to include multiple sample versions inside the extraDevfileEntries.yaml. For example, for go-basic we can have two different versions, one pointing to the default 1.0.2 and one pointing to 2.1.0 which supports schemaVersion >= 2.2.x.

- name: go-basic
  displayName: Basic Go
  description: Go 1.16 application
  icon: https://go.dev/blog/go-brand/Go-Logo/SVG/Go-Logo_Blue.svg
  tags:
    - Go
  projectType: Go
  language: Go
  provider: Red Hat
  versions:
    - version: 1.0.2
      schemaVersion: 2.1.0
      git:
        remotes:
          origin: https://github.com/devfile-samples/devfile-sample-go-basic.git
    - version: 2.1.0
      schemaVersion: 2.2.0
      default: true
      git:
        remotes:
          origin: https://github.com/thepetk/devfile-sample-go-basic.git

Separate Branch, Separate Sample Repo OR Directory with devfiles?

If we follow the above format we will need to support different of devfile samples versions for devfile.yaml. I can see 3 approaches:

[Not Supported ATM] Separate branch per devfile sample repo

This way we can introduce a new branch for every sample repo (e.g. 2.2.x default name) and upon extraDevfileEntries.yaml we can have:

versions:
  - version: 2.1.0
    schemaVersion: 2.2.0
    git:
      checkoutFrom:
        revision: 2.2.x
      remotes:
        origin: https://github.com/devfile-samples/devfile-sample-go-basic.git

If I'm not missing something, right now we don't support this in build-tools. If so, we will have to create a new issue to support this and then we can unblock this approach.

[Not Supported ATM] Directory inside each samples that will introduce different devfile versions

The idea is to have exactly the same structure as the stacks. So for every devfile sample we can have:

sample_root_dir/
...
---   devfiles/
       --- version1/
       --- version2/
       ...
       --- versionX/

Use different repo per devfile.yaml version

Following this solution we will have to use a different repo per devfile version. For example for the go-basic:

  versions:
    - version: 1.0.2
      schemaVersion: 2.1.0
      git:
        remotes:
          origin: <repo1>
    - version: 2.1.0
      schemaVersion: 2.2.0
      default: true
      git:
        remotes:
          origin: <repo2>

thepetk · 2023-08-02T16:06:39Z

As discussed we will proceed with the option "Separate branch per devfile sample repo". As result, I've created an issue for the necessary actions we need to take for the registry-support #1209. I've added label "blocked" as #1209 needs to be completed first. I've also removed the target date

thepetk · 2023-08-04T10:12:57Z

Moving to next sprint as the #1209 is currently under review

thepetk · 2023-09-01T11:17:30Z

I'll move this item to next sprint, as the blocker is still under review.

thepetk · 2023-09-25T10:36:19Z

Resuming working as the work needed to be done for #1209 is now completed

thepetk · 2023-09-28T14:10:05Z

As I'm working on the proposed workaround - having multiple version branches inside a sample repo - I think we should create some rules for every new branch that represents a new sample version.

Branch naming

The first thing we should consider is the naming of those branches. More detailed, I think they should follow a versioning format and every new version should be of type v{major}.{minor}.{bugfix}. Example branch: v1.1.1

Branch rules

Those branches should follow the same rules as the main/master (e.g the default) branch of the repo (more info here).

Update Contributing Guide

As a result, we should also write those details down and update the registry contributing guide for this process in order to make everyone aware of this procedure.

thepetk · 2023-10-11T10:19:39Z

As I'm working on the proposed workaround - having multiple version branches inside a sample repo - I think we should create some rules for every new branch that represents a new sample version.

Branch naming

The first thing we should consider is the naming of those branches. More detailed, I think they should follow a versioning format and every new version should be of type v{major}.{minor}.{bugfix}. Example branch: v1.1.1

Branch rules

Those branches should follow the same rules as the main/master (e.g the default) branch of the repo (more info here).

Update Contributing Guide

As a result, we should also write those details down and update the registry contributing guide for this process in order to make everyone aware of this procedure.

Atm the last actions to complete this issue are the ones mentioned here.

thepetk · 2023-10-16T12:52:05Z

Update for multiple sample versions

For every new sample version which will support the latest parent we will also update the sample port to be 8080 in all samples.

Related PRs

That said the following PRs have been created/updated:

python-basic: Create version 2.0.0 for python sample devfile-samples/devfile-sample-python-basic#35
java-springboot-basic: [DO NOT MERGE] Update port from 8081 to 8080 devfile-samples/devfile-sample-java-springboot-basic#33
code-with-quarkus: [DO NOT MERGE] Update port from 8081 to 8080 devfile-samples/devfile-sample-code-with-quarkus#23
dotnet-basic: [DO NOT MERGE] Update port from 8081 to 8080 devfile-samples/devfile-sample-dotnet60-basic#10
go-basic: Create version 2.0.0 for go-basic sample devfile-samples/devfile-sample-go-basic#29

Two issues need to be addressed before creating those versions:

[UPDATE @thepetk] =====================================

We already use port 8080 in `python-basic`, `java-springboot-basic` and `dotnet-basic`

~~In order to fix that, I've created PR#220.~~
The PR is closed. The devfile/api supports duplicate ports from different components in its latest version

=====================================================

We need to add protection to the version branches before merging

As it is mentioned above we will have to protect the following, already created branches which are pointing to the default sample version:

thepetk · 2023-10-23T19:03:19Z

I can confirm that all branches are protected, as only members of the devfile-samples org can merge PRs in any branch.

Other than that, in order to be able to use consistent port numbers I've created a PR (devfile/registry-support#188) to update the library version, which will let us accept two components declaring the same port.

thepetk · 2023-10-30T16:20:38Z

As seen during testing, the ODC uses index/sample so it always fetches the default branch of each sample.

That said, in order to have always the latest version of every sample on the default branch we should first complete #1136. This update will allow us to have consistent ports (8080) on every latest sample version.

As a result, I'm putting it back to todo and putting the blocked label.

thepetk · 2023-11-02T12:08:19Z

The library version of the registry support has been updated (here) and now we can support building a registry with a sample having multiple components using the same port

thepetk · 2023-11-07T12:42:18Z

Updating this item due to an issue found for parent overrides inside samples.

New scope

This issue will focus on updating the extraDevfileEntries.yaml inside the registry by introducing a new sample version to every sample that has a parent with multiple stack versions. For example, the go-basic parent, go stack, has more than one versions so we will need to add a new go-basic version that will point to the latest parent version (atm 2.1.0).

Default Version = Default Branch

For each sample we will keep the convention of maintaining the default sample version to the default sample branch. For example the default version of the go-basic sample (v1.0.2) will be inside the main branch (default branch of the repo).

Latest Version = v2.x.x Branch

For every sample that needs to have a new version we will create a new branch starting with v2.x.x. format. This will be the latest version of the sample.

Updated extraDevfileEntries.yaml

After the completion of the above steps the new extraDevfileEntries.yaml will look like:

schemaVersion: 1.0.0
samples:
  - name: nodejs-basic
    displayName: Basic Node.js
    description: Node.js 16 application using Express 4.18.x
    icon: https://nodejs.org/static/images/logos/nodejs-new-pantone-black.svg
    tags:
      - Node.js
      - Express
    projectType: Node.js
    language: JavaScript
    provider: Red Hat
    versions:
      - version: 1.0.0
        default: true
        schemaVersion: 2.0.0
        git:
          revision: main           
          remotes:
            origin: https://github.com/redhat-developer/devfile-sample.git
  - name: go-basic
    displayName: Basic Go
    description: Go 1.16 application
    icon: https://go.dev/blog/go-brand/Go-Logo/SVG/Go-Logo_Blue.svg
    tags:
      - Go
    projectType: Go
    language: Go
    provider: Red Hat
    versions:
      - version: 1.1.0
        default: true
        schemaVersion: 2.2.0
        git:
          revision: main
          remotes:
            origin: https://github.com/devfile-samples/devfile-sample-go-basic.git
      - version: 2.0.0
        schemaVersion: 2.2.0
        git:
          revision: v2.0.0
          remotes:
            origin: https://github.com/devfile-samples/devfile-sample-go-basic.git

thepetk · 2023-11-08T12:35:00Z

Removing the blocked label as the updated scope is not affected anymore by the blocker.

Also I've created a new version for each sample having a parent stack with multiple versions:

After those PRs are reviewed we can go ahead and create a PR for the updated extraDevfileEntries.yaml

thepetk · 2023-11-13T16:01:23Z

Removing the blocked label as the updated scope is not affected anymore by the blocker.

Also I've created a new version for each sample having a parent stack with multiple versions:

Create springboot sample 2.0.0 version devfile-samples/devfile-sample-java-springboot-basic#37

Create version 2.0.0 for python sample devfile-samples/devfile-sample-python-basic#35

Create version 2.0.0 for go-basic sample devfile-samples/devfile-sample-go-basic#29

After those PRs are reviewed we can go ahead and create a PR for the updated extraDevfileEntries.yaml

I'm moving this item to review status as the 3 sample related PRs are under review :)

thepetk · 2023-11-27T12:50:01Z

I've splited the current issue into two parts:

Create new sample versions for sample with parent outerloop support #1360 (which is the new versions implementation).
The current issue which will focus on the update of the extraDevfileEntries.yaml. I'm putting it back to in progress and in the next sprint as it will have to wait until the merge of the PRs related to Create new sample versions for sample with parent outerloop support #1360

I've updated the story points of this issue to 5 and the new issue has 3 points (total 8, exactly like the previous story points)

thepetk · 2023-11-28T15:08:28Z

#1360 is now completed we can focus on the update of the extraDevfileEntries.yaml

thepetk self-assigned this May 11, 2023

kim-tsao mentioned this issue May 17, 2023

[java-springboot] Use LTS JDK 17 image devfile/registry#169

Merged

4 tasks

thepetk mentioned this issue Jul 11, 2023

Support multiple sample versions on Registry Viewer #1184

Open

2 tasks

thepetk mentioned this issue Aug 2, 2023

Add support to registry-support for sample checkoutFrom.revision #1209

Closed

2 tasks

thepetk added area/registry Devfile registry for stacks and infrastructure status/blocked labels Aug 2, 2023

thepetk removed the status/blocked label Sep 25, 2023

thepetk mentioned this issue Sep 28, 2023

Create version 2.0.0 for go-basic sample devfile-samples/devfile-sample-go-basic#29

Merged

elsony added the demo Issue or PR candidate for a demo at the end of the Sprint label Oct 2, 2023

thepetk mentioned this issue Oct 9, 2023

Create version 2.0.0 for python sample devfile-samples/devfile-sample-python-basic#35

Merged

thepetk mentioned this issue Oct 23, 2023

Update library version and fix revision usage in cache samples devfile/registry-support#188

Merged

3 tasks

thepetk mentioned this issue Oct 26, 2023

Update devfile library version of registry support #1310

Closed

1 task

thepetk added the status/blocked label Oct 30, 2023

thepetk mentioned this issue Oct 30, 2023

Update ODC's Devfile Library version #1136

Closed

2 tasks

thepetk closed this as completed in devfile/registry-support#188 Nov 2, 2023

thepetk reopened this Nov 2, 2023

thepetk mentioned this issue Nov 7, 2023

Create springboot sample 2.0.0 version devfile-samples/devfile-sample-java-springboot-basic#37

Merged

thepetk removed the status/blocked label Nov 8, 2023

thepetk mentioned this issue Nov 27, 2023

Create new sample versions for sample with parent outerloop support #1360

Closed

3 tasks

thepetk mentioned this issue Nov 30, 2023

Add sample versions in extraDevfileEntries.yaml devfile/registry#254

Merged

4 tasks

thepetk closed this as completed in devfile/registry#254 Dec 5, 2023

thepetk mentioned this issue Dec 6, 2023

Fix registry's nightly run check for RHTAP #1372

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Update Devfile Samples to reference v2.2.0 Parent stacks #1112

Update Devfile Samples to reference v2.2.0 Parent stacks #1112

kim-tsao commented May 2, 2023 •

edited by thepetk

Loading

thepetk commented May 15, 2023 •

edited

Loading

kim-tsao commented May 17, 2023

thepetk commented May 19, 2023

thepetk commented Jun 26, 2023 •

edited

Loading

thepetk commented Jun 28, 2023

thepetk commented Jun 28, 2023 •

edited

Loading

thepetk commented Jun 29, 2023

thepetk commented Jul 3, 2023 •

edited

Loading

thepetk commented Jul 7, 2023

michael-valdron commented Jul 7, 2023

thepetk commented Jul 10, 2023

michael-valdron commented Jul 10, 2023

thepetk commented Jul 11, 2023 •

edited

Loading

thepetk commented Jul 28, 2023 •

edited

Loading

thepetk commented Aug 2, 2023 •

edited

Loading

thepetk commented Aug 2, 2023 •

edited

Loading

thepetk commented Aug 4, 2023

thepetk commented Sep 1, 2023

thepetk commented Sep 25, 2023

thepetk commented Sep 28, 2023 •

edited

Loading

thepetk commented Oct 11, 2023

Branch naming

Branch rules

Update Contributing Guide

thepetk commented Oct 16, 2023 •

edited

Loading

thepetk commented Oct 23, 2023

thepetk commented Oct 30, 2023

thepetk commented Nov 2, 2023 •

edited

Loading

thepetk commented Nov 7, 2023

thepetk commented Nov 8, 2023

thepetk commented Nov 13, 2023

thepetk commented Nov 27, 2023 •

edited

Loading

thepetk commented Nov 28, 2023

Update Devfile Samples to reference v2.2.0 Parent stacks #1112

Update Devfile Samples to reference v2.2.0 Parent stacks #1112

Comments

kim-tsao commented May 2, 2023 • edited by thepetk Loading

Which area this feature is related to?

Which functionality do you think we should add?

Describe the solution you'd like

Describe alternatives you've considered

Additional context

thepetk commented May 15, 2023 • edited Loading

kim-tsao commented May 17, 2023

thepetk commented May 19, 2023

thepetk commented Jun 26, 2023 • edited Loading

thepetk commented Jun 28, 2023

thepetk commented Jun 28, 2023 • edited Loading

thepetk commented Jun 29, 2023

thepetk commented Jul 3, 2023 • edited Loading

thepetk commented Jul 7, 2023

michael-valdron commented Jul 7, 2023

thepetk commented Jul 10, 2023

michael-valdron commented Jul 10, 2023

thepetk commented Jul 11, 2023 • edited Loading

thepetk commented Jul 28, 2023 • edited Loading

thepetk commented Aug 2, 2023 • edited Loading

Multiple sample versions inside extraDevfileEntries

Separate Branch, Separate Sample Repo OR Directory with devfiles?

[Not Supported ATM] Separate branch per devfile sample repo

[Not Supported ATM] Directory inside each samples that will introduce different devfile versions

Use different repo per devfile.yaml version

thepetk commented Aug 2, 2023 • edited Loading

thepetk commented Aug 4, 2023

thepetk commented Sep 1, 2023

thepetk commented Sep 25, 2023

thepetk commented Sep 28, 2023 • edited Loading

Branch naming

Branch rules

Update Contributing Guide

thepetk commented Oct 11, 2023

Branch naming

Branch rules

Update Contributing Guide

thepetk commented Oct 16, 2023 • edited Loading

Update for multiple sample versions

Related PRs

We already use port 8080 in python-basic, java-springboot-basic and dotnet-basic

We need to add protection to the version branches before merging

thepetk commented Oct 23, 2023

thepetk commented Oct 30, 2023

thepetk commented Nov 2, 2023 • edited Loading

thepetk commented Nov 7, 2023

New scope

Default Version = Default Branch

Latest Version = v2.x.x Branch

Updated extraDevfileEntries.yaml

thepetk commented Nov 8, 2023

thepetk commented Nov 13, 2023

thepetk commented Nov 27, 2023 • edited Loading

thepetk commented Nov 28, 2023

kim-tsao commented May 2, 2023 •

edited by thepetk

Loading

thepetk commented May 15, 2023 •

edited

Loading

thepetk commented Jun 26, 2023 •

edited

Loading

thepetk commented Jun 28, 2023 •

edited

Loading

thepetk commented Jul 3, 2023 •

edited

Loading

thepetk commented Jul 11, 2023 •

edited

Loading

thepetk commented Jul 28, 2023 •

edited

Loading

thepetk commented Aug 2, 2023 •

edited

Loading

thepetk commented Aug 2, 2023 •

edited

Loading

thepetk commented Sep 28, 2023 •

edited

Loading

thepetk commented Oct 16, 2023 •

edited

Loading

We already use port 8080 in `python-basic`, `java-springboot-basic` and `dotnet-basic`

thepetk commented Nov 2, 2023 •

edited

Loading

thepetk commented Nov 27, 2023 •

edited

Loading