Skip to content

Commit

Permalink
[Feature Add]: Auto-snap-versioning (#516)
Browse files Browse the repository at this point in the history
* Add version automation code

* update action.yaml

* Update python dependencies

* update version_schema

* Update Readme

* update argument

* Separate manageYAML and Snapcraft in diff files

* Add unit-tests for snap version automation

* Corrected prevdevel & prevstable also removed auto grade change

* corrected gitcommitdate

* update path of snapping repo

* Add unittests for corresct version number

* Removed copy.copy() from test_correct_snap_version

---------

Co-authored-by: Rudra Pratap Singh <[email protected]>
  • Loading branch information
rudra-iitm and Rudra Pratap Singh authored Feb 1, 2024
1 parent e024199 commit 8ed3b7d
Show file tree
Hide file tree
Showing 12 changed files with 457 additions and 107 deletions.
1 change: 1 addition & 0 deletions .github/workflows/updatesnaptests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@ jobs:
python3 -m pip install pyyaml
python3 -m pip install python-debian
python3 -m pip install packaging
python3 -m pip install gitpython
- name: Code tests
env:
GITHUB_USER: ubuntu
Expand Down
32 changes: 32 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,14 @@ For example, to run it locally on another repo (gnome-calculator in this case) t
./updatesnap/updatesnapyaml.py --github-user GITHUB_USER --github-token GITHUB_TOKEN https://github.com/ubuntu/gnome-calculator.git
```

This tool can also be used to automate version updates of snap based on a specified version schema.
When the `--version-schema` (optional) flag is provided as input, the tool will automatically increment the version according to the specified schema.

To include this feature as a github worflow you need to pass an optional input in `with` command.

```
./updatesnap/updatesnapyaml.py --github-user GITHUB_USER --github-token GITHUB_TOKEN --version-schema VERSION_SCHEMA https://github.com/ubuntu/gnome-calculator.git
```
### GitHub action
This action should be utilized by other repos' workflows. The action checks out this repository to use updatesnapyaml.py and replaces the old snapcraft.yaml with the new one.

Expand All @@ -63,3 +71,27 @@ jobs:
token: ${{ secrets.GITHUB_TOKEN }}
repo: ${{ github.repository }}
```

For example, to use snap version automation
```
name: Push new tag update to stable branch
on:
schedule:
# Daily for now
- cron: '9 7 * * *'
workflow_dispatch:
jobs:
update-snapcraft-yaml:
runs-on: ubuntu-latest
steps:
- name: Checkout this repo
uses: actions/checkout@v3
- name: Run desktop-snaps action
uses: ubuntu/desktop-snaps@stable
with:
token: ${{ secrets.GITHUB_TOKEN }}
repo: ${{ github.repository }}
version-schema: '^debian/(\d+\.\d+\.\d+)'
```
31 changes: 27 additions & 4 deletions action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,10 @@ inputs:
description: 'The repo containing the snapcraft.yaml to be updated'
required: true
default: 'None'
version-schema:
description: 'Version schema of snapping repository'
required: false
default: 'None'

# Outputs generated by running this action.
outputs:
Expand All @@ -22,6 +26,7 @@ outputs:
# Global env var that can keep track of if there was a change or not. This determines if there are commits to be pushed.
env:
IS_CHANGE: false
IS_VERSION_CHANGE: false

# The jobs description defining a composite action
runs:
Expand All @@ -36,12 +41,22 @@ runs:
ref: stable
path: desktop-snaps

- name: Install dependencies
run: |
python3 -m pip install --upgrade pip
python3 -m pip install gitpython
shell: bash

# Step to run the script that will generate a new output_file with an updated tag, if one is available. If there was a change, then we move this to the snapcraft.yaml and note that there was a change.
- name: run updatesnapyaml
id: updatesnapyaml
run: |
./desktop-snaps/updatesnap/updatesnapyaml.py --github-user $GITHUB_USER --github-token $GITHUB_TOKEN https://github.com/${{ github.repository }}
./desktop-snaps/updatesnap/updatesnapyaml.py --github-user $GITHUB_USER --github-token $GITHUB_TOKEN --version-schema $VERSION_SCHEMA https://github.com/${{ github.repository }}
# Make sure to put the updated snapcraft.yaml file in the right location if it lives in a snap directory
if [ -f version_file ]; then
echo "IS_VERSION_CHANGE=true" >> $GITHUB_ENV
rm version_file
fi
if [ -f output_file ]; then
echo "IS_CHANGE=true" >> $GITHUB_ENV
if [ -d snap ]; then
Expand All @@ -53,6 +68,7 @@ runs:
env:
GITHUB_USER: ubuntu
GITHUB_TOKEN: ${{ inputs.token }}
VERSION_SCHEMA: ${{ inputs.version-schema }}
shell: bash

# Step to remove the desktop-snaps folder so that when we commit changes in another repo, the desktop-snaps folder is not committed there.
Expand All @@ -63,17 +79,24 @@ runs:

# If there was a change detected, then let's commit the changes
- name: Commit files
if: ${{ env.IS_CHANGE }}
if: ${{ env.IS_CHANGE || env.IS_VERSION_CHANGE}}
run: |
set -x
git config --local user.email "41898282+github-actions[bot]@users.noreply.github.com"
git config --local user.name "github-actions[bot]"
git commit -a -m "Update tag"
commit_msg="Update tag"
if [ $IS_VERSION_CHANGE = true ] && [ $IS_CHANGE = false ]; then
commit_msg="Update snap version"
fi
if [ $IS_VERSION_CHANGE = true ] && [ $IS_CHANGE = true ]; then
commit_msg="Update snap version & update tag"
fi
git commit -a -m "$commit_msg"
shell: bash

# If there was a change detected, then let's push the changes
- name: Push changes
if: ${{ env.IS_CHANGE }}
if: ${{ env.IS_CHANGE || env.IS_VERSION_CHANGE }}
uses: ad-m/github-push-action@master
env:
GITHUB_USER: ubuntu
Expand Down
116 changes: 116 additions & 0 deletions updatesnap/SnapModule/manageYAML.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,116 @@
from typing import Optional

class ManageYAML:
""" This class takes a YAML file and splits it in an array with each
block, preserving the child structure to allow to re-create it without
loosing any line. This can't be done by reading it with the YAML module
because it deletes things like comments. """
def __init__(self, yaml_data: str):
self._original_data = yaml_data
self._tree = self._split_yaml(yaml_data.split('\n'))[1]

def _split_yaml(self, contents: str, level: int = 0, clevel: int = 0,
separator: str = ' ') -> tuple[list, str]:
""" Transform a YAML text file into a tree
Splits a YAML file in lines in a format that preserves the structure,
the order and the comments. """

data = []
while len(contents) != 0:
if len(contents[0].lstrip()) == 0 or contents[0][0] == '#':
if data[-1]['child'] is None:
data[-1]['child'] = []
data[-1]['child'].append({'separator': '',
'data': contents[0].lstrip(),
'child': None,
'level': clevel + 1})
contents = contents[1:]
continue
if not contents[0].startswith(separator * level):
return contents, data
if level == 0:
if contents[0][0] == ' ' or contents[0][0] == '\t':
separator = contents[0][0]
if contents[0][level] != separator:
data.append({'separator': separator * level,
'data': contents[0].lstrip(),
'child': None,
'level': clevel})
contents = contents[1:]
continue
old_level = level
while contents[0][level] == separator:
level += 1
contents, inner_data = self._split_yaml(contents, level, clevel+1, separator)
level = old_level
if data[-1]['child'] is None:
data[-1]['child'] = inner_data
else:
data[-1]['child'] += inner_data
return [], data

def get_part_data(self, part_name: str) -> Optional[dict]:
""" Returns all the entries of an specific part of the current
YAML file. For example, the 'glib' part from a YAML file
with several parts. It returns None if that part doesn't
exist """

for entry in self._tree:
if entry['data'] != 'parts:':
continue
if ('child' not in entry) or (entry['child'] is None):
continue
for entry2 in entry['child']:
if entry2['data'] != f'{part_name}:':
continue
return entry2['child']
return None

def get_part_element(self, part_name: str, element: str) -> Optional[dict]:
""" Returns an specific entry for an specific part in the YAML file.
For example, it can returns the 'source-tag' entry of the part
'glib' from a YAML file with several parts. """

part_data = self.get_part_data(part_name)
if part_data:
for entry in part_data:
if entry['data'].startswith(element):
return entry
return None

def _get_yaml_group(self, group):
data = ""
for entry in group:
data += entry['separator']
data += entry['data']
data += '\n'
if entry['child']:
data += self._get_yaml_group(entry['child'])
return data

def get_yaml(self) -> str:
""" Returns the YAML file updated with the new versions """
data = self._get_yaml_group(self._tree)
data = data.rstrip()
if data[-1] != '\n':
data += '\n'
return data

def get_metadata(self) -> Optional[dict]:
""" Returns metadata in form of list """
data = []
for entry in self._tree:
if entry['data'] == 'part':
continue
data.append(entry)
return data

def get_part_metadata(self, element: str) -> Optional[dict]:
""" Returns specific element of the metadata"""
metadata = self.get_metadata()
if metadata:
for entry in metadata:
if entry['data'].startswith(element):
return entry
return None
129 changes: 32 additions & 97 deletions updatesnap/SnapModule/snapmodule.py
Original file line number Diff line number Diff line change
Expand Up @@ -662,7 +662,8 @@ def process_part(self, part: str) -> Optional[dict]:
"use_tag": False,
"missing_format": False,
"updates": [],
"version_format": {}
"version_format": {},
"source_url": None,
}

if self._config is None:
Expand Down Expand Up @@ -705,6 +706,7 @@ def process_part(self, part: str) -> Optional[dict]:
if "format" not in version_format:
part_data['missing_format'] = True
source = data['source']
part_data["source_url"] = source

if ((not source.startswith('http://')) and
(not source.startswith('https://')) and
Expand Down Expand Up @@ -888,100 +890,33 @@ def _sort_elements(self, part, current_version, elements, text):
for element in newer_elements:
self._print_message(part, f" {element}\n")

def process_metadata(self) -> Optional[dict]:
""" Returns metadata from Snapcraft.yaml file """
metadata = {
"name": None,
"version": None,
"adopt-info": None,
"grade": None,
"upstream-version": None,
"upstream-url": None,
}

class ManageYAML:
""" This class takes a YAML file and splits it in an array with each
block, preserving the child structure to allow to re-create it without
loosing any line. This can't be done by reading it with the YAML module
because it deletes things like comments. """
def __init__(self, yaml_data: str):
self._original_data = yaml_data
self._tree = self._split_yaml(yaml_data.split('\n'))[1]

def _split_yaml(self, contents: str, level: int = 0, clevel: int = 0,
separator: str = ' ') -> tuple[list, str]:
""" Transform a YAML text file into a tree
Splits a YAML file in lines in a format that preserves the structure,
the order and the comments. """

data = []
while len(contents) != 0:
if len(contents[0].lstrip()) == 0 or contents[0][0] == '#':
if data[-1]['child'] is None:
data[-1]['child'] = []
data[-1]['child'].append({'separator': '',
'data': contents[0].lstrip(),
'child': None,
'level': clevel + 1})
contents = contents[1:]
continue
if not contents[0].startswith(separator * level):
return contents, data
if level == 0:
if contents[0][0] == ' ' or contents[0][0] == '\t':
separator = contents[0][0]
if contents[0][level] != separator:
data.append({'separator': separator * level,
'data': contents[0].lstrip(),
'child': None,
'level': clevel})
contents = contents[1:]
continue
old_level = level
while contents[0][level] == separator:
level += 1
contents, inner_data = self._split_yaml(contents, level, clevel+1, separator)
level = old_level
if data[-1]['child'] is None:
data[-1]['child'] = inner_data
else:
data[-1]['child'] += inner_data
return [], data

def get_part_data(self, part_name: str) -> Optional[dict]:
""" Returns all the entries of an specific part of the current
YAML file. For example, the 'glib' part from a YAML file
with several parts. It returns None if that part doesn't
exist """

for entry in self._tree:
if entry['data'] != 'parts:':
continue
if ('child' not in entry) or (entry['child'] is None):
continue
for entry2 in entry['child']:
if entry2['data'] != f'{part_name}:':
continue
return entry2['child']
return None

def get_part_element(self, part_name: str, element: str) -> Optional[dict]:
""" Returns an specific entry for an specific part in the YAML file.
For example, it can returns the 'source-tag' entry of the part
'glib' from a YAML file with several parts. """

part_data = self.get_part_data(part_name)
if part_data:
for entry in part_data:
if entry['data'].startswith(element):
return entry
return None

def _get_yaml_group(self, group):
data = ""
for entry in group:
data += entry['separator']
data += entry['data']
data += '\n'
if entry['child']:
data += self._get_yaml_group(entry['child'])
return data

def get_yaml(self) -> str:
""" Returns the YAML file updated with the new versions """
data = self._get_yaml_group(self._tree)
data = data.rstrip()
if data[-1] != '\n':
data += '\n'
return data
if self._config is None:
return None
data = self._config
if 'name' in data:
metadata['name'] = data['name']

if 'version' in data:
metadata['version'] = data['version']

if 'adopt-info' in data:
metadata['adopt-info'] = data['adopt-info']
upstream_data = self.process_part(data['adopt-info'])
metadata['upstream-url'] = upstream_data['source_url']
if len(upstream_data['updates']) != 0:
metadata['upstream-version'] = upstream_data['updates'][0]

if 'grade' in data:
metadata['grade'] = data['grade']
return metadata
Empty file.
Loading

0 comments on commit 8ed3b7d

Please sign in to comment.