Improve version tracking and deprecate SVN support

[dsavransky]

Is your feature request related to a problem? Please describe.
Currently, the code attempts to determine the software version to help with reproducibility. In the case of installs from repositories, the commit information is also tracked. All of this currently happens in the SurveySimulation prototype.

Describe the solution you'd like
We need multiple updates:

• We should record python and package version as well as EXOSIMS version
• We should deprecate SVN support
• We should keep track git repos with local changes
• All of this should be handled by a utility that can be called from any place in the framework that needs it rather than living in the SurveySimulation prototype.

[VeronicaSergeeva]

So far, I created a utility file which has two functions to track the python versions. Here's my code:

import EXOSIMS
import platform
import pkg_resources
import subprocess

def get_version_info():
python_version = platform.python_version()
packages = {pkg.key: pkg.version for pkg in pkg_resources.working_set}

return python_version, packages

def get_git_info():
try:
commit_hash = subprocess.check_output(["git", "rev-parse", "HEAD"]).strip().decode('utf-8')
uncommitted_changes = subprocess.check_output(["git", "status", "--porcelain"]).strip().decode('utf-8')

    return commit_hash, uncommitted_changes
except Exception as e:
    print(f"Error getting git info: {e}")
    return None, None

[dsavransky]

@VeronicaSergeeva Note that pkg_resources is deprecated. All new code should use importlib instead, where possible. See the migration guide here: https://importlib-resources.readthedocs.io/en/latest/migration.html. Also potentially helpful: https://discuss.python.org/t/will-setuptools-remove-pkg-resources-module-in-the-future/27182/10

[VeronicaSergeeva]

Thank you, here's the version using importlib:

import platform
import importlib.metadata as metadata
import subprocess

def get_version_info():
python_version = platform.python_version()
packages = {dist.metadata["Name"]: dist.version for dist in metadata.distributions()}

return python_version, packages

def get_git_info():
try:
commit_hash = subprocess.check_output(["git", "rev-parse", "HEAD"]).strip().decode('utf-8')
uncommitted_changes = subprocess.check_output(["git", "status", "--porcelain"]).strip().decode('utf-8')

    return commit_hash, uncommitted_changes
except Exception as e:
    print(f"Error getting git info: {e}")
    return None, None

if name == "main":
python_version, packages = get_version_info()
print(f"Python Version: {python_version}")
print("Package Versions:")
for pkg, version in packages.items():
print(f"{pkg}: {version}")

commit_hash, uncommitted_changes = get_git_info()
print(f"Git Commit Hash: {commit_hash}")
if uncommitted_changes:
    print(f"Uncommitted Changes:\n{uncommitted_changes}")

[VeronicaSergeeva]

I think I have come up with the solution and have 2 fully working files. The first file, version_util.py, should be placed in the util folder, this script should output the python version, the EXOSIMS version, and all required packages for EXOSIMS also with versions. The second file, rec_check.py. This is also in a separate file, this checks that the requirements.txt file contains all the correct packages that are needed for EXOSIMS.
For this file, requirements.txt must of course already be downloaded in advance and be in the same folder as the script. If requirements.txt is missing something, it will point out an error and tell you what is missing. Otherwise it will display the names of the packages in turn and messages that they have been found. I will attach the code in the next comment

[VeronicaSergeeva]

version_util.py file:

from importlib import metadata
import platform

def get_version():
python_version = platform.python_version()
exosims_version = metadata.version('EXOSIMS')

reqs = metadata.distribution('EXOSIMS').requires
required_packages = [str(req) for req in reqs]

# Get installed versions of required packages
installed_packages = {dist.metadata['Name']: dist.version for dist in metadata.distributions()}

# Filter installed packages to those listed in requirements
relevant_packages = {pkg: installed_packages.get(pkg.split('>=')[0], "Not installed") for pkg in required_packages}

return {
    'Python': python_version,
    'EXOSIMS': exosims_version,
    'Packages': relevant_packages
}

version_info = get_version()
for key, value in version_info.items():
if isinstance(value, dict):
print(f'{key}:')
for sub_key, sub_value in value.items():
print(f' {sub_key}:'.ljust(25)+f'{sub_value}')
else:
print(f'{key}:'.ljust(25)+f'{value}')

[VeronicaSergeeva]

rec_chekc.py file:

from importlib import metadata
reqs = metadata.distribution('EXOSIMS').requires
required_packages = [str(req).split('>=')[0] for req in reqs]

with open('requirements.txt', 'r') as f:
lines = f.readlines()

for package in required_packages:
flag = False
for line in lines:
if package in line:
flag = True
assert flag, f'{package} not found in requirements.txt'
print(f'{package} found in requirements.txt')

[VeronicaSergeeva]

I have uploaded 2 working files on my fork - the SVN-free SurveySimulation file and version_util for tracking versions and git info. I pasted the working version_util code here too:

from importlib import metadata
import platform
import subprocess
import os

def get_git_info():
"""
Get Git information including commit hash and status of changes
"""
try:
# Check if we are in a Git repository
subprocess.run(['git', 'rev-parse'], stdout=subprocess.PIPE, stderr=subprocess.PIPE, check=True)

    # Get the current commit hash
    commit_hash = subprocess.check_output(['git', 'rev-parse', 'HEAD']).strip().decode('utf-8')

    # Check for uncommitted changes
    status = subprocess.check_output(['git', 'status', '--porcelain']).strip().decode('utf-8')

    uncommitted_changes = status != ''

    return commit_hash, uncommitted_changes
except subprocess.CalledProcessError:
    return None, False  # Not a Git repository

def is_editable_installation():
"""
Check if EXOSIMS is installed in editable mode
"""
try:
# Check if EXOSIMS is installed via pip in editable mode
site_packages = metadata.distribution('EXOSIMS').locate_file('').parent
editable_marker_file = os.path.join(site_packages, 'EXOSIMS.egg-link')
return os.path.exists(editable_marker_file)
except Exception:
return False

def get_version():
python_version = platform.python_version()

exosims_version = metadata.version('EXOSIMS')
editable = is_editable_installation()

if editable:
    print('EXOSIMS is installed in editable mode')
    exosims_version = f'{exosims_version} (editable'
    commit_hash, uncommitted_changes = get_git_info()
    if commit_hash is not None:
        exosims_version = f'{exosims_version} / commit {commit_hash}'
    if uncommitted_changes:
        exosims_version = f'{exosims_version} / warning! uncommited changes'
    exosims_version = f'{exosims_version})'
else:
    exosims_version = f'{exosims_version} (not editable)'

reqs = metadata.distribution('EXOSIMS').requires
required_packages = [str(req) for req in reqs]

# Get installed versions of required packages
installed_packages = {dist.metadata['Name']: dist.version for dist in metadata.distributions()}

# Filter installed packages to those listed in requirements
relevant_packages = {pkg: installed_packages.get(pkg.split('>=')[0], "Not installed") for pkg in required_packages}

return {
    'Python': python_version,
    'EXOSIMS': exosims_version,
    'Packages': relevant_packages
}

version_info = get_version()
for key, value in version_info.items():
if isinstance(value, dict):
print(f'{key}:')
for sub_key, sub_value in value.items():
print(f' {sub_key}:'.ljust(25)+f'{sub_value}')
else:
print(f'{key}:'.ljust(25)+f'{value}')

[VeronicaSergeeva]

The last version_info block is what calls the functions, so it's in the new SurveySimulation file