Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Conda env setup #364

Draft
wants to merge 88 commits into
base: master
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
88 commits
Select commit Hold shift + click to select a range
4198967
lnls-ans-role-conda: Create group and set permissions
carneirofc Jan 31, 2022
6228535
lnls-ans-role-conda-packages: Initial commit
carneirofc Jan 31, 2022
0e2effb
conda role with a bit more functionality
carneirofc Feb 1, 2022
3a284a6
lnls-ans-role-epics: Feature for enabling EPICS Python packages
carneirofc Feb 8, 2022
23fb4e8
lnls-ans-role-conda: Include /etc/profile.d/ template
carneirofc Feb 8, 2022
f8e6c74
lnls-ans-role-epics: Fix include_role
carneirofc Feb 8, 2022
4dc27d3
lnls-ans-role-conda-packages: Update readme
carneirofc Feb 8, 2022
6572881
lnls-ans-role-conda-packages: Update molecule settings
carneirofc Feb 8, 2022
8cb4ea3
lnls-ans-role-conda-packages: Include defaults with description
carneirofc Feb 8, 2022
47a4963
lnls-ans-role-conda-packages: Include lnls-ans-role-epics as dependency
carneirofc Feb 8, 2022
32fb53a
lnls-ans-role-conda-packages: Handy shell script for molecule
carneirofc Feb 8, 2022
5210899
lnls-ans-role-conda-packages: Include jinja templates for activate/de…
carneirofc Feb 8, 2022
4e92eea
lnls-ans-role-conda-packages: Include gnupg and apt https packages fo…
carneirofc Feb 8, 2022
444001c
lnls-ans-role-conda-packages: tasks related to the conda environment
carneirofc Feb 8, 2022
e9ca3b9
lnls-ans-role-conda-packages: Testing group_vars/
carneirofc Feb 8, 2022
4b8d1c4
inventories/sirius: Include conda group with fixed gid
carneirofc Feb 8, 2022
a509cd4
inventories/sirius: update eth-bridge-pru-serial url
carneirofc Feb 8, 2022
90e99b2
library: move conda library from lnls-ans-sirius-hlacon to roles/conda
carneirofc Feb 9, 2022
2c8e022
lnls-ans-role-conda-packages: Fix ansible templates
carneirofc Feb 9, 2022
b73fbde
lnls-ans-role-conda-packages: Remove hardcoded path
carneirofc Feb 9, 2022
a4ae6f5
lnls-ans-role-conda-packages: Fix git target name
carneirofc Feb 9, 2022
d7cf4d8
lnls-ans-role-conda-packages: Update Vagrant variables
carneirofc Feb 9, 2022
2639466
lnls-ans-role-sirius-hla: remove python references, we should be usin…
carneirofc Feb 9, 2022
1429d40
playbooks: Removed deprecated roles
carneirofc Feb 9, 2022
3b5e9a7
inventory/sirius: fac conda variables
carneirofc Feb 9, 2022
367de17
inventory/sirius: Sort variables
carneirofc Feb 9, 2022
88c2537
make: remove python and qt roles
carneirofc Feb 9, 2022
5866ee2
lnls-ans-role-pydm: remove qt role dependency
carneirofc Feb 9, 2022
4659360
lnls-ans-role-pydm: remove qt role dependency
carneirofc Feb 9, 2022
1b849ba
lnls-ans-role-sirius-apps: remove deprecated dependencies
carneirofc Feb 9, 2022
158cd98
lnls-ans-role-conda-packages: Better variable default
carneirofc Feb 9, 2022
60da5ad
lnls-ans-role-sirius-hlacon: Default python and pip to version 3
carneirofc Feb 9, 2022
11029dc
playbooks: Fix playbooks and taks related to conda
carneirofc Feb 10, 2022
6b513f4
merge
carneirofc Feb 10, 2022
3aa5a2e
Modline for yaml
carneirofc Feb 10, 2022
0bdf269
vagrant: Include general usage vagrant setup
carneirofc Feb 15, 2022
5309153
vagrant: Update vagrant settings, virtualbox recommended
carneirofc Feb 15, 2022
5951ed0
lnls-ans-role-cs-studio: remove debug variable
carneirofc Feb 15, 2022
ecf64a6
lnls-ans-role-network: ignore default_release if not defined
carneirofc Feb 15, 2022
2313d59
lnls-ans-role-sirius-bbb: Install rsync dependency
carneirofc Feb 15, 2022
670eae5
lnls-ans-role-sirius-bbb: Fix default mode
carneirofc Feb 16, 2022
1b4ef39
inventories/sirius: Move jupyter from conda packages to pip
carneirofc Feb 18, 2022
4ab42ed
inventories/sirius: Remove fixed pkg_ versions
carneirofc Feb 18, 2022
8119441
make: Include test_vagrant_deploy target
carneirofc Feb 18, 2022
2a790bf
tasks: Change loop var
carneirofc Feb 18, 2022
52d82d0
playbooks: Include kitty terminfo task
carneirofc Feb 18, 2022
38e882a
lnls-ans-role-conda: Include mamba; update requirements
carneirofc Feb 18, 2022
8dcae9f
lnls-ans-role-conda-packages: Using mamba as package manager
carneirofc Feb 18, 2022
f64dd90
lnls-ans-role-sirius-hlacon: mamba package manager, fix pip executabl…
carneirofc Feb 18, 2022
bb6dc6c
Update README
carneirofc Feb 18, 2022
fe1c4f4
Minor fixes
carneirofc Feb 18, 2022
544894b
lnls-ans-role-sirius-hlacon: FIX ansible_python_interpreter var
carneirofc Feb 21, 2022
7e7372f
Merge remote-tracking branch 'origin/master' into fac-conda-env
carneirofc Feb 21, 2022
79e29a7
inventories/sirius: sort settings
carneirofc Feb 21, 2022
885de39
playbooks: FIX role name
carneirofc Feb 21, 2022
0f15a75
inventories/sirius: Include python 2.7 environment
carneirofc Feb 21, 2022
bea56ba
lnls-ans-role-conda-packages: python-sirius symlink
carneirofc Feb 21, 2022
fd32487
lnls-ans-role-sirius-apps: Deprecate pip usaged
carneirofc Feb 21, 2022
ff9bbe9
lnls-ans-role-sirius-hla: Default apt state "present" setup-Debiant
carneirofc Feb 21, 2022
423abde
Update README
carneirofc Feb 21, 2022
f1b046c
lnls-ans-role-desktop-apps: Include ampty pkg_ version vars
carneirofc Feb 21, 2022
b328428
lnls-ans-role-repositories: comment out debian9 apt sources from debi…
carneirofc Feb 21, 2022
1be6357
lnls-ans-role-sirius-hlacon: FIX missing and wrong vars
carneirofc Feb 21, 2022
9cbbb63
lnls-ans-role-epics: Include install from source option
carneirofc Feb 21, 2022
d1df754
lnls-ans-role-repositories: Remove debian stretch...
carneirofc Feb 21, 2022
6e16dd7
inventories/sirius: move hlacon into the global conda setup
carneirofc Feb 21, 2022
bfb157b
Merge branch 'fac-conda-env' of https://github.com/lnls-sirius/lnls-a…
carneirofc Feb 21, 2022
1dae78f
lnls-ans-role-epics: Give the option to install EPICS from source
carneirofc Feb 22, 2022
63216cc
Update all
carneirofc Feb 22, 2022
6f61b8c
Update main.yml
carneirofc Feb 22, 2022
ded8d9a
lnls-ans-role-conda-packages: Include conda-forge when creating a new…
carneirofc Feb 23, 2022
e9486d7
lnls-ans-role-repositories: Remove NSLS2 key from bullseye
carneirofc Feb 23, 2022
7c4467a
Merge branch 'fac-conda-env' of https://github.com/lnls-sirius/lnls-a…
carneirofc Feb 23, 2022
53aea30
lnls-ans-role-conda-packages: FIX set file mode to 775 instead of 755…
carneirofc Feb 23, 2022
26b3df5
playbooks: Include conda and epics install playbook
carneirofc Feb 23, 2022
779ecf7
Remove blank spaces
carneirofc Feb 23, 2022
90b919f
lnls-ans-role-sirius-hla: Include bullseye vars file
carneirofc Feb 24, 2022
d25c43c
FIX: Remove dependencies from global python
carneirofc Feb 24, 2022
7d05c13
roles: trim whitespace
carneirofc Feb 24, 2022
c19fda6
FIX: CI
carneirofc Feb 25, 2022
b80d571
inventories/sirius: Update conda package and python version
carneirofc Feb 25, 2022
ab4037b
README: Include a real world examples
carneirofc Feb 25, 2022
e937f96
inventories/sirius: Update rf host list
carneirofc Feb 25, 2022
fdb24d1
inventories/sirius: Include localhost group
carneirofc Feb 25, 2022
9203130
tasks: dist variables task
carneirofc Feb 25, 2022
2007128
CI: Update ansible-lint version
carneirofc Feb 25, 2022
032542c
tasks: FIX var-spacing
carneirofc Feb 25, 2022
1998db4
Update README.md
carneirofc Feb 25, 2022
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
54 changes: 54 additions & 0 deletions .github/workflows/ansible-lint.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
name: Ansible Lint # feel free to pick your own name

on: [push, pull_request]

jobs:
build:

runs-on: ubuntu-latest

steps:
# Important: This sets up your GITHUB_WORKSPACE environment variable
- uses: actions/checkout@v2

- name: Lint Ansible Playbook
# replace "master" with any valid ref
uses: ansible/ansible-lint-action@master
with:
# [required]
# Paths to ansible files (i.e., playbooks, tasks, handlers etc..)
# or valid Ansible directories according to the Ansible role
# directory structure.
# If you want to lint multiple ansible files, use the following syntax
# targets: |
# playbook_1.yml
# playbook_2.yml
targets: ""
# [optional]
# Arguments to override a package and its version to be set explicitly.
# Must follow the example syntax.
override-deps: |
ansible-lint==5.3.2
# [optional]
# Arguments to be passed to the ansible-lint

# Options:
# -q quieter, although not silent output
# -p parseable output in the format of pep8
# --parseable-severity parseable output including severity of rule
# -r RULESDIR specify one or more rules directories using one or
# more -r arguments. Any -r flags override the default
# rules in ansiblelint/rules, unless -R is also used.
# -R Use default rules in ansiblelint/rules in addition to
# any extra
# rules directories specified with -r. There is no need
# to specify this if no -r flags are used
# -t TAGS only check rules whose id/tags match these values
# -x SKIP_LIST only check rules whose id/tags do not match these
# values
# --nocolor disable colored output
# --exclude=EXCLUDE_PATHS
# path to directories or files to skip. This option is
# repeatable.
# -c C Specify configuration file to use. Defaults to ".ansible-lint"
args: ""
6 changes: 0 additions & 6 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -22,9 +22,3 @@ repos:
additional_dependencies:
- flake8-absolute-import
- flake8-black

- repo: https://github.com/ansible-community/ansible-lint.git
rev: v5.3.2
hooks:
- id: ansible-lint
files: \.(yaml|yml)$
50 changes: 50 additions & 0 deletions .vagrant-boxes/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
# Vagrant Testing

This Vagrant directory contains the necessary infrastructure to test
ansible roles in Vagrant boxes.

Four environments are provided: Debian Stretch, Debian Jessie, Ubuntu Trusty
and Ubuntu Xenial. It is recommended to use the virtualbox provider `--provider=virtualbox` but other options are available.

## Instructions

Start vagrant boxes by issuing:

```bash
vagrant up
```

Or by selecting only one of them:

```bash
vagrant up debian.stretch
```

debugging
```bash
VAGRANT_LOG=debug vagrant up --provider=<provider-name>
```

Run a basic ansible playbook to configure the SSH keys on the vagrant box:

```bash
ansible-playbook -i hosts -u vagrant setup.yml
```

Or by selecting only one of them:

```bash
ansible-playbook -i hosts -l debian.stretch -u vagrant setup.yml
```

Run the playbook agains the boxes:

```bash
ansible-playbook -i hosts -u vagrant ../molecule/default/playbook.yml
```

Or by selecting only one of them:

```bash
ansible-playbook -i hosts -l debian.stretch -u vagrant ../molecule/default/playbook.yml
```
47 changes: 47 additions & 0 deletions .vagrant-boxes/Vagrantfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
# -*- mode: ruby -*-
# vi: set ft=ruby :
#
# https://www.vagrantup.com/docs/boxes/base


VAGRANTFILE_API_VERSION = "2"

hosts = {
"debian.bullseye" => "192.168.56.10", # Debian 11
"debian.stretch" => "192.168.56.11", # Debian 9
}

BOX_MEMORY = 2024 # 2GB
BOX_CPUS = 2

Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|
hosts.each do |name, ip|
config.vm.define name do |machine|

# Box providers
config.vm.provider :libvirt do |vb|
vb.memory = BOX_MEMORY
vb.cpus = BOX_CPUS
end

machine.vm.provider "virtualbox" do |vb|
vb.customize ["modifyvm", :id, "--cpuexecutioncap", "50"]
vb.gui = false
vb.name = name
vb.memory = BOX_MEMORY
vb.cpus = BOX_CPUS
end

# Configure vagrant box
if name.eql? "debian.bullseye" then
machine.vm.box = "debian/bullseye64"
elsif name.eql? "debian.stretch" then
machine.vm.box = "debian/stretch64"
end

# Box network settings
machine.vm.hostname = "%s" % name
machine.vm.network :private_network, ip: ip
end
end
end
1 change: 1 addition & 0 deletions .vagrant-boxes/inventory/group_vars/all.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
---
1 change: 1 addition & 0 deletions .vagrant-boxes/inventory/host_vars/all.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
---
5 changes: 5 additions & 0 deletions .vagrant-boxes/inventory/hosts
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
debian.bullseye ansible_host=192.168.56.10 # Debian 11
debian.stretch ansible_host=192.168.56.11 # Debian 9

[all:vars]
ansible_ssh_private_key_file=.vagrant/machines/{{ inventory_hostname }}/virtualbox/private_key
94 changes: 94 additions & 0 deletions .vagrant-boxes/setup.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,94 @@
---
- hosts: all
become: true
become_user: root
remote_user: sirius
gather_facts: true
tasks:
# This should work too / #2372
# - name: Pushes user key to root's on vagrant boxes
# action: authorized_key key=$FILE($item) user=root
# first_available_file:
# - ~/.ssh/id_dsa.pub
# - ~/.ssh/id_rsa.pub

- name: Wait for ssh to be up
become: false
wait_for:
port: 22
delay: 5
connect_timeout: 5
timeout: 360
host: "{{ ansible_host }}"
delegate_to: localhost

- name: Installs python for Debian based distros
apt:
name: python
state: present
become: true
register: apt_result
until: apt_result is success
retries: 5
delay: 2
when: ansible_os_family == 'Debian'

- name: Installs python for RedHat based distros
package:
name: python
state: present
become: true
register: package_result
until: package_result is success
retries: 5
delay: 2
when: ansible_os_family == 'RedHat'

- name: Creates destination directory
# Workaround for #2372
file:
state: directory
mode: 0700
dest: /root/.ssh/

- name: Pushes user's rsa key to root's vagrant box (it's ok if this TASK fails)
# action: authorized_key user=root key='$FILE(~/.ssh/id_rsa.pub)'
# Workaround for #2372
copy:
src: ~/.ssh/id_rsa.pub
dest: /root/.ssh/authorized_keys
owner: root
mode: 0600
register: rsa
ignore_errors: true

- name: Pushes user's dsa key to root's vagrant box (it's NOT ok if both TASKs fail)
# action: authorized_key user=root key='$FILE(~/.ssh/id_dsa.pub)'
# Workaround for #2372
copy:
src: ~/.ssh/id_dsa.pub
dest: /root/.ssh/authorized_keys
owner: root
mode: 0600
when: rsa is failed

- name: Checks if resolver is working properly (issues with some VBox/Host OS combinations)
command: host -t A ansible.cc
register: ns
ignore_errors: true

- name: Pushes new resolver configuration if resolver fails
lineinfile:
regexp: "^nameserver "
line: "nameserver 8.8.8.8"
dest: /etc/resolv.conf
when: ns is failed

- name: Checks if resolver is working properly with new nameserver
command: host -t A ansible.cc

- name: Final greeting
pause:
seconds: 1
echo: false
prompt: "Don't worry about all the red above; if you made it here, your Vagrant VMs are probably fine !"
2 changes: 2 additions & 0 deletions .yamllint
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
# -*- mode: yaml -*-
# vi: set ft=yaml :
---
# Based on ansible-lint config
extends: default
Expand Down
Loading