Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feat add xdmod supremm pcp #8

Open
wants to merge 20 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
20 commits
Select commit Hold shift + click to select a range
f19c675
Use netw. var names used in gen-vars.py
eatluri Dec 20, 2019
a2604ff
Use build_version var name that's used in gen-vars.
eatluri Dec 20, 2019
3293c92
Use ssh vars consistent with gen-vars.py variables
eatluri Dec 20, 2019
999bfda
Add CRI_Cluster_Monitor submodule.
eatluri Dec 26, 2019
122873e
Update .gitmodules file to include new submodule.
eatluri Dec 26, 2019
380ec4e
Update CRI_XCBC submodule to the latest commit.
eatluri Dec 26, 2019
b0058ef
Create a dir to upload Clustermon ansible scripts
eatluri Dec 27, 2019
4b1acea
Upload CRI_Cluster_Monitor dir with ansible scripts
eatluri Dec 27, 2019
d1dc637
Run clustermon.yml ansible playbook
eatluri Dec 27, 2019
cc8077a
Update Cluster_Monitor submodule to latest commit
eatluri Dec 28, 2019
51f9075
Use site-build.yaml playbook that contains build roles.
eatluri Dec 31, 2019
cb32f8d
Merge branch 'patch-fix-packer-build-playbook-compute' into patch-fix…
eatluri Dec 31, 2019
0b67d01
Merge branch 'patch-fix-vars-in-build-file' into feat-add-xdmod-supre…
eatluri Dec 31, 2019
8784273
Add branch tracking to the submodules
eatluri Jan 22, 2020
a186a71
Open XDMoD initial readme check in.
eatluri Jan 23, 2020
d38e77b
Add usage instrx to build imgs with xdmod,supremm.
eatluri Jan 23, 2020
40d2a4f
Formatting changes to xdmod readme
eesaanatluri Jan 23, 2020
f389d9e
Update .gitmodules file to include new submodule.
eatluri Dec 26, 2019
b2df134
update to latest commit in submodule CRI_Cluster_Monitor
eesaanatluri May 30, 2023
7411cac
Merge branch 'master' into feat-add-xdmod-supremm-pcp
eesaanatluri May 30, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .gitmodules
Original file line number Diff line number Diff line change
@@ -1,3 +1,8 @@
[submodule "CRI_XCBC"]
path = CRI_XCBC
url = https://github.com/jprorama/CRI_XCBC.git

[submodule "CRI_Cluster_Monitor"]
path = CRI_Cluster_Monitor
url = https://github.com/eesaanatluri/CRI_Cluster_Monitor.git

1 change: 1 addition & 0 deletions CRI_Cluster_Monitor
Submodule CRI_Cluster_Monitor added at 15ab55
2 changes: 1 addition & 1 deletion CRI_XCBC
52 changes: 52 additions & 0 deletions XDMOD-README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
## Usage

1. To build images to spin up a basic OHPC cluster with Open XDMoD and SUPReMM do the following.

`git clone [email protected]:eesaanatluri/ohpc-packer.git`

`cd ohpc-packer`

`git checkout feat-add-xdmod-supremm-pcp`

`cd CRI_XCBC/ `

`git checkout feat-add-pcp-on-compute`

`cd ../CRI_Cluster_Monitor/`

`git checkout feat-add-open-xdmod`

`cd ../`

To build a compute image for OHPC cluster do the following

`packer build -var-file=vars.json compute-openstack.json`

To build an OHPC master image do the following
> change the build name in vars.json file to ohpc.

`packer build -var-file=vars.json build.json`

2. If you want to build an image containing XDMoD and SUPReMM with data from prod
- Put the cheaha slurm accounting database dump (slurmdb_backup_20191014_032723.tar.gz) in the ohpc-packer repo directory.
- Put the hierarchy and input files archive (hierarchy-files.tar.gz) in the ohpc-packer repo directory.

`cd CRI_XCBC/`

`git checkout patch-fix-slurmdb-history`

`cd ../CRI_Cluster_Monitor/`

`git checkout feat-add-hierarchies-open-xdmod`

`cd ../`

To build a compute image for OHPC cluster do the following

`packer build -var-file=vars.json compute-openstack.json`

To build an OHPC master image do the following
> change the build name in vars.json file to ohpc.

`packer build -var-file=vars.json build.json`

20 changes: 14 additions & 6 deletions build.json
Original file line number Diff line number Diff line change
Expand Up @@ -2,14 +2,14 @@
"builders": [{
"type": "openstack",
"source_image_name": "{{ user `source_image_name` }}",
"networks": [ "{{ user `external-net-ID` }}","{{ user `internal-net-ID` }}"],
"instance_floating_ip_net": "{{ user `external-net-ID` }}",
"networks": [ "{{ user `external-net` }}","{{ user `internal-net` }}"],
"instance_floating_ip_net": "{{ user `external-net` }}",
"instance_name": "{{ user `build_instance_name` }}",
"image_name": "{{ user `build_instance_name` }}-V{{user `img_build_version`}}",
"image_name": "{{ user `build_instance_name` }}-V{{user `build_version`}}",
"ssh_username": "{{ user `ssh_username` }}",
"ssh_keypair_name": "{{ user `ssh_keypair_name` }}",
"ssh_private_key_file": "{{ user `private_key_file-path` }}",
"ssh_host": "{{ user `ssh_host-IP` }}",
"ssh_private_key_file": "{{ user `private_key_file` }}",
"ssh_host": "{{ user `ssh_host` }}",
"reuse_ips": true,
"flavor": "{{ user `flavor` }}"
}],
Expand All @@ -18,6 +18,7 @@
"type": "shell",
"inline": [
"sudo mkdir -p /CRI_XCBC && sudo chown {{ user `ssh_username` }}: /CRI_XCBC",
"sudo mkdir -p /CRI_Cluster_Monitor && sudo chown {{ user `ssh_username` }}: /CRI_Cluster_Monitor",
"sudo yum install -y epel-release",
"sudo yum install -y ansible git vim bash-completion",
"sudo yum install -y NetworkManager",
Expand All @@ -31,11 +32,18 @@
"source": "CRI_XCBC/",
"destination": "/CRI_XCBC/"
},

{
"type": "file",
"source": "CRI_Cluster_Monitor/",
"destination": "/CRI_Cluster_Monitor/"
},

{
"type": "shell",
"inline": [
"sudo ansible-playbook -c local -i /CRI_XCBC/hosts -l `hostname -s` /CRI_XCBC/site-build.yaml -b"
"sudo ansible-playbook -c local -i /CRI_XCBC/hosts -l `hostname -s` /CRI_XCBC/site-build.yaml -b",
"sudo ansible-playbook -c local -i /CRI_XCBC/hosts -l `hostname -s` /CRI_Cluster_Monitor/clustermon.yml -b"
]
}
]
Expand Down
2 changes: 1 addition & 1 deletion compute-openstack.json
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@
},
{
"type": "ansible-local",
"playbook_file": "CRI_XCBC/site.yaml",
"playbook_file": "CRI_XCBC/site-build.yaml",
"playbook_dir": "CRI_XCBC",
"inventory_groups": "compute",
"extra_arguments": [ "-b" ]
Expand Down