Merge "doc: further expand filter options & lshw." into latest

redhat-performance · Oct 9, 2024 · 05f0d2d · 05f0d2d
2 parents 3fdea3f + f5f176e
commit 05f0d2d
Show file tree

Hide file tree

Showing 4 changed files with 99 additions and 13 deletions.
diff --git a/README.md b/README.md
@@ -283,7 +283,13 @@ systemctl restart nginx
    - QUADS relies on calling an external script, trigger or workflow to enact the actual provisioning of machines. You can look at and modify our [move-and-rebuild-hosts](https://github.com/redhat-performance/quads/blob/latest/src/quads/tools/move_and_rebuild.py) tool to suit your environment for this purpose.  Read more about this in the [move-host-command](https://github.com/redhat-performance/quads#quads-move-host-command) section below.
 
 ### Making QUADS Run
-   - QUADS is a passive service and does not do anything you do not tell it to do.  We control QUADS with cron, please copy and modify our [example cron commands](https://raw.githubusercontent.com/redhat-performance/quads/latest/cron/quads) to your liking, adjust as needed.
+   - QUADS is a passive service and does not do anything you do not tell it to do.  We control QUADS with [cron commands](https://raw.githubusercontent.com/redhat-performance/quads/latest/cron/quads).
+   - We ship an example cron file and install it for you that should work out of the box, it just has entries commented out.
+   - To enable QUADS services to run you'll need to **uncomment them**.
+
+```
+crontab -e
+```
 
 #### Major Components
 

diff --git a/cron/quads b/cron/quads
@@ -22,7 +22,7 @@ SHELL=/bin/bash
 ##### CONTAINER QUADS SERVER ENTRIES
 ##### ONLY USE WITH CONTAINER DEPLOYMENT
 ########################################
-# * * * * * ( echo "=== Move and Rebuild == @ " $(date) ; flock -n /tmp/moveandrebuild.lock -c "podman exec quads quads --move-hosts" ) 1>>/var/log/move-and-rebuild.log 2>&1
+# * * * * * flock -n /tmp/moveandrebuild.lock -c "echo '=== Move and Rebuild == Started @ ' $(date) ; podman exec quads quads --move-hosts ; echo '=== Move and Rebuild == Finished @ ' $(date)" 1>/dev/null 2>&1
 # * * * * * ( echo "=== Validating == @ " $(date) ; flock -n /tmp/validateenv.lock -c "podman exec quads quads --validate-env" ) 1>/dev/null 2>&1
 # 0 0 * * * ( echo "=== Sending Notifications == @ " $(date) ; flock -n /tmp/notify.lock -c "podman exec quads quads --notify" ) 1>/dev/null 2>&1
 # 0 0 * * * ( echo "=== Regenerate Heat Map == @ " $(date) ; flock -n /tmp/heatmap.lock -c "podman exec quads quads --regen-heatmap" ) 1>/dev/null 2>&1
@@ -35,7 +35,7 @@ SHELL=/bin/bash
 ##### (RECOMMENDED)
 #########################################
 #########################################
-#*/5 * * * * ( echo "=== Move and Rebuild == @ " $(date) ; flock -n /tmp/moveandrebuild.lock -c "quads --move-hosts" ) 1>>/var/log/move-and-rebuild.log 2>&1
+#*/5 * * * * flock -n /tmp/moveandrebuild.lock -c "echo '=== Move and Rebuild == Started @ ' $(date) ; quads --move-hosts ; echo '=== Move and Rebuild == Finished @ ' $(date)" 1>>/var/log/move-and-rebuild.log 2>&1
 #*/25 * * * * ( echo "=== Validating == @ " $(date) ; flock -n /tmp/validateenv.lock -c "quads --validate-env" ) 1>>/var/log/validate-env.log 2>&1
 #*/10 * * * * ( echo "=== Sending Notifications == @ " $(date) ; flock -n /tmp/notify.lock -c "quads --notify" ) 1>>/var/log/quads-notify.log 2>&1
 #*/20 * * * * ( echo "=== Regenerate Heat Map == @ " $(date) ; flock -n /tmp/heatmap.lock -c "quads --regen-heatmap" ) 1>/dev/null 2>&1

diff --git a/docs/quads-host-metadata-search.md b/docs/quads-host-metadata-search.md
@@ -26,20 +26,21 @@ In QUADS `1.1.4` and above we've implemented a metadata model in the QUADS datab
   * We can use the popular [lshw](https://linux.die.net/man/1/lshw) tool to gather hardware details into JSON
   * We ship a tool called `lshw2meta.py` to transform this into a format for updating host metadata into QUADS.
 
-First, install `lshw` on your target host(s)
+First, install `lshw` on your target QUADS-managed host(s)
 
 ```
 dnf install lshw
 ```
 
-Next run `lshw` to capture all the hardware details of your host in JSON.
+Next run `lshw` to capture all the hardware details of each remote host in JSON.
 
 ```
 lshw -json > $(hostname).json
 ```
 
-Next, copy the JSON file(s) over to your QUADS host `quads:/opt/quads/lshw`
-Now on your QUADS host use the `lshw2meta.py` tool to convert this data and import it directly into the QUADS database for each host.
+Next, copy the JSON file(s) over to your QUADS host here: `quads:/opt/quads/lshw/`
+
+Now, back on your QUADS host use the `lshw2meta.py` tool to convert this data and import it directly into the QUADS database for each host.
 
 ```
 python3 /usr/lib/python3.12/site-packages/quads/tools/lshw2meta.py
@@ -51,7 +52,7 @@ python3 /usr/lib/python3.12/site-packages/quads/tools/lshw2meta.py
   * This assumes all of your hosts are in `cloud01` and powered on and accessible
   * This assumes you have `lshw` installed as well on every remote host
 
-First, gather all of the JSON metadata from the hosts:
+First, gather all of the JSON metadata from your remote QUADS-managed host(s):
 ```
 python3 /usr/lib/python3.12/site-packages/quads/tools/lshw.py
 ```
@@ -92,61 +93,107 @@ quads --export-host-details /tmp/my_host_data.yml
 ```
 
 ## Querying Host Information
-  * A new sub-command of `--filter` has been added to the `--ls-available` and `--ls-hosts` commands.
+  * The sub-command `--filter` can be used with `--ls-available` and `--ls-hosts` commands.
 
-| Component              | Field Type | Syntax                       | Operators       |
+| Component              | Field Type | Description                  | Operators       |
 |------------------------|------------|------------------------------|-----------------|
-| model                  |  string    | exact match                  | ==,!=           |
+| model                  |  string    | defined system model         | ==,!=           |
 | disks.size_gb          |  integer   | disk size in GB              | ==,!=,<,<=,>,>= |
 | disks.disks_type       |  string    | nvme,sata,ssd                | ==,!=           |
 | disks.count            |  integer   | number of disks              | ==,!=,<,<=,>,>= |
 | interfaces__size       |  integer   | number of interfaces         | ==,!=,<,<=,>,>= |
 | interfaces.name        |  string    | name of interface            | ==,!=           |
 | interfaces.mac_address |  string    | mac address                  | ==,!=           |
 | interfaces.switch_port |  string    | switch port                  | ==,!=           |
+| interfaces.switch_ip   |  string    | switch ip address per port   | ==,!=           |
 | interfaces.speed       |  integer   | link speed                   | ==,!=,<,<=,>,>= |
 | interfaces.vendor      |  string    | interface vendor             | ==,!=           |
 | interfaces.maintenance |  boolean   | interface maintenance status | ==,!=           |
+| interfaces.bios_id     |  string    | interface BIOS boot devname  | ==,!=           |
 | build                  |  boolean   | build status                 | ==,!=           |
 | validated              |  boolean   | validated status             | ==,!=           |
 | broken                 |  boolean   | broken status                | ==,!=           |
 | retired                |  boolean   | retired status               | ==,!=           |
-| switch_config_applied  |  boolean   | switch configuration status  | ==,!=           |
-
+| switch_config_applied  |  boolean   | host switch config status    | ==,!=           |
+| memory.handle          |  string    | DIMM details                 | ==,!=           |
+| memory.size_gb         |  integer   | amount of system memory      | ==,!=,<,<=,>,>= |
+| processors.handle      |  string    | CPU details                  | ==,!=           |
+| processors.vendor      |  string    | CPU vendor information       | ==,!=           |
+| processors.product     |  string    | CPU model information        | ==,!=           |
+| processors.cores       |  integer   | CPU cores in the system      | ==,!=,<,<=,>,>= |
+| processors.threads     |  integer   | CPU threads in the system    | ==,!=,<,<=,>,>= |
 
 ### Example Filter Searches
   * Accepted operators are `==, !=, <, <=, >, >=`
+  * For the REST API the operators must be appended to the key like so `disks.size_gb__gte=2000`
+
+| Operator | Appended |
+|----------|----------|
+| ==       |          |
+| !=       | __ne     |
+| <        | __lt     |
+| <=       | __lte    |
+| >        | __gt     |
+| >=       | __gte    |
+| like     | __like   |
+| ilike    | __ilike  |
+| in       | __in     |
 
 #### Example Hardware Filter Searches
 
   * Search for systems with disk type NVMe, with a size of 2TB or more and available between `2020-07-20 17:00` and `2020-07-22 13:00`
 
 ```
 quads --ls-available --schedule-start "2020-07-20 17:00" --schedule-end "2020-07-22 13:00" --filter "disks.disk_type==nvme,disks.size_gb>=2000"
+```
+  * Via the API
+
+```
+curl https://quads.example.com/api/v3/available?start=2020-07-20T17:00&end=2020-07-22T13:00&disks.disk_type=nvme&disks.size_gb__gte=2000
 ```
 
   * Search for systems with SATA disks available from `2020-07-20 17:00` until `2020-07-22 13:00`
 
 ```
 quads --ls-available --schedule-start "2020-07-20 17:00" --schedule-end "2020-07-22 13:00" --filter "disks.disk_type==sata"
+```
+  * Via the API
+
+```
+curl https://quads.example.com/api/v3/available?start=2020-07-20T17:00&end=2020-07-22T13:00&disks.disk_type=sata
 ```
 
   * Search for systems of model type `1029U-TRTP` available from `2020-07-20 17:00` until `2020-07-22 13:00`
 
 ```
 quads --ls-available --schedule-start "2020-07-20 17:00" --schedule-end "2020-07-22 13:00" --filter "model==1029U-TRTP"
+```
+  * Via the API
+
+```
+curl https://quads.example.com/api/v3/available?start=2020-07-20T17:00&end=2020-07-22T13:00&model=1029U-TRTP
 ```
 
   * Search for systems with **two NVMe** disks **and** disk size of **more than** 2TB, available from `2020-07-20 17:00` until `2020-07-22 13:00`
 
 ```
 quads --ls-available --schedule-start "2020-07-20 17:00" --schedule-end "2020-07-22 13:00" --filter "disks.disk_type==nvme,disks.count>2, disks.size_gb<2000"
+```
+  * Via the API
+
+```
+curl https://quads.example.com/api/v3/available?start=2020-07-20T17:00&end=2020-07-22T13:00&disks.disk_type=nvme&disks.count__gt=2&disks.size_gb__lt=2000
 ```
 
   * Search all systems by model and number of interfaces.
 
 ```
 quads --ls-hosts --filter "model==FC640,interfaces__size==5"
+```
+  * Via the API
+
+```
+curl https://quads.example.com/api/v3/hosts?model=FC640&interfaces__size=5
 ```
 
 #### Example Network Filter Searches
@@ -156,20 +203,35 @@ quads --ls-hosts --filter "model==FC640,interfaces__size==5"
 
 ```
 quads --ls-hosts --filter "interfaces.mac_address==ac:1f:6b:2d:19:48"
+```
+  * Via the API
+
+```
+curl https://quads.example.com/api/v3/hosts?interfaces.mac_address=ac:1f:6b:2d:19:48
 ```
 
   * Find hosts by switch IP address.
   * Shows all hosts connected to a particular switch
 
 ```
 quads --ls-host --filter "interfaces.ip_address==10.1.34.216"
+```
+  * Via the API
+
+```
+curl https://quads.example.com/api/v3/hosts?interfaces.ip_address=10.1.34.216
 ```
 
   * Find hosts by physical switchport
   * Shows all hosts that have a specific physical switchport name
 
 ```
 quads --ls-host --filter "interfaces.switch_port==et-0/0/7:1"
+```
+  * Via the API
+
+```
+curl https://quads.example.com/api/v3/hosts?interfaces.switch_port=et-0/0/7:1
 ```
 
 ##### Combined Network Search Example
@@ -179,6 +241,11 @@ quads --ls-host --filter "interfaces.switch_port==et-0/0/7:1"
 
 ```
 quads --ls-hosts --filter "interfaces.ip_address==10.1.34.216,interfaces.switch_port==et-0/0/7:1"
+```
+  * Via the API
+
+```
+curl https://quads.example.com/api/v3/hosts?interfaces.ip_address=10.1.34.216&interfaces.switch_port=et-0/0/7:1
 ```
 
 ## Querying Host Status
@@ -191,10 +258,20 @@ quads --ls-hosts --filter "interfaces.ip_address==10.1.34.216,interfaces.switch_
 
 ```
 quads --ls-hosts --filter "retired==True"
+```
+  * Via the API
+
+```
+curl https://quads.example.com/api/v3/hosts?retired=True
 ```
 
   * List retired hosts and filter by model
 
 ```
 quads --ls-hosts --filter "retired==True,model==1029P"
+```
+  * Via the API
+
+```
+curl https://quads.example.com/api/v3/hosts?retired=True&model=1029P
 ```
diff --git a/rpm/quads.spec b/rpm/quads.spec
@@ -116,6 +116,7 @@ cp -rf systemd/quads-server.service %{buildroot}/etc/systemd/system/
 cp -rf systemd/quads-web.service %{buildroot}/etc/systemd/system/
 cp -rf systemd/quads-db.service %{buildroot}/etc/systemd/system/
 cp -rf conf/logrotate_quads.conf %{buildroot}/etc/logrotate.d/
+cp -rf cron/quads %{buildroot}/opt/quads/conf/quads.cron.example
 cp -rf container/etc/nginx/conf.d/apiv3.conf %{buildroot}/etc/nginx/conf.d/
 cp -rf container/etc/nginx/conf.d/apiv3_ssl.conf.example %{buildroot}/etc/nginx/conf.d/
 cp -rf container/etc/postfix/postfix-files.d/quads.cf %{buildroot}/etc/postfix/postfix-files.d/
@@ -135,6 +136,7 @@ rm -rf %{buildroot}
 /etc/profile.d/quads.sh
 /etc/nginx/*
 /opt/quads/conf/logrotate_quads.conf
+/opt/quads/conf/quads.cron.example
 /usr/bin/quads
 %config(noreplace) /opt/quads/conf/quads.yml
 %config(noreplace) /opt/quads/conf/vlans.yml
@@ -188,6 +190,7 @@ if [ "$1" -eq 1 ]; then
     sed -i 's/ident/password/g' /opt/quads/db/data/pg_hba.conf
     /usr/bin/systemctl start quads-db
     /usr/bin/systemctl start haveged
+    /usr/bin/crontab /opt/quads/conf/quads.cron.example
     cd /var/lib/pgsql && sudo -u postgres psql -c "ALTER USER postgres WITH PASSWORD 'postgres';"
     /usr/bin/systemctl start nginx
 fi