Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Initiating Nautobot jobs, next-2.0 rebase #270

Merged
merged 29 commits into from
Mar 13, 2024
Merged
Show file tree
Hide file tree
Changes from 11 commits
Commits
Show all changes
29 commits
Select commit Hold shift + click to select a range
c17166c
copy init-jobs changes fom #224 with rebase to next-2.0 branch
Sep 26, 2023
67d545d
Rebased PR for next-2.0 branch, had some trouble with fork
Sep 26, 2023
e01dbe6
Correct change fragment, #270 will replace #224, and closes #223
Sep 26, 2023
99e4512
Add kwargs for get_jobs() that allows user-specified header items to …
Sep 26, 2023
0d623ef
Minor update
Sep 26, 2023
0f28144
Catch job_result.result == "FAILURE"
Sep 26, 2023
9d99ca7
run black
Sep 26, 2023
4d0d877
pydocstyle, flake8 fixes
Sep 26, 2023
fe07fc3
black formatter
Sep 26, 2023
1c8da1a
pylint
Sep 26, 2023
3fc453a
remove runnable references
Sep 26, 2023
d7e4ccc
reuse existing user instance (dispatch.user)
Sep 27, 2023
1423b8c
typo correction for dispatcher.user
Sep 27, 2023
50725bb
Replace list comp with generator
smk4664 Sep 29, 2023
42e1931
Add init-job-form nautobot subcommand
Oct 31, 2023
f074f92
flake8 fix, catch explicit exception
Oct 31, 2023
6910196
Remove block response, leaving testing commented
Oct 31, 2023
98c2db3
pylint8 fixes
Oct 31, 2023
9a259a5
CI history expired, re-running checks
Feb 5, 2024
8dd56fe
pylint CI fixes: W1113, R0914, C0301
Feb 5, 2024
195c888
Black code formatter
Feb 6, 2024
28c1d95
pylint ignore too-many-locals for init_job
Feb 6, 2024
cb26441
Replace execute_job with enqueue_job, which is correct Nautobot 2.x p…
Feb 6, 2024
64ed936
Fix missing variable
Feb 6, 2024
00eb275
Update breadcrumb changes
Feb 6, 2024
7470ec8
Add max iterations for waiting on job to enter ready state in the dat…
Feb 6, 2024
0ab7e78
Rename `init_job` subcommand to `run_job`
Feb 6, 2024
875ca64
Update error for initiated job status failure state
Feb 7, 2024
e6c14c6
Rub Black formatter
Feb 7, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions changes/270.added
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
Add init_jobs Nautobot subcommand.
Add kwargs input to init_jobs as JSON-string to init_jobs.
Add get_jobs Nautobot subcommand, which returns all Nautobot jobs viewable to user.
Add filter_jobs Nautobot subcommand, which returns filtered set of Nautobot jobs viewable to user.
140 changes: 139 additions & 1 deletion nautobot_chatops/workers/nautobot.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,9 @@
"""Worker functions for interacting with Nautobot."""

import json


from django.contrib.auth import get_user_model
from django.core.exceptions import ValidationError
from django.db.models import Count
from django.contrib.contenttypes.models import ContentType
Expand All @@ -10,7 +14,7 @@
from nautobot.dcim.models import Device, DeviceType, Location, LocationType, Manufacturer, Rack, Cable
from nautobot.ipam.models import VLAN, Prefix, VLANGroup
from nautobot.tenancy.models import Tenant
from nautobot.extras.models import Role, Status
from nautobot.extras.models import Job, JobResult, Role, Status

from nautobot_chatops.choices import CommandStatusChoices
from nautobot_chatops.workers import subcommand_of, handle_subcommands
Expand Down Expand Up @@ -1045,6 +1049,140 @@ def get_circuit_providers(dispatcher, *args):
return CommandStatusChoices.STATUS_SUCCEEDED


@subcommand_of("nautobot")
def filter_jobs(dispatcher, job_filters: str = ""): # We can use a Literal["enabled", "installed"] here instead
"""Get a filtered list of jobs from Nautobot that the request user have view permissions for.

Args:
job_filters (str): Filter job results by literals in a comma-separated string.
Available filters are: enabled, installed.
"""
# Check for filters in user supplied input
job_filters_list = [item.strip() for item in job_filters.split(",")] if isinstance(job_filters, str) else ""
filters = ["enabled", "installed"]
if any([key in job_filters for key in filters]):
filter_args = {key: True for key in filters if key in job_filters_list}
jobs = Job.objects.restrict(dispatcher.user, "view").filter(**filter_args) # enabled=True, installed=True
else:
jobs = Job.objects.restrict(dispatcher.user, "view").all()

header = ["Name", "ID", "Enabled"]
rows = [
(
str(job.name),
str(job.id),
str(job.enabled),
)
for job in jobs
]

dispatcher.send_large_table(header, rows)

return CommandStatusChoices.STATUS_SUCCEEDED


@subcommand_of("nautobot")
def get_jobs(dispatcher, kwargs: str = ""):
"""Get all jobs from Nautobot that the requesting user have view permissions for.

Args:
kwargs (str): JSON-string array of header items to be exported.
"""
# Confirm kwargs is valid JSON
json_args = ["Name", "Id", "Enabled"]
try:
if kwargs:
json_args = json.loads(kwargs)
except json.JSONDecodeError:
dispatcher.send_error(f"Invalid JSON-string, cannot decode: {kwargs}")
return (CommandStatusChoices.STATUS_FAILED, f"Invalid JSON-string, cannot decode: {kwargs}")

jobs = Job.objects.restrict(dispatcher.user, "view").all()

# Check if all items in json_args are valid keys (assuming all keys of job object are valid)
valid_keys = [attr for attr in dir(Job) if not callable(getattr(Job, attr)) and not attr.startswith("_")]
for item in json_args:
if item not in valid_keys:
dispatcher.send_error(f"Invalid item provided: {item}")
return (CommandStatusChoices.STATUS_FAILED, f"Invalid item provided: {item}")

# TODO: Check json_args are all valid keys
header = [item.capitalize() for item in json_args]
rows = [(tuple(str(getattr(job, item, "")) for item in json_args)) for job in jobs]

dispatcher.send_large_table(header, rows)

return CommandStatusChoices.STATUS_SUCCEEDED


@subcommand_of("nautobot")
def init_job(dispatcher, job_name: str, kwargs: str = ""):
"""Initiate a job in Nautobot by job name.

Args:
job_name (str): Name of Nautobot job to run.
kwargs (str): JSON-string dictionary for input keyword arguments for job run.
#profile (str): Whether to profile the job execution.
"""
# Confirm kwargs is valid JSON
json_args = {}
try:
if kwargs:
json_args = json.loads(kwargs)
except json.JSONDecodeError:
dispatcher.send_error(f"Invalid JSON-string, cannot decode: {kwargs}")
return (CommandStatusChoices.STATUS_FAILED, f"Invalid JSON-string, cannot decode: {kwargs}")

profile = False
if json_args.get("profile") and json_args["profile"] is True:
profile = True

# Get instance of the user who will run the job
user = get_user_model()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You do not need to get the user model, or get the user, as dispatcher.user is already a user instance.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks, pushed d7e4ccc

try:
user_instance = user.objects.get(username=dispatcher.user)
except user.DoesNotExist: # Unsure if we need to check this case?
dispatcher.send_error(f"User {dispatcher.user} not found")
return (CommandStatusChoices.STATUS_FAILED, f'User "{dispatcher.user}" was not found')

# Get the job model instance using job name
try:
job_model = Job.objects.restrict(dispatcher.user, "view").get(name=job_name)
except Job.DoesNotExist:
dispatcher.send_error(f"Job {job_name} not found")
return (CommandStatusChoices.STATUS_FAILED, f'Job "{job_name}" was not found')

if not job_model.enabled:
dispatcher.send_error(f"The requested job {job_name} is not enabled")
return (CommandStatusChoices.STATUS_FAILED, f'Job "{job_name}" is not enabled')

job_class_path = job_model.class_path

# TODO: Check if json_args keys are valid for this job model
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It might also be a good idea to ensure the user has permissions to run the job.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

does the restrict() a few lines up do this? I can test to confirm

job_model = Job.objects.restrict(dispatcher.user, "view").get(name=job_name)```

job_result = JobResult.execute_job(
job_model=job_model,
user=user_instance,
profile=profile,
**json_args,
)

if job_result and job_result.status == "FAILURE":
dispatcher.send_error(f"The requested job {job_name} failed to initiate. Result: {job_result.result}")
return (CommandStatusChoices.STATUS_FAILED, f'Job "{job_name}" failed to initiate. Result: {job_result.result}')
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is checking if the task failed, but then sending a message that the task failed to initiate. The task was initiated but failed.

Copy link
Contributor Author

@MeganerdDev MeganerdDev Feb 7, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point, I adjusted the phrasing in 875ca64, but let me know if you have anything specific in mind maybe


# TODO: need base-domain, this yields: /extras/job-results/<job_id>/
job_url = job_result.get_absolute_url()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can use dispatcher.context['request_schema'] and dispatcher.context['request_host'] to get the base-domain. But this won't work in Slack Socket Mode. I added the SLACK_SOCKET_STATIC_HOST to solve that issue for static urls, but since static could point to S3, we would need another solution here.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

noted, will explore this

blocks = [
dispatcher.markdown_block(
f"The requested job {job_class_path} was initiated! [`click here`]({job_url}) to open the job."
),
]

dispatcher.send_blocks(blocks)

return CommandStatusChoices.STATUS_SUCCEEDED


@subcommand_of("nautobot")
def about(dispatcher, *args):
"""Provide link for more information on Nautobot Apps."""
Expand Down