Skip to content

Commit

Permalink
[HUD] Look for queued jobs in past month only (#6166)
Browse files Browse the repository at this point in the history
Make results for queued_jobs_by_label more similar
* Change queued_jobs_by_label to only look at jobs in the past weekly
like the queued_jobs query
* Use job.created_at instead of started_at for queued_jobs_by_label. I
don't think there's a difference for jobs that are queued, only jobs
that are running or completed

This helps resolve a problem where a job showed up in one HUD table but
not in the other
  • Loading branch information
clee2000 authored Jan 13, 2025
1 parent 9d37fcf commit adf0025
Showing 1 changed file with 4 additions and 2 deletions.
6 changes: 4 additions & 2 deletions torchci/clickhouse_queries/queued_jobs_by_label/query.sql
Original file line number Diff line number Diff line change
@@ -1,9 +1,11 @@
--- This query is used by HUD metrics page to get the list of queued jobs grouped by their labels
WITH possible_queued_jobs as (
select id, run_id from default.workflow_job where status = 'queued'
AND created_at < (CURRENT_TIMESTAMP() - INTERVAL 5 MINUTE)
AND created_at > (CURRENT_TIMESTAMP() - INTERVAL 1 WEEK)
), queued_jobs AS (
SELECT
DATE_DIFF('second', job.started_at, CURRENT_TIMESTAMP()) AS queue_s,
DATE_DIFF('second', job.created_at, CURRENT_TIMESTAMP()) AS queue_s,
CONCAT(workflow.name, ' / ', job.name) AS name,
job.html_url,
IF(
Expand All @@ -27,7 +29,7 @@ WITH possible_queued_jobs as (
and workflow.id in (select run_id from possible_queued_jobs)
and workflow.repository. 'full_name' = 'pytorch/pytorch'
AND job.status = 'queued'
AND job.started_at < (CURRENT_TIMESTAMP() - INTERVAL 5 MINUTE)
AND job.created_at < (CURRENT_TIMESTAMP() - INTERVAL 5 MINUTE)
/* These two conditions are workarounds for GitHub's broken API. Sometimes */
/* jobs get stuck in a permanently "queued" state but definitely ran. We can */
/* detect this by looking at whether any steps executed (if there were, */
Expand Down

0 comments on commit adf0025

Please sign in to comment.