Skip to content

Commit

Permalink
python: use file inventory modification time to identify non-ingested…
Browse files Browse the repository at this point in the history
… telem
  • Loading branch information
joseph-long committed May 18, 2024
1 parent b2205b6 commit f8495dd
Showing 1 changed file with 10 additions and 2 deletions.
12 changes: 10 additions & 2 deletions python/magaox/db/ingest.py
Original file line number Diff line number Diff line change
Expand Up @@ -82,8 +82,16 @@ def identify_non_ingested_telem(cur: psycopg.Cursor, host: str) -> list[str]:
LEFT JOIN file_ingest_times fit ON
fi.origin_host = fit.origin_host AND
fi.origin_path = fit.origin_path
WHERE fit.origin_host IS NULL AND
fit.origin_path IS NULL AND
WHERE
(
(
fit.origin_host IS NULL AND
fit.origin_path IS NULL
) OR (
fit.ingested_at < fi.modification_time
)
)
AND
fi.origin_host = %s AND
fi.origin_path LIKE '%%.bintel'
;
Expand Down

0 comments on commit f8495dd

Please sign in to comment.