You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
We've had an issue where after a backward incompatible change in GitLab 16.0 in how it populates variables, DD_AGENT_HOST variable suddenly became an empty string, and all places where hot-shots is used to send metrics from CI stopped sending metrics, although there were no errors in logs, and it took a while to understand why things have broken. Because by reading logs, everything looked "normal".
I managed to figure it out by reading this comment:
We've had an issue where after a backward incompatible change in GitLab 16.0 in how it populates variables,
DD_AGENT_HOST
variable suddenly became an empty string, and all places wherehot-shots
is used to send metrics from CI stopped sending metrics, although there were no errors in logs, and it took a while to understand why things have broken. Because by reading logs, everything looked "normal".I managed to figure it out by reading this comment:
#197 (comment)
and logging the value of
DD_AGENT_HOST
in CI and seeing it's empty.Relevant piece of code:
hot-shots/lib/statsd.js
Line 61 in b442244
So it would be good to improve at least this particular aspect, but also in general the debugging is not always obvious, see #257
The text was updated successfully, but these errors were encountered: