Client functions neither work, nor error

[lyle-kargo]

Initialization

Initialize by the following:

const {
        METRICS_HOST: host, // telegraf
        METRICS_PORT: portStr, // 8125
        METRICS_ENV: env // development
    } = process.env;

    if (host && portStr && env) {
        const port = parseInt(portStr);
        logger.info(`initializing telegraf metrics at ${host}:${port} with env: ${env}`);
        return new TelegrafClient(host, port, env);
    } else {
        // i create mock client here in certain circumstances, not relevant here
    }

// new TelegrafClient(host,port,env):
constructor(host, port, env) {
        const prefix = `my-app_${env}_`;
        this.client = new StatsD({
            host,
            port,
            prefix,
            telegraf: true,
            errorHandler: function (error) {
                console.log("Socket errors caught here: ", error);
            },
        });
    }

Functions

Currently only trying the following:

incrementMetric(stat, tags) {
  this.client.increment(stat, tags, function(error, bytes) {
      //this only gets called once after all messages have been sent
      if (error) {
          console.error('Oh noes! There was an error:', error);
      } else {
          console.log('Successfully sent', bytes, 'bytes');
      }
  });
}

recordResponseTime(stat, endTime, startTime, tags) {
        this.client.timing(stat, (endTime - startTime), 0, tags, function(error, bytes) {
            //this only gets called once after all messages have been sent
            if (error) {
                console.error('Oh noes! There was an error:', error);
            } else {
                console.log('Successfully sent', bytes, 'bytes');
            }
        });
    }

Problem

This neither fails, nor runs. I have updated my node-modules/hot-shots code with the following:

const createUdpTransport = args => {
  console.log('udp');
  const socket = dgram.createSocket(args.udpSocketOptions);
  // do not block node from shutting down
  socket.unref();

  const dnsResolutionData = {
    timestamp: new Date(0),
    resolvedAddress: undefined
  };

  const sendUsingDnsCache = (callback, buf) => {
    const now = Date.now();
    if (dnsResolutionData.resolvedAddress === undefined || (now - dnsResolutionData.timestamp > args.cacheDnsTtl)) {
      dns.lookup(args.host, (error, address) => {
        if (error) {
          callback(error);
          return;
        }
        dnsResolutionData.resolvedAddress = address;
        dnsResolutionData.timestamp = now;
        socket.send(buf, 0, buf.length, args.port, dnsResolutionData.resolvedAddress, callback);
      });
    } else {
      console.log('worked!');
      socket.send(buf, 0, buf.length, args.port, dnsResolutionData.resolvedAddress, callback);
    }
  };

return {
    emit: socket.emit.bind(socket),
    on: socket.on.bind(socket),
    removeListener: socket.removeListener.bind(socket),
    send: function (buf, callback) {
      if (args.cacheDns) {
        sendUsingDnsCache(callback, buf);
      } else {
        callback(undefined, buf.length);
        console.log(args.port);
        console.log(args.host);
        socket.send(buf, 0, buf.length, args.port, args.host, callback);
      }
    },
    close: socket.close.bind(socket),
    unref: socket.unref.bind(socket)
  };

The console.log('worked!') IS reached, but at that point it's lost and unfortunately adding logs to the node.js core library under dgram doesn't actually make it's way into the code. (At least from my IDE, I wouldn't call myself a node expert, so I may be misunderstanding something).

You can also see that in the return statement of the socket, I've updated the send function so that it calls the callback. This is the only time I have been able to see the callback run.

Telegraf Configuration

[global_tags]
[agent]
  interval = "60s"
  round_interval = true
  metric_batch_size = 1000
  metric_buffer_limit = 10000
  collection_jitter = "0s"
  flush_interval = "10s"
  flush_jitter = "0s"
  precision = ""
  hostname = "192.xxx.0.xxx"
  omit_hostname = false
  debug = true
[[outputs.influxdb_v2]]
  urls = ["http://influxdb:8086"]

    ## API token for authentication.
  token = "my-token"

    ## Organization is the name of the organization you wish to write to; must exist.
  organization = "my-org"

    ## Destination bucket to write into.
  bucket = "my-bucket"
[[inputs.statsd]]
  protocol = "udp"
  service_address = ":8125"
  delete_gauges = true
  delete_counters = true
  delete_sets = true
  delete_timings = true
  interval = "10s"
  percentiles = [50,90,99]
  metric_separator = "_"
  parse_data_dog_tags = false
  allowed_pending_messages = 6000000
  percentile_limit = 1000

Notes

Telegraf, grafana, influxdb, and my application are all running on the same docker network.

If I run a udp thing like:
echo "foobar:1|c" | nc -u -w1 telegraf 8125 from my node application container, it functions as expected, and the information percolates to Grafana. So the e2e connection between the applications is established and working, this (seems to me at least) to be solely an issue with the client functions in hot-shots.

If you need anything else from me to help debug, please let me know, and any help is greatly appreciated! I've been banging my head against a wall for quite some time here.

[bdeitte]

Apologies but I tend to be the only person responding to these, and life got in the way here for quite awhile in looking at these. Doing some catchup this weekend- is this still an issue you are interested in a response on? Thanks.