Values pushed to pushgateway are being modified

[rajkamalhm]

Bug Report

What did you do?

cat <<EOF | curl -k --data-binary @- https://test-pushgateway.net/metrics/job/testmetric
# TYPE testmetric counter
testmetric 1654052803456764534
EOF

What did you expect to see?

curl -s -k "https://test-prometheus.net/api/v1/query?query=testmetric"
{
  "status": "success",
  "data": {
    "resultType": "vector",
    "result": [
      {
        "metric": {
          "__name__": "testmetric",
          "exported_job": "testmetric",
          "instance": "<redacted>:9091",
          "job": "pushgateway"
        },
        "value": [
          1657105962.683,
          "1654052803456764534"
        ]
      }
    ]
  }
}

What did you see instead? Under which circumstances?

curl -s -k "https://test-prometheus.net/api/v1/query?query=testmetric"
{
  "status": "success",
  "data": {
    "resultType": "vector",
    "result": [
      {
        "metric": {
          "__name__": "testmetric",
          "exported_job": "testmetric",
          "instance": "<redacted>:9091",
          "job": "pushgateway"
        },
        "value": [
          1657105962.683,
          "1654052803456764400"
        ]
      }
    ]
  }
}

Environment

• System information:

Linux 4.15.0-96-generic x86_64

• Pushgateway version:

pushgateway, version 1.3.0 (branch: HEAD, revision: c28992c985ce6c4fcf4247ba9736b72a3d43882f)
  build user:       root@69165ae53e
  build date:       20201001-12:03:34
  go version:       go1.15.2
  platform:         linux/amd64

The last 3 digits of the value are being modified by pushgateway. Tested on both untyped and counter type metrics and the results are the same.

EDIT:
Results from pushgateway

{
  "time_stamp": "2022-07-06T11:37:20.864270713Z",
  "type": "COUNTER",
  "metrics": [
    {
      "labels": {
        "instance": "",
        "job": "testmetric"
      },
      "value": "1.6540528034567644e+18"
    }
  ]
}

[beorn7]

Prometheus uses float64 for all metric values internally. This gives you 53 bits of precision, which "ought to be enough for anyone". ;) But it is indeed less than what a 64 bit integer gives you.

There are very few valid use cases where the 53 bits of precision aren't sufficient. A discussion around if and how to address those rare use cases has happened for as long as Prometheus exists. Maybe Prometheus 3.x will address it. For the time being, the Pushgateway is just following along with the Prometheus data model.

In your example above, you are pushing a very large integer, larger than 2^53, which explains the rounding you can see.