Skip to content

merge current upstream state #8

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 35 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
35 commits
Select commit Hold shift + click to select a range
6f36adf
Update pg_long_running_transactions.go (#1092)
jyothikirant-sayukth Dec 22, 2024
5bb1702
Fix to replace dashes with underscore in the metric names (#1103)
aagarwalla-fx Dec 22, 2024
5145620
Bump github.com/prometheus/exporter-toolkit from 0.13.1 to 0.13.2 (#1…
dependabot[bot] Jan 1, 2025
bea2609
Checkpoint related columns in PG 17 have been moved from pg_stat_bgwr…
n-rodriguez Jan 1, 2025
ecb5ec5
Update common Prometheus files (#1090)
prombot Jan 7, 2025
9de4f19
Update common Prometheus files (#1110)
prombot Jan 7, 2025
7d4c278
Add Postgres 17 for CI test (#1105)
khiemdoan Jan 13, 2025
3acc479
Bump github.com/prometheus/common from 0.61.0 to 0.62.0 (#1118)
dependabot[bot] Feb 13, 2025
d85a771
Update common Prometheus files (#1124)
prombot Feb 14, 2025
072864d
pg_stat_statements PG17 (#1114)
NevermindZ4 Feb 15, 2025
9e42fc0
fix: handle pg_replication_slots on pg<13 (#1098)
michael-todorovic Feb 15, 2025
2ee2a8f
feat: add wait/backend to pg_stat_activity (#1106)
fgalind1 Feb 15, 2025
c3885e8
Export last replay age in replication collector (#1085)
bitfehler Feb 15, 2025
99e1b51
Bump github.com/prometheus/exporter-toolkit from 0.13.2 to 0.14.0 (#1…
dependabot[bot] Feb 15, 2025
4c170ed
Fix missing dsn sanitization for logging (#1104)
sysadmind Feb 15, 2025
8bb1a41
Skip pg_stat_checkpointer collector if pg<17 (#1112)
n-rodriguez Feb 20, 2025
51006ab
Prep for v0.17 (#1127)
sysadmind Feb 21, 2025
2869087
Fix: Handle incoming labels with invalid UTF-8 (#1131)
vancwo Feb 26, 2025
1e574cf
Release v0.17.1 (#1132)
SuperQ Feb 26, 2025
457b6fa
Bump github.com/prometheus/client_golang from 1.20.5 to 1.21.0 (#1133)
dependabot[bot] Mar 3, 2025
602302f
Update common Prometheus files (#1137)
prombot Mar 8, 2025
b0e61bf
Bump golang.org/x/net from 0.33.0 to 0.36.0 (#1138)
dependabot[bot] Mar 16, 2025
2ce65c3
Update common Prometheus files (#1140)
prombot Mar 23, 2025
fca2ad8
Update common Prometheus files (#1142)
prombot Mar 30, 2025
9e86f1e
Adds pg_stat_progress_vacuum collector (#1141)
ianbibby Apr 3, 2025
8d5ec4b
Update Go (#1147)
SuperQ Apr 3, 2025
43576ac
Bump github.com/prometheus/client_golang from 1.21.0 to 1.21.1 (#1144)
dependabot[bot] Apr 3, 2025
f8b7139
Bump github.com/prometheus/common from 0.62.0 to 0.63.0 (#1143)
dependabot[bot] Apr 3, 2025
5edc331
Record table only size bytes as well in addition to the total size by…
Sticksman Apr 18, 2025
6526065
Bump golang.org/x/net from 0.36.0 to 0.38.0 (#1152)
dependabot[bot] Apr 23, 2025
5ade81c
Update common Prometheus files (#1155)
prombot May 7, 2025
bd8a613
Bump github.com/prometheus/client_golang from 1.21.1 to 1.22.0 (#1154)
dependabot[bot] May 7, 2025
de42ef7
Export query itself together with queryId in stat_statement metrics (…
Delorien84 May 16, 2025
d1a957f
Update common Prometheus files (#1159)
prombot May 17, 2025
94e8399
Feat: Improve Error Handling for Server.Scrape (#1158)
BoweFlex May 29, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 3 additions & 2 deletions .circleci/config.yml
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ executors:
# This must match .promu.yml.
golang:
docker:
- image: cimg/go:1.23
- image: cimg/go:1.24

jobs:
test:
Expand All @@ -23,7 +23,7 @@ jobs:

integration:
docker:
- image: cimg/go:1.23
- image: cimg/go:1.24
- image: << parameters.postgres_image >>
environment:
POSTGRES_DB: circle_test
Expand Down Expand Up @@ -63,6 +63,7 @@ workflows:
- cimg/postgres:14.9
- cimg/postgres:15.4
- cimg/postgres:16.0
- cimg/postgres:17.0
- prometheus/build:
name: build
parallelism: 3
Expand Down
8 changes: 4 additions & 4 deletions .github/workflows/golangci-lint.yml
Original file line number Diff line number Diff line change
Expand Up @@ -26,14 +26,14 @@ jobs:
- name: Checkout repository
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
- name: Install Go
uses: actions/setup-go@41dfa10bad2bb2ae585af6ee5bb4d7d973ad74ed # v5.1.0
uses: actions/setup-go@0aaccfd150d50ccaeb58ebd88d36e91967a5f35b # v5.4.0
with:
go-version: 1.23.x
go-version: 1.24.x
- name: Install snmp_exporter/generator dependencies
run: sudo apt-get update && sudo apt-get -y install libsnmp-dev
if: github.repository == 'prometheus/snmp_exporter'
- name: Lint
uses: golangci/golangci-lint-action@971e284b6050e8a5849b72094c50ab08da042db8 # v6.1.1
uses: golangci/golangci-lint-action@1481404843c368bc19ca9406f87d6e0fc97bdcfd # v7.0.0
with:
args: --verbose
version: v1.61.0
version: v2.1.5
53 changes: 33 additions & 20 deletions .golangci.yml
Original file line number Diff line number Diff line change
@@ -1,23 +1,36 @@
---
version: "2"
linters:
enable:
- misspell
- revive

issues:
exclude-rules:
- path: _test.go
linters:
- errcheck

linters-settings:
errcheck:
exclude-functions:
# Never check for logger errors.
- (github.com/go-kit/log.Logger).Log
revive:
- misspell
- revive
settings:
errcheck:
exclude-functions:
- (github.com/go-kit/log.Logger).Log
revive:
rules:
- name: unused-parameter
severity: warning
disabled: true
exclusions:
generated: lax
presets:
- comments
- common-false-positives
- legacy
- std-error-handling
rules:
# https://github.com/mgechev/revive/blob/master/RULES_DESCRIPTIONS.md#unused-parameter
- name: unused-parameter
severity: warning
disabled: true
- linters:
- errcheck
path: _test.go
paths:
- third_party$
- builtin$
- examples$
formatters:
exclusions:
generated: lax
paths:
- third_party$
- builtin$
- examples$
2 changes: 1 addition & 1 deletion .promu.yml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
go:
# This must match .circle/config.yml.
version: 1.23
version: 1.24
repository:
path: github.com/prometheus-community/postgres_exporter
build:
Expand Down
2 changes: 1 addition & 1 deletion .yamllint
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
extends: default
ignore: |
ui/react-app/node_modules
**/node_modules

rules:
braces:
Expand Down
26 changes: 26 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,29 @@
## 0.17.1 / 2025-02-26

* [BUGFIX] Fix: Handle incoming labels with invalid UTF-8 #1131

## 0.17.0 / 2025-02-16

## What's Changed
* [ENHANCEMENT] Add Postgres 17 for CI test by @khiemdoan in https://github.com/prometheus-community/postgres_exporter/pull/1105
* [ENHANCEMENT] Add wait/backend to pg_stat_activity by @fgalind1 in https://github.com/prometheus-community/postgres_exporter/pull/1106
* [ENHANCEMENT] Export last replay age in replication collector by @bitfehler in https://github.com/prometheus-community/postgres_exporter/pull/1085
* [BUGFIX] Fix pg_long_running_transactions time by @jyothikirant-sayukth in https://github.com/prometheus-community/postgres_exporter/pull/1092
* [BUGFIX] Fix to replace dashes with underscore in the metric names by @aagarwalla-fx in https://github.com/prometheus-community/postgres_exporter/pull/1103
* [BIGFIX] Checkpoint related columns in PG 17 have been moved from pg_stat_bgwriter to pg_stat_checkpointer by @n-rodriguez in https://github.com/prometheus-community/postgres_exporter/pull/1072
* [BUGFIX] Fix pg_stat_statements for PG17 by @NevermindZ4 in https://github.com/prometheus-community/postgres_exporter/pull/1114
* [BUGFIX] Handle pg_replication_slots on pg<13 by @michael-todorovic in https://github.com/prometheus-community/postgres_exporter/pull/1098
* [BUGFIX] Fix missing dsn sanitization for logging by @sysadmind in https://github.com/prometheus-community/postgres_exporter/pull/1104

## New Contributors
* @jyothikirant-sayukth made their first contribution in https://github.com/prometheus-community/postgres_exporter/pull/1092
* @aagarwalla-fx made their first contribution in https://github.com/prometheus-community/postgres_exporter/pull/1103
* @NevermindZ4 made their first contribution in https://github.com/prometheus-community/postgres_exporter/pull/1114
* @michael-todorovic made their first contribution in https://github.com/prometheus-community/postgres_exporter/pull/1098
* @fgalind1 made their first contribution in https://github.com/prometheus-community/postgres_exporter/pull/1106

**Full Changelog**: https://github.com/prometheus-community/postgres_exporter/compare/v0.16.0...v0.17.0

## 0.16.0 / 2024-11-10

BREAKING CHANGES:
Expand Down
13 changes: 9 additions & 4 deletions Makefile.common
Original file line number Diff line number Diff line change
Expand Up @@ -61,7 +61,8 @@ PROMU_URL := https://github.com/prometheus/promu/releases/download/v$(PROMU_
SKIP_GOLANGCI_LINT :=
GOLANGCI_LINT :=
GOLANGCI_LINT_OPTS ?=
GOLANGCI_LINT_VERSION ?= v1.60.2
GOLANGCI_LINT_VERSION ?= v2.1.5
GOLANGCI_FMT_OPTS ?=
# golangci-lint only supports linux, darwin and windows platforms on i386/amd64/arm64.
# windows isn't included here because of the path separator being different.
ifeq ($(GOHOSTOS),$(filter $(GOHOSTOS),linux darwin))
Expand Down Expand Up @@ -156,9 +157,13 @@ $(GOTEST_DIR):
@mkdir -p $@

.PHONY: common-format
common-format:
common-format: $(GOLANGCI_LINT)
@echo ">> formatting code"
$(GO) fmt $(pkgs)
ifdef GOLANGCI_LINT
@echo ">> formatting code with golangci-lint"
$(GOLANGCI_LINT) fmt $(GOLANGCI_FMT_OPTS)
endif

.PHONY: common-vet
common-vet:
Expand Down Expand Up @@ -248,8 +253,8 @@ $(PROMU):
cp $(PROMU_TMP)/promu-$(PROMU_VERSION).$(GO_BUILD_PLATFORM)/promu $(FIRST_GOPATH)/bin/promu
rm -r $(PROMU_TMP)

.PHONY: proto
proto:
.PHONY: common-proto
common-proto:
@echo ">> generating code from proto files"
@./scripts/genproto.sh

Expand Down
11 changes: 10 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@

Prometheus exporter for PostgreSQL server metrics.

CI Tested PostgreSQL versions: `11`, `12`, `13`, `14`, `15`, `16`
CI Tested PostgreSQL versions: `11`, `12`, `13`, `14`, `15`, `16`, `17`.

## Quick Start
This package is available for Docker:
Expand Down Expand Up @@ -144,9 +144,18 @@ This will build the docker image as `prometheuscommunity/postgres_exporter:${bra
* `[no-]collector.stat_database`
Enable the `stat_database` collector (default: enabled).

* `[no-]collector.stat_progress_vacuum`
Enable the `stat_progress_vacuum` collector (default: enabled).

* `[no-]collector.stat_statements`
Enable the `stat_statements` collector (default: disabled).

* `[no-]collector.stat_statements.include_query`
Enable selecting statement query together with queryId. (default: disabled)

* `--collector.stat_statements.query_length`
Maximum length of the statement text. Default is 120.

* `[no-]collector.stat_user_tables`
Enable the `stat_user_tables` collector (default: enabled).

Expand Down
2 changes: 1 addition & 1 deletion VERSION
Original file line number Diff line number Diff line change
@@ -1 +1 @@
0.16.0
0.17.1
4 changes: 2 additions & 2 deletions cmd/postgres_exporter/pg_setting.go
Original file line number Diff line number Diff line change
Expand Up @@ -67,7 +67,7 @@ type pgSetting struct {
func (s *pgSetting) metric(labels prometheus.Labels) prometheus.Metric {
var (
err error
name = strings.Replace(s.name, ".", "_", -1)
name = strings.ReplaceAll(strings.ReplaceAll(s.name, ".", "_"), "-", "_")
unit = s.unit // nolint: ineffassign
shortDesc = fmt.Sprintf("Server Parameter: %s", s.name)
subsystem = "settings"
Expand Down Expand Up @@ -131,7 +131,7 @@ func (s *pgSetting) normaliseUnit() (val float64, unit string, err error) {
case "B", "kB", "MB", "GB", "TB", "1kB", "2kB", "4kB", "8kB", "16kB", "32kB", "64kB", "16MB", "32MB", "64MB":
unit = "bytes"
default:
err = fmt.Errorf("Unknown unit for runtime variable: %q", s.unit)
err = fmt.Errorf("unknown unit for runtime variable: %q", s.unit)
return
}

Expand Down
4 changes: 2 additions & 2 deletions cmd/postgres_exporter/pg_setting_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -214,7 +214,7 @@ var fixtures = []fixture{
n: normalised{
val: 10,
unit: "",
err: `Unknown unit for runtime variable: "nonexistent"`,
err: `unknown unit for runtime variable: "nonexistent"`,
},
},
}
Expand All @@ -240,7 +240,7 @@ func (s *PgSettingSuite) TestNormaliseUnit(c *C) {
func (s *PgSettingSuite) TestMetric(c *C) {
defer func() {
if r := recover(); r != nil {
if r.(error).Error() != `Unknown unit for runtime variable: "nonexistent"` {
if r.(error).Error() != `unknown unit for runtime variable: "nonexistent"` {
panic(r)
}
}
Expand Down
5 changes: 4 additions & 1 deletion cmd/postgres_exporter/postgres_exporter.go
Original file line number Diff line number Diff line change
Expand Up @@ -251,6 +251,9 @@ var builtinMetricMaps = map[string]intermediateMetricMap{
"state": {LABEL, "connection state", nil, semver.MustParseRange(">=9.2.0")},
"usename": {LABEL, "connection usename", nil, nil},
"application_name": {LABEL, "connection application_name", nil, nil},
"backend_type": {LABEL, "connection backend_type", nil, nil},
"wait_event_type": {LABEL, "connection wait_event_type", nil, nil},
"wait_event": {LABEL, "connection wait_event", nil, nil},
"count": {GAUGE, "number of connections in this state", nil, nil},
"max_tx_duration": {GAUGE, "max duration in seconds any active transaction has been running", nil, nil},
},
Expand Down Expand Up @@ -678,7 +681,7 @@ func (e *Exporter) scrape(ch chan<- prometheus.Metric) {
if err := e.scrapeDSN(ch, dsn); err != nil {
errorsCount++

logger.Error("error scraping dsn", "err", err, "dsn", dsn)
logger.Error("error scraping dsn", "err", err, "dsn", loggableDSN(dsn))

if _, ok := err.(*ErrorConnectToServer); ok {
connectionErrorsCount++
Expand Down
9 changes: 8 additions & 1 deletion cmd/postgres_exporter/queries.go
Original file line number Diff line number Diff line change
Expand Up @@ -115,6 +115,9 @@ var queryOverrides = map[string][]OverrideQuery{
tmp.state,
tmp2.usename,
tmp2.application_name,
tmp2.backend_type,
tmp2.wait_event_type,
tmp2.wait_event,
COALESCE(count,0) as count,
COALESCE(max_tx_duration,0) as max_tx_duration
FROM
Expand All @@ -133,9 +136,13 @@ var queryOverrides = map[string][]OverrideQuery{
state,
usename,
application_name,
backend_type,
wait_event_type,
wait_event,
count(*) AS count,
MAX(EXTRACT(EPOCH FROM now() - xact_start))::float AS max_tx_duration
FROM pg_stat_activity GROUP BY datname,state,usename,application_name) AS tmp2
FROM pg_stat_activity
GROUP BY datname,state,usename,application_name,backend_type,wait_event_type,wait_event) AS tmp2
ON tmp.state = tmp2.state AND pg_database.datname = tmp2.datname
`,
},
Expand Down
9 changes: 7 additions & 2 deletions cmd/postgres_exporter/server.go
Original file line number Diff line number Diff line change
Expand Up @@ -119,12 +119,17 @@ func (s *Server) Scrape(ch chan<- prometheus.Metric, disableSettingsMetrics bool
if !disableSettingsMetrics && s.master {
if err = querySettings(ch, s); err != nil {
err = fmt.Errorf("error retrieving settings: %s", err)
return err
}
}

errMap := queryNamespaceMappings(ch, s)
if len(errMap) > 0 {
err = fmt.Errorf("queryNamespaceMappings returned %d errors", len(errMap))
if len(errMap) == 0 {
return nil
}
err = fmt.Errorf("queryNamespaceMappings errors encountered")
for namespace, errStr := range errMap {
err = fmt.Errorf("%s, namespace: %s error: %s", err, namespace, errStr)
}

return err
Expand Down
2 changes: 1 addition & 1 deletion cmd/postgres_exporter/util.go
Original file line number Diff line number Diff line change
Expand Up @@ -159,7 +159,7 @@ func dbToString(t interface{}) (string, bool) {
// Try and convert to string
return string(v), true
case string:
return v, true
return strings.ToValidUTF8(v, "�"), true
case bool:
if v {
return "true", true
Expand Down
7 changes: 4 additions & 3 deletions collector/collector.go
Original file line number Diff line number Diff line change
Expand Up @@ -37,8 +37,9 @@ const (
// Namespace for all metrics.
namespace = "pg"

defaultEnabled = true
defaultDisabled = false
collectorFlagPrefix = "collector."
defaultEnabled = true
defaultDisabled = false
)

var (
Expand Down Expand Up @@ -74,7 +75,7 @@ func registerCollector(name string, isDefaultEnabled bool, createFunc func(colle
}

// Create flag for this collector
flagName := fmt.Sprintf("collector.%s", name)
flagName := collectorFlagPrefix + name
flagHelp := fmt.Sprintf("Enable the %s collector (default: %s).", name, helpDefaultState)
defaultValue := fmt.Sprintf("%v", isDefaultEnabled)

Expand Down
20 changes: 10 additions & 10 deletions collector/collector_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -48,15 +48,15 @@ func readMetric(m prometheus.Metric) MetricResult {

func sanitizeQuery(q string) string {
q = strings.Join(strings.Fields(q), " ")
q = strings.Replace(q, "(", "\\(", -1)
q = strings.Replace(q, "?", "\\?", -1)
q = strings.Replace(q, ")", "\\)", -1)
q = strings.Replace(q, "[", "\\[", -1)
q = strings.Replace(q, "]", "\\]", -1)
q = strings.Replace(q, "{", "\\{", -1)
q = strings.Replace(q, "}", "\\}", -1)
q = strings.Replace(q, "*", "\\*", -1)
q = strings.Replace(q, "^", "\\^", -1)
q = strings.Replace(q, "$", "\\$", -1)
q = strings.ReplaceAll(q, "(", "\\(")
q = strings.ReplaceAll(q, "?", "\\?")
q = strings.ReplaceAll(q, ")", "\\)")
q = strings.ReplaceAll(q, "[", "\\[")
q = strings.ReplaceAll(q, "]", "\\]")
q = strings.ReplaceAll(q, "{", "\\{")
q = strings.ReplaceAll(q, "}", "\\}")
q = strings.ReplaceAll(q, "*", "\\*")
q = strings.ReplaceAll(q, "^", "\\^")
q = strings.ReplaceAll(q, "$", "\\$")
return q
}
12 changes: 7 additions & 5 deletions collector/pg_long_running_transactions.go
Original file line number Diff line number Diff line change
Expand Up @@ -50,11 +50,13 @@ var (
)

longRunningTransactionsQuery = `
SELECT
COUNT(*) as transactions,
MAX(EXTRACT(EPOCH FROM clock_timestamp())) AS oldest_timestamp_seconds
FROM pg_catalog.pg_stat_activity
WHERE state is distinct from 'idle' AND query not like 'autovacuum:%'
SELECT
COUNT(*) as transactions,
MAX(EXTRACT(EPOCH FROM clock_timestamp() - pg_stat_activity.xact_start)) AS oldest_timestamp_seconds
FROM pg_catalog.pg_stat_activity
WHERE state IS DISTINCT FROM 'idle'
AND query NOT LIKE 'autovacuum:%'
AND pg_stat_activity.xact_start IS NOT NULL;
`
)

Expand Down
Loading