Skip to content

Refactor PostgreSQL configuration and remove deprecated database setup #215

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 6 commits into from
Apr 29, 2025

Conversation

emmanuelmathot
Copy link
Contributor

Overview

This PR implements the unified PostgreSQL configuration as proposed in issue #209. It provides a cleaner, more standardized approach to managing PostgreSQL database access across different deployment scenarios.

Changes

  1. Added new postgresql configuration section in values.yaml with three supported types:

    • postgrescluster: Default using CrunchyData PGO
    • external-plaintext: For user-managed PostgreSQL with direct credentials
    • external-secret: For user-managed PostgreSQL with secret references
  2. Standardized environment variables to use official PostgreSQL variables:

    • PGHOST, PGPORT, PGUSER, PGPASSWORD, PGDATABASE
    • Legacy POSTGRES_* variables maintained for backward compatibility
  3. Added new helper functions in _helpers.tpl:

    • eoapi.postgresqlEnv: Main function for handling all PostgreSQL configurations
    • eoapi.validatePostgresql: Validation for PostgreSQL configuration
    • eoapi.mapLegacyPostgresql: Backward compatibility mapping
  4. Removed deprecated db section from values.yaml

  5. Updated service deployments and pgstacbootstrap jobs to use the new configuration

Benefits

  • Unified Configuration: A single, consistent way to configure PostgreSQL access
  • Standard Environment Variables: Using standard PostgreSQL environment variables improves compatibility
  • Flexible Deployment Options: Support for all three required deployment scenarios
  • Improved Maintainability: Cleaner, more organized code that follows Helm best practices
  • Better Validation: Comprehensive validation to prevent misconfiguration

Breaking Changes

  • Environment variables standardized to PG* format
  • Deprecated redundant variables (POSTGRES_HOST_READER, POSTGRES_HOST_WRITER, DATABASE_URL)
  • Removed db section from values.yaml

Migration

Users should update their values.yaml to use the new postgresql section instead of the deprecated db section:

# Using postgrescluster (default)
postgresql:
  type: "postgrescluster"

postgrescluster:
  enabled: true
  # ... other postgrescluster settings

- Introduced a unified PostgreSQL configuration structure in values.yaml, replacing the old db configuration.
- Added new helper functions for managing PostgreSQL environment variables and secrets based on the selected configuration type (postgrescluster, external-plaintext, external-secret).
- Removed old database-related templates (ConfigMap, Deployment, PVC, Secrets, Service) that are no longer needed.
- Updated the pgstacbootstrap job and configmap templates to align with the new PostgreSQL configuration.
- Implemented validation for PostgreSQL settings to ensure required fields are set based on the selected type.
@emmanuelmathot emmanuelmathot marked this pull request as draft April 17, 2025 12:07
@emmanuelmathot emmanuelmathot marked this pull request as ready for review April 17, 2025 12:27
Copy link

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR refactors the PostgreSQL configuration to provide a unified, standardized approach for managing database access while removing deprecated configuration sections. Key changes include the removal of the old db section in values.yaml, the addition of a new postgresql configuration section with multiple connection types, and updates to deployments and bootstrap jobs to use new helper functions and standardized environment variables.

Reviewed Changes

Copilot reviewed 4 out of 10 changed files in this pull request and generated no comments.

File Description
helm-chart/eoapi/values.yaml Removed deprecated db section and introduced unified PostgreSQL config
helm-chart/eoapi/templates/services/deployment.yaml Updated to use new validation and environment variable injection
helm-chart/eoapi/templates/pgstacbootstrap/job.yaml Refactored job configuration to align with new PostgreSQL environment
helm-chart/eoapi/templates/pgstacbootstrap/configmap.yaml Simplified conditional logic for fixture loading in the ConfigMap
Files not reviewed (6)
  • helm-chart/eoapi/templates/_helpers.tpl: Language not supported
  • helm-chart/eoapi/templates/db/configmap.yaml: Language not supported
  • helm-chart/eoapi/templates/db/deployment.yaml: Language not supported
  • helm-chart/eoapi/templates/db/pvc.yaml: Language not supported
  • helm-chart/eoapi/templates/db/secrets.yaml: Language not supported
  • helm-chart/eoapi/templates/db/service.yaml: Language not supported
Comments suppressed due to low confidence (2)

helm-chart/eoapi/templates/pgstacbootstrap/job.yaml:89

  • The removal of the condition checking 'eq .Values.pgstacBootstrap.settings.envVars.LOAD_FIXTURES "true"' may change the behavior for loading fixtures. Please verify that this change is intentional.
{{- if and .Values.pgstacBootstrap.enabled .Values.pgstacBootstrap.settings.loadSamples }}

helm-chart/eoapi/templates/pgstacbootstrap/configmap.yaml:18

  • The simplified conditional for fixture loading removes the check for 'LOAD_FIXTURES' from envVars. Confirm that this change aligns with the intended behavior.
{{- if .Values.pgstacBootstrap.settings.loadSamples }}

@emmanuelmathot emmanuelmathot linked an issue Apr 17, 2025 that may be closed by this pull request
@emmanuelmathot emmanuelmathot added the enhancement New feature or request label Apr 17, 2025
@emmanuelmathot emmanuelmathot linked an issue Apr 20, 2025 that may be closed by this pull request
2 tasks
…he external secret (host, port, database) will override the corresponding values defined in external.host, external.port, and external.database.

Confirmed that the conditional blocks in deployment.yaml were already consolidated to eliminate redundancy. The file was already using a single include statement for PostgreSQL environment variables:

env:
  {{- include "eoapi.postgresqlEnv" $ | nindent 12 }}
Removed the unused eoapi.mapLegacyPostgresql helper function from _helpers.tpl as it wasn't being referenced anywhere in the codebase.
@emmanuelmathot emmanuelmathot requested review from sunu and Copilot April 28, 2025 15:34
Copy link

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR refactors the PostgreSQL database configuration to a unified scheme and removes the deprecated db setup, simplifying and standardizing the Helm chart configuration for PostgreSQL.

  • Introduces a new postgresql section in values.yaml with three deployment types
  • Updates helper functions and environment variable usage to rely on PostgreSQL’s standard PG* variables
  • Removes obsolete db configuration and legacy conditions from deployment and job templates

Reviewed Changes

Copilot reviewed 4 out of 10 changed files in this pull request and generated no comments.

File Description
helm-chart/eoapi/values.yaml Adds unified PostgreSQL configuration and removes the deprecated db section
helm-chart/eoapi/templates/services/deployment.yaml Replaces legacy validation and secret inclusion with new helper functions for PostgreSQL configuration
helm-chart/eoapi/templates/pgstacbootstrap/job.yaml Updates job template environment configuration and release name interpolation, and simplifies sample loading condition
helm-chart/eoapi/templates/pgstacbootstrap/configmap.yaml Simplifies condition for loading sample data in the ConfigMap
Files not reviewed (6)
  • helm-chart/eoapi/templates/_helpers.tpl: Language not supported
  • helm-chart/eoapi/templates/db/configmap.yaml: Language not supported
  • helm-chart/eoapi/templates/db/deployment.yaml: Language not supported
  • helm-chart/eoapi/templates/db/pvc.yaml: Language not supported
  • helm-chart/eoapi/templates/db/secrets.yaml: Language not supported
  • helm-chart/eoapi/templates/db/service.yaml: Language not supported
Comments suppressed due to low confidence (3)

helm-chart/eoapi/templates/services/deployment.yaml:71

  • [nitpick] Ensure that the change from $.Release.Name to .Release.Name is intentional and consistent with the desired template context, so that release name resolution remains correct across all resources.
          failureThreshold: 60

helm-chart/eoapi/templates/pgstacbootstrap/job.yaml:89

  • Verify that the removal of the extra condition (checking envVars.LOAD_FIXTURES) does not unintentionally prevent sample data from loading in scenarios that relied on the legacy variable.
{{- if and .Values.pgstacBootstrap.enabled .Values.pgstacBootstrap.settings.loadSamples }}

helm-chart/eoapi/templates/pgstacbootstrap/configmap.yaml:18

  • Confirm that the simplified condition for loading sample configuration in this ConfigMap covers all required scenarios previously handled by the dual check (including envVars.LOAD_FIXTURES).
{{- if .Values.pgstacBootstrap.settings.loadSamples }}

@emmanuelmathot emmanuelmathot merged commit 9d7f3b7 into main Apr 29, 2025
2 checks passed
@emmanuelmathot emmanuelmathot deleted the unified_pg branch April 29, 2025 08:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Unified PostgreSQL Configuration Review PostgreSQL approach
2 participants