feat(taps): Allow SQL stream implementations to override the SELECT statement generation without modifying extraction logic with SQLStream.build_query(...)

[edgarrmondragon]

Summary by Sourcery

Provide extension points for customizing SQL query generation by extracting replication filtering and limit logic into separate hook methods (build_query, apply_replication_filter, and apply_abort_query_limit) and refactoring get_records to use them.

New Features:

• Add build_query method to allow streams to override base SELECT query
• Introduce apply_replication_filter hook to encapsulate replication key ordering and filtering
• Introduce apply_abort_query_limit hook to encapsulate record limit enforcement

Enhancements:

• Refactor get_records to delegate query construction to new hook methods
• Mark unsupported partitioning branch with # pragma: no cover

[sourcery-ai]

Reviewer's Guide

Introduce a new, overridable query‐building API—extracting SELECT, replication filter, and limit logic into build_query, apply_replication_filter, and apply_abort_query_limit—and refactor get_records to use these methods, simplifying replication key start‐value handling.

Sequence Diagram: Refactored get_records method in SQLStream

sequenceDiagram
    participant C as Client Code
    participant S as SQLStream

    C->>S: get_records(context)
    activate S
    S->>S: build_query(table)
    S-->>S: base_query
    S->>S: apply_replication_filter(base_query, table, context)
    S-->>S: filtered_query
    S->>S: apply_abort_query_limit(filtered_query)
    S-->>S: final_query
    deactivate S
    S-->>C: Iterable[dict]

Loading

Class Diagram: Updates to SQLStream for Customizable Query Building

classDiagram
    class SQLStream {
        +build_query(table: sa.Table) : sa.Select
        +apply_replication_filter(query: sa.Select, table: sa.Table, context: Context | None) : sa.Select
        +apply_abort_query_limit(query: sa.Select) : sa.Select
        +get_records(context: Context | None) : Iterable[dict[str, Any]]
    }

Loading

File-Level Changes

Change	Details	Files
Modularize query assembly into overridable helper methods	Extract initial SELECT logic into a new build_query stub Isolate replication filter logic in apply_replication_filter (using a walrus operator for start value) Isolate record‐limit logic in apply_abort_query_limit	singer_sdk/streams/sql.py
Refactor get_records to delegate to the new methods	Replace inline SELECT, ordering, and limit code with calls to build_query, apply_replication_filter, and apply_abort_query_limit Remove redundant replication key and ABORT_AT_RECORD_COUNT logic Add a pragma no cover on the unsupported partitioning context check	singer_sdk/streams/sql.py

---

Tips and commands

Interacting with Sourcery

• Trigger a new review: Comment @sourcery-ai review on the pull request.
• Continue discussions: Reply directly to Sourcery's review comments.
• Generate a GitHub issue from a review comment: Ask Sourcery to create an
  issue from a review comment by replying to it. You can also reply to a
  review comment with @sourcery-ai issue to create an issue from it.
• Generate a pull request title: Write @sourcery-ai anywhere in the pull
  request title to generate a title at any time. You can also comment
  @sourcery-ai title on the pull request to (re-)generate the title at any time.
• Generate a pull request summary: Write @sourcery-ai summary anywhere in
  the pull request body to generate a PR summary at any time exactly where you
  want it. You can also comment @sourcery-ai summary on the pull request to
  (re-)generate the summary at any time.
• Generate reviewer's guide: Comment @sourcery-ai guide on the pull
  request to (re-)generate the reviewer's guide at any time.
• Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
  pull request to resolve all Sourcery comments. Useful if you've already
  addressed all the comments and don't want to see them anymore.
• Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
  request to dismiss all existing Sourcery reviews. Especially useful if you
  want to start fresh with a new review - don't forget to comment
  @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

• Enable or disable review features such as the Sourcery-generated pull request
  summary, the reviewer's guide, and others.
• Change the review language.
• Add, remove or edit custom review instructions.
• Adjust other review settings.

Getting Help

• Contact our support team for questions or feedback.
• Visit our documentation for detailed guides and information.
• Keep in touch with the Sourcery team by following us on X/Twitter, LinkedIn or GitHub.

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 91.69%. Comparing base (491e062) to head (07874d0).

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #3050      +/-   ##
==========================================
+ Coverage   91.64%   91.69%   +0.04%     
==========================================
  Files          62       62              
  Lines        5315     5320       +5     
  Branches      686      685       -1     
==========================================
+ Hits         4871     4878       +7     
+ Misses        312      311       -1     
+ Partials      132      131       -1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[codspeed-hq]

CodSpeed Performance Report

Merging #3050 will not alter performance

_{Comparing refactor/sql-stream-build-query (07874d0) with main (491e062)}

Summary

✅ 8 untouched benchmarks

[edgarrmondragon]

@sourcery-ai review

[sourcery-ai]

Hey @edgarrmondragon - I've reviewed your changes - here's some feedback:

• The context parameter in build_query is currently unused—either implement context‐based partition filtering there or remove the parameter to avoid confusing the API.
• Consider extracting the replication key filter and abort‐limit logic into helper methods so custom build_query overrides can reuse those behaviors without duplicating code.

Here's what I looked at during the review

• 🟡 General issues: 2 issues found
• 🟢 Security: all looks good
• 🟢 Testing: all looks good
• 🟢 Documentation: all looks good

---

Sourcery is free for open source - if you like our reviews please consider sharing them ✨

• X
• Mastodon
• LinkedIn
• Facebook

_{Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.}

[edgarrmondragon]

@sourcery-ai review

[sourcery-ai]

Hey @edgarrmondragon - I've reviewed your changes and they look great!

Here's what I looked at during the review

• 🟡 General issues: 1 issue found
• 🟢 Security: all looks good
• 🟢 Testing: all looks good
• 🟢 Documentation: all looks good

---

Sourcery is free for open source - if you like our reviews please consider sharing them ✨

• X
• Mastodon
• LinkedIn
• Facebook

_{Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.}