Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update to BigQuery read API v1 interface #5431

Open
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

kmjung
Copy link

@kmjung kmjung commented Jul 19, 2024

This change updates Scio's usage of the BigQuery storage read API to use the v1 interface rather than v1beta1, which will eventually be deprecated. The two interfaces are very similar, to the extent that this change is mostly string substitution.

I haven't yet been able to run end-to-end tests with this change. If there is no automated test coverage as part of the PR approval process, then I will figure out how to do so.

Copy link

codecov bot commented Jul 23, 2024

Codecov Report

Attention: Patch coverage is 0% with 4 lines in your changes missing coverage. Please review.

Project coverage is 61.13%. Comparing base (b52036a) to head (31fcc0d).
Report is 6 commits behind head on main.

Files Patch % Lines
...la/com/spotify/scio/bigquery/client/TableOps.scala 0.00% 4 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #5431      +/-   ##
==========================================
+ Coverage   61.09%   61.13%   +0.03%     
==========================================
  Files         310      311       +1     
  Lines       11060    11073      +13     
  Branches      769      771       +2     
==========================================
+ Hits         6757     6769      +12     
- Misses       4303     4304       +1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@RustedBones
Copy link
Contributor

Thanks for the heads up! We completely missed this upgrade.
CI fails with some expected binary compatibility issue. We'll plan this upgrade in next 0.15 release

@RustedBones RustedBones added this to the 0.15.0 milestone Jul 24, 2024
@kmjung
Copy link
Author

kmjung commented Jul 24, 2024

Thanks for the heads up! We completely missed this upgrade.

To be clear, you didn't miss anything per se -- the official deprecation notice hasn't been sent out yet (although this is coming). However, Scio is one of the last read API use cases that I'm aware of which is in wide use and has yet to migrate to v1. If we can start the adoption process for the newer API surface now, the deprecation process will hopefully be less impactful for users.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants