Skip to content

Stabilising changes for version 3.0.0 #108

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 4 commits into
base: main
Choose a base branch
from
Draft

Stabilising changes for version 3.0.0 #108

wants to merge 4 commits into from

Conversation

amunra
Copy link
Collaborator

@amunra amunra commented Jun 3, 2025

Overview

This PR is about tying up loose ends for a stable v. 3.0.0 of the client.
If we get feedback, we should add any extra fixup to the list below.

Task list

  • @javier: Updated front page doc and examples that are consistent across all clients.
  • JSON-driven array tests
  • When writing an array, check if the data buffer can be written as-is.
    • If PyArray_ISCARRAY_RO returns true, it should call just call
      line_sender_buffer_column_f64_arr_row_major.
    • This should be a simpler and faster path that determining if the strides
      are default strides or not.
  • Investigate what is earliest version of NumPy that our code supports.
    I've tested back to Numpy 1.26, but older versions may work as well.
    Once found, update pyproject.toml, then:
  • Add a CI test run that just the Buffer Python tests, across different
    versions of NumPy.
    • To run the specific tests, run:
      export TEST_QUESTDB_PATCH_PATH=1
      python3 test/test.py -v TestBufferProtocolVersionV2
    • To quickly run Python with specific dependencies in a virtual environment,
      use uv: https://docs.astral.sh/uv/getting-started/installation/
      Then run uv run --with 'numpy==1.26.0' test/test.py -v TestBufferProtocolVersionV2
      See: https://docs.astral.sh/uv/guides/scripts/#running-a-script-with-dependencies
    • Context: The uv tool installs Python packages much faster than Python's
      default pip tool. It's a really cool tool, worth learning about.
    • I would suggest a completely separate CI step that only executes on Linux.
      There's no point doing this cross-platforms.
    • Ideally without a for loop, instead just flat steps in CI, one per numpy
      version.
    • We obviously don't need to test every single version. Take 5 or so "sample"
      versions. This is just to ensure that we don't regress the code in the future.
  • Fix Sphinx API docs
    • Fix datetime.datetime in Python Sphinx API docs
    • Run ./proj clean && ./proj build && ./proj docs to regenerate the docs
    • Clean-building the docs will issue more warnings, these need to be fixed.
    • The issue is that the docs reference datetime which is only imported in
      Cython and not in Python, so the type can't be found.
    • Same for cross-linking to numpy.ndarray in the docs.
    • Tip: Run ./proj docs 1 to launch docs in the web browser.
  • Fix a SyntaxWarning in the tests:
    test_dataframe.py:1572: SyntaxWarning: invalid escape sequence '\['
    "Unsupported dtype int16\[pyarrow\] for column 'a'.*github"):
    
  • Update and document (front page + examples + api docs) which is the first
    version of QuestDB supports arrays.
    My guess is that it is going to be 8.4.0, but we need to confirm this.
    (I've also added this todo to the Rust/C/C++ client TODO list.)
  • Once the next QuestDB release is cut, we should ensure that the CI tests
    pick up the latest version of QuestDB and run protocol version 2 tests against
    the server. We should also retain a CI test run that runs
    against an older version of QuestDB (e.g. 8.3.3) to ensure that we don't
    introduce regressions against protocol version 1.

Tandem PR

Related c-questdb-client PR: questdb/c-questdb-client#105

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants