Skip to content

Ignite 18006 #2

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 62 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
62 commits
Select commit Hold shift + click to select a range
51e9c82
IGNITE-13903: Add tox, docker-compose and travis integration
ivandasch Dec 28, 2020
99ed735
IGNITE-11303: Partition Awareness for Python Thin
isapego Jan 25, 2021
644de99
IGNITE-14056: Fixed README and documentation
isapego Jan 26, 2021
4ca871d
IGNITE-14058: Bool arrays now return as bool array, not int array
isapego Jan 26, 2021
c746329
IGNITE-14072 Refactor, remove duplicates and optimize Response and SQ…
ivandasch Jan 27, 2021
18d32bb
IGNITE-14057 Support big-endian systems
ptupitsyn Jan 27, 2021
b13d43b
IGNITE-14075: Fix hash code calculation for composite keys
alamar Jan 27, 2021
e0c22ef
IGNITE-14059: Fix hashing of complex objects
isapego Feb 2, 2021
2c0ecb2
IGNITE-13863: Fix Null reading and writing
isapego Feb 2, 2021
d7d6d35
IGNITE-11528: Deprecate SqlQuery API
isapego Feb 4, 2021
83208c2
IGNITE-12975: SQL query do not create cache
isapego Feb 4, 2021
0f2828b
IGNITE-14127: Default sql page size from 1 => 1024
isapego Feb 4, 2021
2ead7b9
IGNITE-13967: Optimizations and refactoring of parsing
ivandasch Feb 8, 2021
ba268cc
IGNITE-14154 Remove unnecessary test, remove duplicates
ivandasch Feb 10, 2021
7743b23
IGNITE-14167 Simplify reconnecting, fix affinity topology change dete…
ivandasch Feb 15, 2021
e5ca3fc
IGNITE-14186 Implement C module to speedup hashcode
ivandasch Feb 18, 2021
672a767
IGNITE-14211 Remove existing cache requirement from SQL API
isapego Feb 19, 2021
b2030be
IGNITE-14240 Re-factor tests
ivandasch Feb 25, 2021
6f56a3b
IGNITE-14245 Fix infinite loop while trying to get affinity mapping o…
ivandasch Feb 26, 2021
cfc2fe4
IGNITE-13911 Asyncio version of client
ivandasch Mar 23, 2021
0bcd77f
IGNITE-13862 Add test case for put_all large amount of complex maps
ivandasch Mar 26, 2021
f00d70f
IGNITE-14432 Implement connection context managers for clients
ivandasch Mar 29, 2021
a7392fc
IGNITE-14429 Fix cache.get_size with non-default PeekModes
ivandasch Mar 29, 2021
2fd7fda
IGNITE-13405 Fix cache configuration serialization/deserialization
ivandasch Mar 30, 2021
7cbfe32
IGNITE-14444 Move affinity mapping storage and best node calculation …
ivandasch Mar 31, 2021
7c1d0cc
IGNITE-14465 Add the ability to set and get cluster state
isapego Apr 3, 2021
e48f4be
IGNITE-14472 Multiple performance improvements
ivandasch Apr 5, 2021
70bb1d9
IGNITE-14418 Add async client documentation, update examples
ivandasch Apr 8, 2021
3586db7
IGNITE-14511 Fix serialization of bytes, improve serialization-deseri…
ivandasch Apr 12, 2021
fb400f1
IGNITE-14518 Add proper license and long description to package info …
ivandasch Apr 12, 2021
453ea5d
IGNITE-14534 Add script to build wheels on Windows
isapego Apr 14, 2021
572c8b9
Update version for next release
ivandasch Apr 14, 2021
4b39956
Add release notes for 0.4.0
ivandasch Apr 14, 2021
8b1e61c
IGNITE-14564 Add LICENSE and NOTICE files to all artifacts - Fixes #34.
ivandasch Apr 16, 2021
9f72781
IGNITE-14595 Implement ExpiryPolicy support - Fixes #35.
ivandasch Apr 23, 2021
746dd13
IGNITE-14705 Fix handling collections with binary objects - Fixes #37.
ivandasch May 13, 2021
8c8a006
IGNITE-14738 Remove obsolete setup dependency to pytest-runner - Fixe…
ivandasch May 19, 2021
fa364df
IGNITE-14686 Fix incorrect type hint for cache's get_all - Fixes #36.
Jun 8, 2021
9945ecb
IGNITE-12467 Implement transactions, rewrite async connections using …
ivandasch Jun 9, 2021
a53afcc
IGNITE-14850 Add documentation to transaction's api - Fixes #41.
ivandasch Jun 10, 2021
0afab5b
Fix transactions code example in docs
ivandasch Jun 10, 2021
c64fb10
Fix async transactions code example in docs
ivandasch Jun 10, 2021
fbe61f3
IGNITE-14851 Enable partition awareness by default, fix unnecessary d…
ivandasch Jun 11, 2021
365b16e
Update version for the next release
ivandasch Jun 15, 2021
05413e7
Add release notes for 0.5.0 - Fixes #43.
ivandasch Jun 15, 2021
92a115c
IGNITE-14911 Unify timeouts, add support for datetime.timedelta for e…
ivandasch Jun 16, 2021
dd3b280
Update to latest version of released pyignite in README.md
ivandasch Jun 21, 2021
8fc14f8
IGNITE-15103 Implement debug and error logging of connections and que…
ivandasch Jul 14, 2021
82f29e2
IGNITE-15102 Implement event handling and monitoring for python thin …
ivandasch Jul 20, 2021
de07126
IGNITE-15118 Implement handshake timeout - Fixes #47.
ivandasch Jul 22, 2021
05dc38e
Add release notes for 0.5.1
ivandasch Jul 22, 2021
be18440
IGNITE-15266 Fix nested object arrays deserialization - Fixes #48.
bojidar-bg Aug 6, 2021
ef8687e
Fix tests local run with tests/__init__.py added (#49)
nizhikov Sep 6, 2021
3bf1cc1
IGNITE-15479 Fix incorrect partial read from socket in sync client - …
ivandasch Sep 9, 2021
abbff02
IGNITE-16599 Fix tests by setting strict version of MarkupSafe (#51)
ivandasch Feb 21, 2022
a1a920e
Add release notes for 0.5.2
ivandasch Sep 9, 2021
ac12197
IGNITE-17023 Remove Python 3.6, add 3.10.
isapego May 26, 2022
bf5ac4f
IGNITE-17054 Fix examples and documentation (#53)
isapego Jun 6, 2022
bc5a52c
IGNITE-17494 use_ssl is not set when auth used (#55)
isapego Aug 9, 2022
a0309cd
IGNITE-18006 Fix timeouts.
ivandasch Oct 31, 2022
ef1d248
IGNITE-18006 Add reproducers
ivandasch Nov 2, 2022
0dd471b
IGNITE-18006 Remove travis, add github actions
ivandasch Nov 2, 2022
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
34 changes: 34 additions & 0 deletions .asf.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

github:
description: "Apache Ignite Python Thin Client"
homepage: https://ignite.apache.org/
labels:
- ignite
- python
features:
wiki: false
issues: false
projects: false
enabled_merge_buttons:
squash: true
merge: false
rebase: false
notifications:
commits: [email protected]
issues: [email protected]
pullrequests: [email protected]
jira_options: link label worklog
58 changes: 58 additions & 0 deletions .github/workflows/pr_check.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

name: Check code style and run tests
on: [ push, pull_request ]

env:
IGNITE_VERSION: 2.14.0
IGNITE_HOME: /opt/ignite

jobs:
build:
runs-on: ubuntu-latest
continue-on-error: true
strategy:
fail-fast: false
matrix:
cfg:
- { python: "3.7", toxenv: "py37" }
- { python: "3.8", toxenv: "py38" }
- { python: "3.9", toxenv: "py39" }
- { python: "3.10", toxenv: "py310" }
- { python: "3.11", toxenv: "py311" }
- { python: "3.11", toxenv: "codestyle" }

steps:
- uses: actions/checkout@v3
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v4
with:
python-version: ${{ matrix.cfg.python}}
- name: Install Apache Ignite
run: |
curl -L https://apache-mirror.rbc.ru/pub/apache/ignite/${IGNITE_VERSION}/apache-ignite-slim-${IGNITE_VERSION}-bin.zip > ignite.zip
unzip ignite.zip -d /opt
mv /opt/apache-ignite-slim-${IGNITE_VERSION}-bin /opt/ignite
mv /opt/ignite/libs/optional/ignite-log4j2 /opt/ignite/libs/

- name: Install tox
run: |
pip install tox

- name: Run tests
run: |
pip install tox
tox -e ${{ matrix.cfg.toxenv }}
16 changes: 16 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
.idea
.benchmarks
.vscode
.eggs
.pytest_cache
.tox
*.so
build
distr
docs/generated
tests/config/*.xml
junit*.xml
pyignite.egg-info
ignite-log-*
__pycache__
venv
13 changes: 13 additions & 0 deletions MANIFEST.in
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
graft requirements
graft examples
graft docs
recursive-exclude docs/generated *
graft tests
recursive-exclude tests/config *.xml
recursive-exclude tests/logs *
global-exclude *.py[cod]
global-exclude *__pycache__*
include tox.ini
include README.md
include LICENSE
include NOTICE
5 changes: 5 additions & 0 deletions NOTICE
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
Apache Ignite binary client Python API
Copyright 2021 The Apache Software Foundation

This product includes software developed at
The Apache Software Foundation (http://www.apache.org/).
90 changes: 69 additions & 21 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,24 +3,23 @@ Apache Ignite thin (binary protocol) client, written in Python 3.

## Prerequisites

- Python 3.4 or above (3.6 is tested),
- Python 3.7 or above (3.7, 3.8, 3.9 and 3.10 are tested),
- Access to Apache Ignite node, local or remote. The current thin client
version was tested on Apache Ignite 2.7.0 (binary client protocol 1.2.0).
version was tested on Apache Ignite 2.10 (binary client protocol 1.7.0).

## Installation

#### *for end user*
### *for end user*
If you only want to use the `pyignite` module in your project, do:
```
```bash
$ pip install pyignite
```

#### *for developer*
### *for developer*
If you want to run tests, examples or build documentation, clone
the whole repository:
```
$ git clone [email protected]:apache/ignite.git
$ cd ignite/modules/platforms/python
```bash
$ git clone [email protected]:apache/ignite-python-thin-client.git
$ pip install -e .
```

Expand All @@ -31,45 +30,94 @@ in the `pip` manual.

Then run through the contents of `requirements` folder to install
the additional requirements into your working Python environment using
```
```bash
$ pip install -r requirements/<your task>.txt
```

You may also want to consult the `setuptools` manual about using `setup.py`.

### *optional C extension*
There is an optional C extension to speedup some computational intensive tasks. If it's compilation fails
(missing compiler or CPython headers), `pyignite` will be installed without this module.

- On Linux or MacOS X only C compiler is required (`gcc` or `clang`). It compiles during standard setup process.
- For building universal `wheels` (binary packages) for Linux, just invoke script `./scripts/create_distr.sh`.

***NB!* Docker is required.**

- On Windows MSVC 14.x required, and it should be in path, also python versions 3.7, 3.8, 3.9 and 3.10 both for x86 and
x86-64 should be installed. You can disable some of these versions but you'd need to edit script for that.
- For building `wheels` for Windows, invoke script `.\scripts\BuildWheels.ps1` using PowerShell. Just make sure that
your execution policy allows execution of scripts in your environment.

Ready wheels for `x86` and `x86-64` for different python versions (3.7, 3.8, 3.9 and 3.10) will be
located in `distr` directory.

### Updating from older version

To upgrade an existing package, use the following command:
```bash
pip install --upgrade pyignite
```

To install the latest version of a package:
```bash
pip install pyignite
```

To install a specific version:
```bash
pip install pyignite==0.5.1
```

## Documentation
[The package documentation](https://apache-ignite-binary-protocol-client.readthedocs.io)
is available at *RTD* for your convenience.

If you want to build the documentation from source, do the developer
installation as described above, then run the following commands:
```
$ cd ignite/modules/platforms/python
installation as described above, then run the following commands from the
client's root directory:
```bash
$ pip install -r requirements/docs.txt
$ cd docs
$ make html
```

Then open `ignite/modules/platforms/python/docs/generated/html/index.html`
in your browser.
Then open `docs/generated/html/index.html` in your browser.

## Examples
Some examples of using pyignite are provided in
`ignite/modules/platforms/python/examples` folder. They are extensively
commented in the
Some examples of using pyignite are provided in `examples` folder. They are
extensively commented in the
“[Examples of usage](https://apache-ignite-binary-protocol-client.readthedocs.io/en/latest/examples.html)”
section of the documentation.

This code implies that it is run in the environment with `pyignite` package
installed, and Apache Ignite node is running on localhost:10800.

## Testing
Run
*NB!* It is recommended installing `pyignite` in development mode.
Refer to [this section](#for-developer) for instructions.

Do not forget to install test requirements:
```bash
$ pip install -r requirements/install.txt -r requirements/tests.txt
```

Also, you'll need to have a binary release of Ignite with `log4j2` enabled and to set
`IGNITE_HOME` environment variable:
```bash
$ cd <ignite_binary_release>
$ export IGNITE_HOME=$(pwd)
$ cp -r $IGNITE_HOME/libs/optional/ignite-log4j2 $IGNITE_HOME/libs/
```
### Run basic tests
```bash
$ pytest
```
$ cd ignite/modules/platforms/python
$ python setup.py pytest
### Run with examples
```bash
$ pytest --examples
```

*NB!* All tests require Apache Ignite node running on localhost:10800.
If you need to change the connection parameters, see the documentation on
[testing](https://apache-ignite-binary-protocol-client.readthedocs.io/en/latest/readme.html#testing).
48 changes: 48 additions & 0 deletions RELEASE_NOTES.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
Apache Ignite python thin client
================================

0.5.2
--------------------------------
* Fixed incorrect partial read from socket in sync client
* Fixed nested object arrays deserialization

0.5.1
--------------------------------
* Added logging of connection and queries events
* Added event listeners to connection events and query events
* Added client's side handshake timeout
* Fixed excessive deprecation warnings on python 3.7
* Fixed request to failed node when querying replicated cache
* Fixed excessive partition mapping requests

0.5.0
--------------------------------
* Added transaction API support (sync and async versions, async version supports only python 3.7+)
* Added ExpiryPolicy (TTL) support
* Improved performance of asyncio version by reimplementing network code using asyncio transports and protocols
* Enabled partition awareness by default
* Fixed handling collections of binary objects

0.4.0
--------------------------------
* Added partition awareness support
* Added asyncio support
* Added C module to speedup hashcode calculation
* Implement context management for connection method
* Implement cursors and context management for ScanQuery, SqlQuery and SqlFieldsQuery
* Add the ability to activate/deactivate the cluster
* Implement support for big-endianness
* Implement support of password for certificates
* Fix performance issues while working with big bytearrays and binary objects
* Fix serialization/deserialization of cache configuration
* Fix handling of null fields
* Fix SQL API
* Fix UUID serialization/deserialization
* Fix nested complex objects
* Fix incorrect hash code calculation for classes as composite keys
* Fix hashing of complex object
* Fix insert and select VARBINARY data type through SQL
* Fix wrong order of the SQL query result
* Fix handling of bytes and bytearrays
* Fix bool arrays handling
* Fix cache.get_size with non-default PeekModes
Loading