Update data_interfaces lib to include opensearch provider #30

WRFitch · 2023-02-08T20:45:28Z

Proposal

Updates data_interfaces lib to include objects and events to manage the opensearch provider lib, as well as TLS.

Context

While the changes in the data_interfaces library aren't unique to this repo, I've chosen to add them here first so we can ensure they're fit for purpose, and then we can add the finished library to the data-integrator repo.
The data-integrator changes are duplicated across the opensearch and integration test application charm lib/ directories. This should be an exact duplication.
rollingops lib has been updated to update locks on leader_elected. If the leader is replaced during a rolling op, the new leader never receives the process_locks event, so it never runs the op. A PR has been opened for this here, but again, I'm testing my changes here before merging.
This PR is definitely getting a merge commit into main so I don't nuke our commit history

Changelog

Updated data_interfaces lib to include OpenSearchProvider and OpenSearchRequirer objects, and events associated with opensearch.
data_interfaces lib has been updated to the correct ops version
Added TLS to opensearch provider relation
Added data_interfaces OpenSearchProvider to provider relation
Updated endpoint update method to guarantee we have the correct number of endpoints in the databag when scaling
added data_interfaces OpenSearchRequirer to requirer application test charm
Deferred starting opensearch if the init script fails to connect
Possible fix for a bug where update-status fails if we remove the tls-operator
Fixed some typos and formatting
Updated rollingops lib to update locks on leader_elected
Updated integration tests

Testing

Added unit tests
Added more extensive integration tests, which test the following:
- opensearch API access while scaling
- Checks that index data persists across relations

Adding fmt to default tox runs means we don't have to run it after finding lint doesn't work moving lib tests to a dedicated folder means they're easier to find/differentiate from charm-specific tests

…earch-operator into dev/add-relation

# PR link: #26 Some QoL changes from pgbouncer that may be useful, copyright updates, and a library import to declutter PR #25 ## Commits * updated gitignore and copyright string * imported lib and updated CI * imported data_interfaces lib * update libjuju for testing * Update ci.yaml

…l/opensearch-operator into dev/add-relation

… charm and relation files

welpaolo

Looks good to me, please add the function requested due to the change of the charm-relation-interfaces.

welpaolo · 2023-03-07T07:44:19Z

lib/charms/data_platform_libs/v0/data_interfaces.py

+            relation_id: the identifier for a particular relation.
+            version: database version.
+        """
+        self._update_relation_data(relation_id, {"version": version})


For the provider side, you will need to include the index parameter: canonical/charm-relation-interfaces#42.
In this case you will need to add a new function set_index in the provider.

thanks for letting me know, I've also opened a PR in that repo since the opensearch spec didn't get the update :(

this has been added in ce1ded6

deusebio

Looks all good to me!

lib/charms/opensearch/v0/opensearch_relation_provider.py

…ite right

lib/charms/opensearch/v0/opensearch_relation_provider.py

Handler is never called Handler was first added in #30 but a call to the handler was not added

WRFitch and others added 30 commits January 18, 2023 11:17

some quality of life suggestions

23ce1a1

Adding fmt to default tox runs means we don't have to run it after finding lint doesn't work moving lib tests to a dedicated folder means they're easier to find/differentiate from charm-specific tests

added basic skeleton for relation

bfcfd93

fix data_interfaces name

8b9fe45

updated copyright string, and added test shell

5758fed

format

f8c7d3f

added testing infra

bf2aaca

lint

1b3f8f3

update copyright notice

d623669

testing

f9d7842

lint

ba1f465

small fixes, setting up testing

1ffc8d6

learning about opensearch py lib

9ad5f4c

merge

061f9b8

Delete juju-crashdump-5a62e3a0-5d70-4cb7-8d6b-07b04b94f45d.tar.xz

ea140de

removed unnecessary updates to operator charm

bb91a53

Merge branch 'dev/add-relation' of https://github.com/canonical/opens…

cc788f2

…earch-operator into dev/add-relation

update to match the rest of the charm

4d016de

fix tests

ca90489

Merge branch 'feature/client-relation' of https://github.com/canonica…

74a48d7

…l/opensearch-operator into dev/add-relation

added some prelim relation data

c1e40ee

removed superfluous PR pipeline

fdcb7fd

made peer relation name a constant to prevent circular import between…

d7e2311

… charm and relation files

added user management

a52bbab

added relation teardown

3482c03

fixing up test charm

84cf77d

unit testing opensearch_users.py

ec1ad5f

finished unit tests

76f7e62

fixed users, updated tests

9ec6fa1

added small preliminary changes to clean up major relation PR

1734415

WRFitch and others added 12 commits March 3, 2023 17:44

fixed await

4e26c02

updating endpoints everywhere

7eceac9

lint

a3af2e9

removing endpoint updates

6af639e

Update test_opensearch_provider.py

bfd387a

lint

3e75fe6

fixed endpoint assignment, and triggered it during relation-departed

487b0fb

lib and unit test fixes

9b22c52

adding endpoint updates to status hook

a0b3347

small fixes

6d72368

committing again to verify tests aren't totally transient

9e8da66

get nodes using _get_nodes

333fc7c

welpaolo approved these changes Mar 7, 2023

View reviewed changes

WRFitch added 2 commits March 7, 2023 11:29

reverting rollingops changes to check if they're necessary

9edb2f4

added index functions to provider

ce1ded6

deusebio approved these changes Mar 7, 2023

View reviewed changes

lib/charms/opensearch/v0/opensearch_relation_provider.py Outdated Show resolved Hide resolved

WRFitch and others added 5 commits March 7, 2023 20:53

Merge branch 'main' into dev/add-lib

6d36e89

lint. I have to stop using the github merge tool, lint never works qu…

7795fa5

…ite right

reset rolling ops

2ffd925

Update test_opensearch_tls.py

91c1d7a

final cleanup for PR

6ec371c

delgod requested changes Mar 8, 2023

View reviewed changes

lib/charms/opensearch/v0/opensearch_relation_provider.py Outdated Show resolved Hide resolved

WRFitch and others added 2 commits March 8, 2023 12:28

Update opensearch_relation_provider.py

6917f42

lint

1c1dd66

WRFitch requested a review from delgod March 8, 2023 13:29

delgod approved these changes Mar 8, 2023

View reviewed changes

delgod merged commit 63b13c1 into main Mar 8, 2023

delgod deleted the dev/add-lib branch March 8, 2023 14:08

carlcsaposs-canonical added a commit that referenced this pull request Mar 18, 2024

Remove unused event handler

80ef9c5

Handler is never called Handler was first added in #30 but a call to the handler was not added

carlcsaposs-canonical mentioned this pull request Mar 18, 2024

Remove unused event handler #205

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Update data_interfaces lib to include opensearch provider #30

Update data_interfaces lib to include opensearch provider #30

WRFitch commented Feb 8, 2023 •

edited

Loading

welpaolo left a comment

welpaolo Mar 7, 2023

WRFitch Mar 7, 2023

WRFitch Mar 7, 2023

deusebio left a comment

Update data_interfaces lib to include opensearch provider #30

Update data_interfaces lib to include opensearch provider #30

Conversation

WRFitch commented Feb 8, 2023 • edited Loading

Proposal

Context

Changelog

Testing

welpaolo left a comment

Choose a reason for hiding this comment

welpaolo Mar 7, 2023

Choose a reason for hiding this comment

WRFitch Mar 7, 2023

Choose a reason for hiding this comment

WRFitch Mar 7, 2023

Choose a reason for hiding this comment

deusebio left a comment

Choose a reason for hiding this comment

WRFitch commented Feb 8, 2023 •

edited

Loading