Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow removal of global configuration properties #3080

Closed

Conversation

cdegroc
Copy link
Contributor

@cdegroc cdegroc commented Jun 16, 2022

Fixes #555

Usage

mgmt = graph.openManagement()
mgmt.set(<config_key>, null)
mgmt.commit()

I've confirmed locally that, once removed, configuration keys can't be retrieved using ManagementSystem#get() and don't appear anymore in graph.getBackend().getGlobalSystemConfig().


Thank you for contributing to JanusGraph!

In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:

For all changes:

  • Is there an issue associated with this PR? Is it referenced in the commit message?
  • Does your PR body contain #xyz where xyz is the issue number you are trying to resolve?
  • Has your PR been rebased against the latest commit within the target branch (typically master)?
  • Is your initial contribution a single, squashed commit?

For code changes:

  • Have you written and/or updated unit tests to verify your changes?
  • If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
  • If applicable, have you updated the LICENSE.txt file, including the main LICENSE.txt file in the root of this repository?
  • If applicable, have you updated the NOTICE.txt file, including the main NOTICE.txt file found in the root of this repository?

For documentation related changes:

  • Have you ensured that format looks appropriate for the output in which it is rendered?

@janusgraph-bot janusgraph-bot added the cla: external Externally-managed CLA label Jun 16, 2022
@li-boxuan li-boxuan added this to the Release v0.6.3 milestone Jun 16, 2022
Copy link
Member

@FlorianHockmann FlorianHockmann left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The changes look good to me in general, but should we briefly mention this in the docs?

Also what about this comment from @robertdale:

If there becomes the ability to clear index.*.backend properties, be sure to document that indexes on that backend would first have to be DISABLE and REMOVE (which effectively disables using that backend).

How will JanusGraph behave if an index backend is configured and in use for some mixed index and the index backend is then removed from the config? Will it still try to use it for traversals that could use that index?

@cdegroc cdegroc force-pushed the removable-global-configuration branch from dbd6547 to 0ea6d70 Compare June 17, 2022 12:13
@li-boxuan
Copy link
Member

How will JanusGraph behave if an index backend is configured and in use for some mixed index and the index backend is then removed from the config? Will it still try to use it for traversals that could use that index?

This definitely needs experiments (or even better, having a test case to showcase that), but AFAIK this will cause read & writes involving that indexing backend to fail. The reason is that schema information is stored in the storage backend, not in the index backend. So JanusGraph will assume the mixed indexes still exist and try connecting to the indexing backend.

I think it could be a good improvement to prevent users from removing an index backend in config if there is an active mixed index for that index backend.

@cdegroc
Copy link
Contributor Author

cdegroc commented Jun 28, 2022

How will JanusGraph behave if an index backend is configured and in use for some mixed index and the index backend is then removed from the config? Will it still try to use it for traversals that could use that index?

Agreed. We should add tests and update the documentation.

Nonetheless, in my understanding, this change is not specific to index backends (it allows removal of any configuration property). In addition, removing an index backend can already be done when using the ConfiguredGraphFactory as shown in this test: cdegroc@e37c9b4. Though, to be fair, I don't think this is documented in the ConfiguredGraphFactory doc.

Copy link
Member

@li-boxuan li-boxuan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am okay with this PR - What @FlorianHockmann said here made sense, but we could create a follow-up issue for this. For now, my suggestion is that we can:

Rather than this breaking change, add a remove API and add some Javadoc saying that removal of config might not work properly under all circumstances (e.g. index settings) so use it at your own risk.

After all, it is clear that this functionality is useful in many cases, so some advanced users could still utilize it. We don't want to mention it in the user doc until we have figured out and documented all edge cases.

@cdegroc
Copy link
Contributor Author

cdegroc commented Aug 2, 2022

Rather than this breaking change, add a remove API and add some Javadoc saying that removal of config might not work properly under all circumstances (e.g. index settings) so use it at your own risk.

After all, it is clear that this functionality is useful in many cases, so some advanced users could still utilize it. We don't want to mention it in the user doc until we have figured out and documented all edge cases.

Thanks @li-boxuan. Adding a remove API would be cleaner and the "use at your own risk" approach sounds right to me.

The disadvantage is that this API will be different from ConfiguredGraphFactory, but we could in turn update ConfiguredGraphFactory to have a similar API (for instance, add a removeConfiguration(String graphName, Configuration config) method.

One issue is that Configuration are key/value collections and so we would have to use removeConfiguration(String graphName, Iterable<String> configKeys).

@li-boxuan
Copy link
Member

The disadvantage is that this API will be different from ConfiguredGraphFactory, but we could in turn update ConfiguredGraphFactory to have a similar API (for instance, add a removeConfiguration(String graphName, Configuration config) method

That sounds good to me. @FlorianHockmann what do you think?

@FlorianHockmann
Copy link
Member

That sounds good to me. @FlorianHockmann what do you think?

Sure, sounds good to me! 👍

@FlorianHockmann
Copy link
Member

FlorianHockmann commented Nov 29, 2022

Just stumbled upon this PR again when looking into the open issues for the 0.6.3 milestone. It's not clear to me any more what we agreed on here exactly 🙁

If I understand our discussion here correctly then what's mostly missing is some addition to the Java docs that explains how this removal works but also warns that it won't work for all options.
Is that correct?
And @cdegroc, are you planning to come back to this PR any time soon? Then we can include it in the 0.6.3 release.

If you come back to this PR, then it probably also makes sense to change the target branch to master and add the labels backport and backport/v0.6. We have recently changed our merging strategy: Instead of requiring contributors to target v0.6 and then merge the PR first into v0.6 and then merge that into master, we now only target master and let a bot create a separate backporting PR if we want a change to also land into v0.6.
But we can alternatively probably also merge this into v0.6 and then cherry-pick the commit into master.

This seems to be still open:

The disadvantage is that this API will be different from ConfiguredGraphFactory, but we could in turn update ConfiguredGraphFactory to have a similar API (for instance, add a removeConfiguration(String graphName, Configuration config) method.

Do we want to do this as part of this PR or in a follow-up one? Doing it in a different PR is OK to me, but could you please create an issue for that, @cdegroc? (I think you have a better understanding of this to properly explain what needs to be done.)

@porunov
Copy link
Member

porunov commented Feb 17, 2023

To me it looks like a good feature to have but not a critical bug. Thus, I propose to reschedule this PR to v0.6.4 or 1.0.0. And start 0.6.3 release process.

@li-boxuan
Copy link
Member

li-boxuan commented Oct 7, 2023

I would like to have this PR in 1.0.0, so I am inclined to merge it. We could add an one-liner in release notes that it's a breaking change. Since it's a breaking change, maybe we shouldn't put it in 0.6.4, but I don't have a strong opinion yet.

@li-boxuan li-boxuan force-pushed the removable-global-configuration branch from 0ea6d70 to c7f8d58 Compare October 7, 2023 07:19
@li-boxuan li-boxuan force-pushed the removable-global-configuration branch from c7f8d58 to 3c2fb66 Compare October 7, 2023 07:20
@cdegroc
Copy link
Contributor Author

cdegroc commented Oct 17, 2023

👋 Sorry for not answering earlier. I'll try to catch up and see if there's anything I can help with.

@cdegroc
Copy link
Contributor Author

cdegroc commented Oct 17, 2023

Created #4061 to address comments regarding the interface. Let me know what you think.

@li-boxuan li-boxuan closed this Oct 18, 2023
@li-boxuan li-boxuan removed this from the Release v1.0.0 milestone Oct 18, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cla: external Externally-managed CLA
Projects
None yet
Development

Successfully merging this pull request may close these issues.

JanusGraphManagement does not allow to remove any property once set
5 participants