Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Hook capability for event changes and deletions #1092

Merged
merged 39 commits into from
Mar 2, 2024

Conversation

freakmaxi
Copy link
Contributor

Implemented the none and RabbitMQ hook capability for events

@Unrud
Copy link
Collaborator

Unrud commented Aug 17, 2020

I don't think this will work.

  • Changes to the storage and running hooks are not atomic. Both can easily get out-of-sync in case of some error or system crash.
  • The filesystem storage can get changed manually or by external programs. Hooks will never run for those changes.
  • You are missing operations that change the storage (e.g. moving items)

My suggestion: Write a program that reads the filesystem storage folder directly and sends messages to RabbitMQ (as an example) for all new and deleted items since it was last run. Start the program via storage->hook. Additionally run it on startup and at time intervals to catch external changes and crashes.

@freakmaxi
Copy link
Contributor Author

Honestly, manually or external program changes should be omitted because if we will consider touching files outside of the service, what will be the point of having the radicale. So I can write a program which will be the replacement of caldav service. Because of this reason, my hook implementation is just covering the changes that are triggered by the requests made to radicale.

Currently, I'm testing the service stability on my software. Around 30k people are using right now and planning to tune the performance for 5M people and also thinking to add clustering capability or piling/grouping by hash and make the service reachable by a reverse proxy head because one service will not be enough for all those people and radicale should be scalable somehow.

You are right, I've not implemented the hook for moving items.

PS: Nobody needed to approve this pull request. They can wait until I make it stabile after heavy usage or if they want, they can completely drop it. Returning the improvements on the code to the source is my responsibility. That's why we have this PR.

@pbiering pbiering added feature need:reporter feedback feedback from reporter required labels Mar 1, 2024
@pbiering
Copy link
Collaborator

pbiering commented Mar 1, 2024

@freakmaxi : this changes a lot, is this still valid?

@freakmaxi
Copy link
Contributor Author

Hey,

Yes it is valid and working and help us to get the "change" notifications immediately in our backend services.

Thanks

@pbiering
Copy link
Collaborator

pbiering commented Mar 1, 2024

Yes it is valid and working and help us to get the "change" notifications immediately in our backend services.

In this case please rebase to Radicale 3 "latest"

@freakmaxi
Copy link
Contributor Author

This can be a bit tricky but I'll try next week. Or if you don't have time to wait, you can fork from my repo and merge with the source to solve the conflicts

@pbiering pbiering added this to the 3.2.x milestone Mar 2, 2024
@pbiering
Copy link
Collaborator

pbiering commented Mar 2, 2024

No hurry here as being a new feature and if then integrated in 3.2.x

@freakmaxi
Copy link
Contributor Author

I told next week but plans are changed on my side. I've done it this week. it is currently ready. you can take a look

@pbiering
Copy link
Collaborator

pbiering commented Mar 2, 2024

Is pika>=1.3.2 a hard requirement? I'm asking because EPEL9 only supports currently 1.2.1 and Fedora 39 only 1.3.1, would mean in RPMs it must be bundled.

@freakmaxi
Copy link
Contributor Author

nope 1.1.0 and up is okay.

@pbiering pbiering mentioned this pull request Mar 2, 2024
@pbiering
Copy link
Collaborator

pbiering commented Mar 2, 2024

nope 1.1.0 and up is okay.

can you please update your PR?

@pbiering
Copy link
Collaborator

pbiering commented Mar 2, 2024

workflow check triggered but not successful, please investigate, details see above.

@freakmaxi
Copy link
Contributor Author

retry please

@pbiering
Copy link
Collaborator

pbiering commented Mar 2, 2024

still not happy, next issue found...

@freakmaxi
Copy link
Contributor Author

it'll be okay this time. re-trigger please...

@pbiering pbiering removed the need:reporter feedback feedback from reporter required label Mar 2, 2024
@pbiering pbiering changed the base branch from master to v3.2-devel March 2, 2024 19:04
@pbiering pbiering self-assigned this Mar 2, 2024
@pbiering pbiering merged commit 989cbef into Kozea:v3.2-devel Mar 2, 2024
22 checks passed
@pbiering
Copy link
Collaborator

pbiering commented Mar 2, 2024

finally looks good, merged into new v3.2-devel branch

@pbiering pbiering modified the milestones: 3.2.x, 3.2.0 Mar 13, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants