Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

new(anomalydetection): Initial Scope - CountMinSketch Powered Probabilistic Counting and Filtering #419

Open
wants to merge 17 commits into
base: main
Choose a base branch
from

Conversation

incertum
Copy link
Contributor

@incertum incertum commented Feb 27, 2024

What type of PR is this?

Uncomment one (or more) /kind <> lines:

/kind bug

/kind cleanup

/kind design

/kind documentation

/kind failing-test

/kind feature

Any specific area of the project related to this PR?

Uncomment one (or more) /area <> lines:

/area plugins

/area registry

/area build

/area documentation

What this PR does / why we need it:

Introduce a new anomalydetection plugin, as outlined in the Proposal.

Which issue(s) this PR fixes:

Fixes falcosecurity/falco#3117

falcosecurity/falco#3117

Fixes #

Special notes for your reviewer:

Sharing some hopefully useful notes:

  • Check out the README first, as it summarizes current limitations and the initial scope
  • Much of the code was duplicated from falcosecurity/libs; I’ve indicated this to show that this code doesn’t need review here. If you find issues, let’s address them in libs instead
  • Hot reloading seems to be working (no segfaults observed), except toggling enabled/disabled during a hot reload doesn’t work, which we can likely ignore
  • The project builds in a container suitable for building Falco, so we can set up CI
  • Initial local tests confirm the intended functionality, including the reset timers
  • Unit tests cover the most critical functions but could still be improved
  • cms.h contains the algorithm/math (it's probabilistic counting only, not AI/ML); the rest is mostly generic plugin setup or configuration.
  • Config initialization is designed to error out if the plugin is used for currently unsupported cases
  • The profile field filtercheck code might be removed if libs filterchecks are opened to plugins, which would make it more stable and less risky. Additionally, translating Falco filtercheck logic to the plugin framework was very tedious
  • Further testing on more robust servers is needed to check for performance and stability issues, such as CPU spikes or segfaults after extended runs
  • Most things had to be wrapped in try-catch statements, as simple checks for nullptr were not possible ...

For this first version, I am primarily concerned about performance and stability (no segfaults). I would truly appreciate a very critical review in this regard. Thanks a bunch in advance!

incertum and others added 15 commits August 21, 2024 15:49
…ts + start fd related filterchecks 2/n

Signed-off-by: Melissa Kilby <[email protected]>
…n cases of missing fd table entry

Signed-off-by: Melissa Kilby <[email protected]>
…+ better comments/docs clarity

Signed-off-by: Melissa Kilby <[email protected]>
@incertum incertum marked this pull request as ready for review August 22, 2024 19:47
@incertum incertum changed the title wip: new(anomalydetection): Initial Scope - CountMinSketch Powered Probabilistic Counting and Filtering new(anomalydetection): Initial Scope - CountMinSketch Powered Probabilistic Counting and Filtering Aug 22, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[TRACKING] New anomalydetection Plugin - Targeting First Release for Falco 0.39.0
3 participants