feat: increase chunk_size for lightweight monitors !! #978
Comments
|
I like the idea of having different limits for the Lightweight and Browser monitors. But we need to do some stress testing on the Kibana side as it might cause some memory spikes when processing 1000s of monitors s well as the how it affects the Integration policies on the Private Locations. @shahzad31 Is there a plan to also increase the limits between Kibana and our globally managed locations? I believe that is already at 250 per location? |
shahzad31
added a commit
to elastic/kibana
that referenced
this issue
Jan 16, 2025
) ## Summary This is to support elastic/synthetics#978 Increase lightweight monitors project page size, size of light weight monitors is minimal, heaving a small size is more of a burden then advantage since we do batch operations in kibana !! ### Why Since limit is only mostly applicable for browser monitors size, for lightweight we can safely do bulk operation on large number of monitors without hititng memory or size issues --------- Co-authored-by: kibanamachine <[email protected]> Co-authored-by: Justin Kambic <[email protected]>
shahzad31
added a commit
to shahzad31/kibana
that referenced
this issue
Jan 20, 2025
…tic#198696) ## Summary This is to support elastic/synthetics#978 Increase lightweight monitors project page size, size of light weight monitors is minimal, heaving a small size is more of a burden then advantage since we do batch operations in kibana !! ### Why Since limit is only mostly applicable for browser monitors size, for lightweight we can safely do bulk operation on large number of monitors without hititng memory or size issues --------- Co-authored-by: kibanamachine <[email protected]> Co-authored-by: Justin Kambic <[email protected]> (cherry picked from commit bfcffa1) # Conflicts: # x-pack/platform/plugins/private/translations/translations/zh-CN.json # x-pack/solutions/observability/plugins/synthetics/server/routes/monitor_cruds/project_monitor/add_monitor_project.ts
cqliu1
pushed a commit
to cqliu1/kibana
that referenced
this issue
Jan 21, 2025
…tic#198696) ## Summary This is to support elastic/synthetics#978 Increase lightweight monitors project page size, size of light weight monitors is minimal, heaving a small size is more of a burden then advantage since we do batch operations in kibana !! ### Why Since limit is only mostly applicable for browser monitors size, for lightweight we can safely do bulk operation on large number of monitors without hititng memory or size issues --------- Co-authored-by: kibanamachine <[email protected]> Co-authored-by: Justin Kambic <[email protected]>
viduni94
pushed a commit
to viduni94/kibana
that referenced
this issue
Jan 23, 2025
…tic#198696) ## Summary This is to support elastic/synthetics#978 Increase lightweight monitors project page size, size of light weight monitors is minimal, heaving a small size is more of a burden then advantage since we do batch operations in kibana !! ### Why Since limit is only mostly applicable for browser monitors size, for lightweight we can safely do bulk operation on large number of monitors without hititng memory or size issues --------- Co-authored-by: kibanamachine <[email protected]> Co-authored-by: Justin Kambic <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
We have 100 default chunk_size for pushing monitors. This makes sense for browser monitors, since there bundle size can be big or small. So we set a very small number of 100, but it doesn't make sense for lightweight monitors, which are very small in size.
Why
small chunk_size add extra load on kibana in terms of processing with subsequent requests, we make bulk operations in kibana and having size of 100 or 1000 won't make a difference but having back to back 10 requests makes batching useless in some aspect.
Solution
We should increase size of chunk for lightweight monitors to 1000 and for browser monitors we can keep it to 100.
Kibana changes
We will also need to update kibana to accept more monitors, since it's also set to 250 at the moment.
The text was updated successfully, but these errors were encountered: