[DRAFT] Skeleton changes for new memory limiter models #12558
+269
−79
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Introduces the changes described in the comment open-telemetry/opentelemetry-go#6404 (comment). This adds configuration to the
memorylimterextension
package with amodel
("gcstats" or "admission") and a new sub-struct containing the admission-control-based configuration.This also adds a
Controller
interface with a no-op implementation. The real implementation of this is in collector-contrib/internal/otelarrow.Our goal is to automate the insertion of gRPC and HTTP-specific interceptors to make it easy to ues the controller interface defined here. I believe the way we will do this is to modify the existing code points, such as inside (*configgrpc.ServerConfig) getGrpcServerOptions(). In these locations, we will apply an inteceptor to count the bytes of the request and call the controller to:
I'm not sure precisely how the controller will be injected to the handler used in the receivers. In other words, while constructing the receivers we want to easily (for all components) inspect the Host for a memorylimiter extension, which will be either GCStats or Admission-based, but either way implement the controller interface. The interceptors will be provided with the controller, etc.
Note that some receivers will require direct access to the controller because they do not use interceptors, or because they use custom gRPC protocols with internal compression. For example, a Prometheus receiver would call controller.Acquire() directly. For example, the OTel-Arrow stream receiver decompresses each stream request and Acquires the uncompressed size before returning from
Send()
.Link to tracking issue
Part of #9591.
Testing
The OTel-Arrow components, where the Admission-based logic originates, have an end-to-end test of the behavior of this component.
Documentation
TODO: The README.md for memorylimiter extension refers to the memorylimiter processor. This situation should be reversed. The memorylimiter processor shall be scheduled for removal after these changes are complete.