Skip to content

Commit

Permalink
[Integration][Gitlab] Fix Memory Leak Identified In Live Events Sync (#…
Browse files Browse the repository at this point in the history
…1371)

##Description

What

Currently, when a live event is received, we pass a deep copy of the
event object to each observer. This was originally implemented to ensure
that every observer received its own, uniquely referenced copy of the
event. However, in our current workflow, observers do not modify the
event data—they only use it to fetch resources from GitLab.

Why
- Since the observers only perform read operations (i.e., fetching data)
and do not modify the event object, creating deep copies is redundant.
- It has been observed that the deep‑copied objects are not being
garbage collected after use, which results in increased memory usage and
could eventually lead to memory exhaustion.
- The deep copy operation introduces additional processing time and
memory overhead that is not needed given the current use case.

How
- Modify the event handling logic to pass the original event object (or
a shallow copy if absolutely necessary for safety) to observers instead
of performing a deep copy.
- Since observers only read from the event data, they can safely operate
on the same shared object without risking race conditions or data
corruption.

## Type of change

Please leave one option from the following and delete the rest:

- [x] Bug fix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] New Integration (non-breaking change which adds a new integration)
- [ ] Breaking change (fix or feature that would cause existing
functionality to not work as expected)
- [ ] Non-breaking change (fix of existing functionality that will not
change current behavior)
- [ ] Documentation (added/updated documentation)

<h4> All tests should be run against the port production
environment(using a testing org). </h4>

### Core testing checklist

- [ ] Integration able to create all default resources from scratch
- [ ] Resync finishes successfully
- [ ] Resync able to create entities
- [ ] Resync able to update entities
- [ ] Resync able to detect and delete entities
- [ ] Scheduled resync able to abort existing resync and start a new one
- [ ] Tested with at least 2 integrations from scratch
- [ ] Tested with Kafka and Polling event listeners
- [ ] Tested deletion of entities that don't pass the selector


### Integration testing checklist

- [ ] Integration able to create all default resources from scratch
- [ ] Resync able to create entities
- [ ] Resync able to update entities
- [ ] Resync able to detect and delete entities
- [ ] Resync finishes successfully
- [ ] If new resource kind is added or updated in the integration, add
example raw data, mapping and expected result to the `examples` folder
in the integration directory.
- [ ] If resource kind is updated, run the integration with the example
data and check if the expected result is achieved
- [ ] If new resource kind is added or updated, validate that
live-events for that resource are working as expected
- [ ] Docs PR link [here](#)

### Preflight checklist

- [ ] Handled rate limiting
- [ ] Handled pagination
- [ ] Implemented the code in async
- [ ] Support Multi account

## Screenshots

Include screenshots from your environment showing how the resources of
the integration will look.

## API Documentation

Provide links to the API documentation used for this integration.
  • Loading branch information
mk-armah authored Feb 11, 2025
1 parent 055a7b7 commit ee72cf9
Show file tree
Hide file tree
Showing 3 changed files with 11 additions and 3 deletions.
8 changes: 8 additions & 0 deletions integrations/gitlab/CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,14 @@ this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.htm

<!-- towncrier release notes start -->

0.2.34 (2025-02-11)
===================

### Improvements

- Removed the need for sending a deepcopy of the event body to all listeners, which previously led to a memory leak


0.2.33 (2025-02-09)
===================

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -98,7 +98,7 @@ async def _notify(self, event: str, body: dict[str, Any]) -> None:
event=event,
handler=handler,
)
asyncio.create_task(observer(event, deepcopy(body))) # type: ignore
asyncio.create_task(observer(event, body)) # type: ignore


class SystemEventHandler(BaseEventHandler):
Expand All @@ -119,7 +119,7 @@ async def _notify(self, event: str, body: dict[str, Any]) -> None:
# access the project
results = await asyncio.gather(
*(
hook_handler(client).on_hook(event, deepcopy(body))
hook_handler(client).on_hook(event, body)
for client in self._clients
for hook_handler in self._hook_handlers.get(event, [])
),
Expand Down
2 changes: 1 addition & 1 deletion integrations/gitlab/pyproject.toml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[tool.poetry]
name = "gitlab"
version = "0.2.33"
version = "0.2.34"
description = "Gitlab integration for Port using Port-Ocean Framework"
authors = ["Yair Siman-Tov <[email protected]>"]

Expand Down

0 comments on commit ee72cf9

Please sign in to comment.