Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[GPU] fix memory conflict for multi iteration in loop. #28487

Merged
merged 1 commit into from
Jan 17, 2025

Conversation

timxu826
Copy link
Contributor

Cause is shown in graph below.

The black, green, and dotted line donates the primitive dependency, memory buffer reuse, and memory conflict (at the second iteration) respectively.
The body of the loop is compiled as a separate model and is not aware the data reuse in backedge in multiple iteration.

Tickets:
CVS-158017

@timxu826 timxu826 requested review from a team as code owners January 16, 2025 12:43
@github-actions github-actions bot added the category: GPU OpenVINO GPU plugin label Jan 16, 2025
@sys-openvino-ci sys-openvino-ci added the ExternalIntelPR External contributor from Intel label Jan 16, 2025
@timxu826
Copy link
Contributor Author

Run perf test on bs=2 on TF_Mask_RCNN_Inception_ResNet_v2_atrous_coco

w/ changes:

[ INFO ] Count:               16 iterations
[ INFO ] Duration:            98201.39 ms
[ INFO ] Latency:
[ INFO ]    Median:           24458.30 ms
[ INFO ]    Average:          22789.46 ms
[ INFO ]    Min:              9271.47 ms
[ INFO ]    Max:              27776.03 ms
[ INFO ] Throughput:          0.33 FPS

w/o changes:
[ INFO ] Execution Devices: [ GPU.0 ]
[ INFO ] Count:               16 iterations
[ INFO ] Duration:            98279.91 ms
[ INFO ] Latency:
[ INFO ]    Median:           24466.44 ms
[ INFO ]    Average:          22804.37 ms
[ INFO ]    Min:              9301.48 ms
[ INFO ]    Max:              27782.94 ms
[ INFO ] Throughput:          0.33 FPS

No performance drop observed.

@artanokhov
Copy link
Contributor

@timxu826 please rebase your PR, the fix for doc merged 1h ago
9d06bdf

@p-durandin
Copy link
Contributor

build_jenkins

@yuanxion
Copy link
Contributor

yuanxion commented Jan 17, 2025

@timxu826 please rebase your PR, the fix for doc merged 1h ago 9d06bdf

@artanokhov Done, rebased from master.

@yuanxion
Copy link
Contributor

This PR is a continuation of the previous PR #28056.

@yuanxion
Copy link
Contributor

build_jenkins

@yuanxion yuanxion added this pull request to the merge queue Jan 17, 2025
Merged via the queue into openvinotoolkit:master with commit 7da364d Jan 17, 2025
168 checks passed
github-merge-queue bot pushed a commit that referenced this pull request Jan 17, 2025
### Details:
 - backport of #28487

### Tickets:
 - 158017

Co-authored-by: Xu, Jia3 <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
category: GPU OpenVINO GPU plugin ExternalIntelPR External contributor from Intel
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants