Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix pushing large files over SCP #661

Merged
merged 5 commits into from
Mar 24, 2025
Merged

Fix pushing large files over SCP #661

merged 5 commits into from
Mar 24, 2025

Conversation

Jakuje
Copy link
Contributor

@Jakuje Jakuje commented Nov 15, 2024

SUMMARY

The SCP is trying to read the whole file into memory and send it as a whole to the server. This is very prone to hit some memory limits of the machine with larger files.

This changes the implements similar approach to the get() operation as done in #621, including very similar test.

Fixes: #654

ISSUE TYPE
  • Bugfix Pull Request

@psf-chronographer psf-chronographer bot added the bot:chronographer:provided There is a change note present in this PR label Nov 15, 2024
Copy link

Congratulations! One of the builds has completed. 🍾

You can install the built RPMs by following these steps:

  • sudo yum install -y dnf-plugins-core on RHEL 8
  • sudo dnf install -y dnf-plugins-core on Fedora
  • dnf copr enable packit/ansible-pylibssh-661
  • And now you can install the packages.

Please note that the RPMs should be used only in a testing environment.

@webknjaz
Copy link
Member

Looks like tests/unit/channel_test.py::test_request_exec times out under Python 3.9 on macOS 13 (and possibly others?)

(https://github.com/ansible/pylibssh/actions/runs/12434120667/job/34717468112?pr=661#step:17:68)

@Jakuje
Copy link
Contributor Author

Jakuje commented Dec 23, 2024

Looks like tests/unit/channel_test.py::test_request_exec times out under Python 3.9 on macOS 13 (and possibly others?)

(https://github.com/ansible/pylibssh/actions/runs/12434120667/job/34717468112?pr=661#step:17:68)

This should be fixed by #658 I hope.

@webknjaz
Copy link
Member

Let's rebase this.

@Jakuje
Copy link
Contributor Author

Jakuje commented Mar 21, 2025

Let's rebase this.

Sure. I guess I will rework the test similarly as we did in the sftp case with parametrization once I will touch it.

Jakuje added 2 commits March 21, 2025 09:24
Similarly as the sftp tests do now.

Signed-off-by: Jakub Jelen <[email protected]>
Signed-off-by: Jakub Jelen <[email protected]>

This comment was marked as spam.

Copy link

codecov bot commented Mar 21, 2025

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 95.53%. Comparing base (4250532) to head (d1ed4b7).
Report is 9 commits behind head on devel.

❌ Your project status has failed because the head coverage (95.53%) is below the target coverage (100.00%). You can increase the head coverage or adjust the target coverage.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@webknjaz webknjaz merged commit 80bccfe into ansible:devel Mar 24, 2025
72 of 77 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bot:chronographer:provided There is a change note present in this PR
Projects
None yet
Development

Successfully merging this pull request may close these issues.

scp.put fails with large files
2 participants