Skip to content

🐛 [Bug] Weight streaming test fail #3460

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
keehyuna opened this issue Apr 2, 2025 · 0 comments · Fixed by #3461
Closed

🐛 [Bug] Weight streaming test fail #3460

keehyuna opened this issue Apr 2, 2025 · 0 comments · Fixed by #3461
Assignees
Labels
bug Something isn't working

Comments

@keehyuna
Copy link
Collaborator

keehyuna commented Apr 2, 2025

Bug Description

Segmentation fault error from test_004_weight_streaming test. cudagraph.reset() is in call stack

File "/usr/local/lib/python3.12/dist-packages/torch/cuda/graphs.py", line 92 in reset
File "/usr/local/lib/python3.12/dist-packages/torch_tensorrt/dynamo/runtime/_CudaGraphsTorchTensorRTModule.py", line 83 in forward
File "/usr/local/lib/python3.12/dist-packages/torch/nn/modules/module.py", line 1762 in _call_impl
File "/usr/local/lib/python3.12/dist-packages/torch/nn/modules/module.py", line 1751 in _wrapped_call_impl
File "/opt/pytorch/torch_tensorrt/tests/py/dynamo/runtime/test_004_weight_streaming.py", line 364 in test_trt_model
File "/opt/pytorch/torch_tensorrt/tests/py/dynamo/runtime/test_004_weight_streaming.py", line 385 in test_runtime_state_change

To Reproduce

Steps to reproduce the behavior:

  1. pytest test_004_weight_streaming.py

Expected behavior

Environment

Build information about Torch-TensorRT can be found by turning on debug messages

  • Torch-TensorRT Version (e.g. 1.0.0):
  • PyTorch Version (e.g. 1.0):
  • CPU Architecture:
  • OS (e.g., Linux):
  • How you installed PyTorch (conda, pip, libtorch, source):
  • Build command you used (if compiling from source):
  • Are you using local sources or building from archives:
  • Python version:
  • CUDA version:
  • GPU models and configuration:
  • Any other relevant information:

Additional context

@keehyuna keehyuna added the bug Something isn't working label Apr 2, 2025
@keehyuna keehyuna self-assigned this Apr 2, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant