Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Raise inference failure exceptions in default handlers #883

Merged
merged 3 commits into from
Jun 30, 2023
Merged

Raise inference failure exceptions in default handlers #883

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
2c82f33
raise inference failure exceptions in default handlers
rohithkrn Jun 29, 2023
ff1574a
do not catch exception in default handlers
rohithkrn Jun 29, 2023
2ff12f5
Merge branch 'master' into mme_test
rohithkrn Jun 29, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion engines/python/setup/djl_python/deepspeed.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -368,7 +368,7 @@ def inference(self, inputs: Input):
outputs.add_property("content-type", "application/json")
except Exception as e:
logging.exception("DeepSpeed inference failed")
outputs = Output().error((str(e)))
raise e
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we rethrow the exception, we don't need to catch it at the first place

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah, I thought about it too. I did not remove it as I was not sure if the log statement right above provided any value.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the exception will be caught at higher level and will be logged there

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done, updated

return outputs


Expand Down
2 changes: 1 addition & 1 deletion engines/python/setup/djl_python/fastertransformer.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -172,7 +172,7 @@ def inference(self, inputs: Input):
outputs.add(generated_text, key=inputs.get_content().key_at(i))
except Exception as e:
logging.exception("FasterTransformer inference failed")
outputs = Output().error((str(e)))
raise e

return outputs

Expand Down
3 changes: 1 addition & 2 deletions engines/python/setup/djl_python/huggingface.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -215,8 +215,7 @@ def inference(self, inputs):
offset += input_size[i]
except Exception as e:
logging.exception("Huggingface inference failed")
# error handling
outputs = Output().error(str(e))
raise e

return outputs

Expand Down
3 changes: 1 addition & 2 deletions engines/python/setup/djl_python/transformers-neuronx.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -216,8 +216,7 @@ def infer(self, inputs):

except Exception as e:
logging.exception("TransformerNeuronX inference failed")
outputs = Output().error((str(e)))
return outputs
raise e


_service = TransformersNeuronXService()
Expand Down
Loading