Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Kf 2271 feat tensorflow serving rock #32

Merged
merged 7 commits into from
Jul 28, 2023

Conversation

i-chvets
Copy link
Contributor

@i-chvets i-chvets commented Jul 26, 2023

Design and implementation details are in canonical/seldon-core-operator#133

Summary of changes:

  • Created rockcraft file for tensorflow/serving rock according to best practices spec.

NOTE: This is heavy ROCK. It builds in around 2 hours on 32 core machine.

Manual testing shows that all artefacts are in place:

$ docker run tensorflow-serving:2.13.0_20.04_1 exec pebble exec bash -c 'tensorflow_model_server --port=8500 --rest_api_port=8501 --model_name=${MODEL_NAME} --model_base_path=${MODEL_BASE_PATH}/${MODEL_NAME} "$@"'
2023-07-26T13:31:39.690Z [pebble] Started daemon.
2023-07-26T13:31:39.699Z [pebble] POST /v1/exec 8.818296ms 202
2023-07-26T13:31:39.712Z [pebble] GET /v1/tasks/1/websocket/control 12.437858ms 200
2023-07-26T13:31:39.713Z [pebble] GET /v1/tasks/1/websocket/stdio 121.315µs 200
2023-07-26T13:31:39.713Z [pebble] GET /v1/tasks/1/websocket/stderr 52.198µs 200
2023-07-26T13:31:39.727Z [pebble] POST /v1/exec 8.69893ms 202
2023-07-26T13:31:39.735Z [pebble] GET /v1/tasks/2/websocket/control 8.076374ms 200
2023-07-26T13:31:39.736Z [pebble] GET /v1/tasks/2/websocket/stdio 56.035µs 200
2023-07-26T13:31:39.736Z [pebble] GET /v1/tasks/2/websocket/stderr 54.31µs 200
2023-07-26 13:31:39.801036: I external/org_tensorflow/tensorflow/core/util/port.cc:110] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
2023-07-26 13:31:39.802675: I tensorflow_serving/model_servers/server.cc:74] Building single TensorFlow model file config:  model_name:  model_base_path: /
2023-07-26 13:31:39.802808: I tensorflow_serving/model_servers/server_core.cc:465] Adding/updating models.
2023-07-26 13:31:39.802814: I tensorflow_serving/model_servers/server_core.cc:594]  (Re-)adding model: 

Ivan Chvets added 4 commits July 8, 2023 13:05
canonical/seldon-core-operator#133

Summary of changes:
- Initial rockcraft file and tox.ini
Summary of changes:
- Added organize.
- Finalized bazel build.
Summary of changes:
- Initial tensorflow/serving rock.
@i-chvets i-chvets requested a review from a team as a code owner July 26, 2023 13:20
@i-chvets
Copy link
Contributor Author

i-chvets commented Jul 26, 2023

Fails to be started by workload in integration tests:

 microk8s.kubectl -n test-seldon-servers-nsoi logs tfserving-default-0-classifier-7ddd9bd9dc-twk67
Defaulted container "classifier" out of: classifier, seldon-container-engine, classifier-model-initializer (init)
error: unknown flag `port'

Port is part of config for tensorflow/seving. Needs more debugging.

@i-chvets
Copy link
Contributor Author

@SamirPS Please review. Integration tests will be addressed in another work item that is planned next. For now just a review of the ROCK itself.

Copy link

@SamirPS SamirPS left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In order to improve the user experience and ensure the maintaining of this ROCK, it may be good to split the Python and Bazel steps into separate parts, if possible.

tensorflow-serving/rockcraft.yaml Show resolved Hide resolved
tensorflow-serving/rockcraft.yaml Show resolved Hide resolved
tensorflow-serving/rockcraft.yaml Outdated Show resolved Hide resolved
tensorflow-serving/rockcraft.yaml Outdated Show resolved Hide resolved
@i-chvets
Copy link
Contributor Author

@SamirPS Installing python3.8 via packages worked fine. Ready for review.

tensorflow-serving/rockcraft.yaml Outdated Show resolved Hide resolved
tensorflow-serving/rockcraft.yaml Outdated Show resolved Hide resolved
tensorflow-serving/rockcraft.yaml Outdated Show resolved Hide resolved
@i-chvets i-chvets merged commit c7a28af into main Jul 28, 2023
@i-chvets i-chvets deleted the kf-2271-feat-tensorflow-serving-rock branch July 28, 2023 13:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants