From 610f4c22b0ca718ff8a17e844be1dd49f2f4df0a Mon Sep 17 00:00:00 2001 From: dm4 Date: Mon, 28 Oct 2024 16:52:38 +0800 Subject: [PATCH] [Docs] ggml: add notes about using wasi_nn config Signed-off-by: dm4 --- wasmedge-ggml/README.md | 11 +++++++++-- 1 file changed, 9 insertions(+), 2 deletions(-) diff --git a/wasmedge-ggml/README.md b/wasmedge-ggml/README.md index ca8148d..e8e9248 100644 --- a/wasmedge-ggml/README.md +++ b/wasmedge-ggml/README.md @@ -207,9 +207,16 @@ Currently, the WASI-NN ggml plugin supports several ways to set the metadata for When constructing the graph, you can set the metadata by using the `config` method. ```rust -... wasi_nn::GraphBuilder::new(...).config(serde_json::to_string(&options).unwrap()) ... +wasmedge_wasi_nn::GraphBuilder::new(...) + .config(serde_json::to_string(&options).unwrap()) + .build_from_cache(...) + .unwrap(); ``` +> [!NOTE] +> The config will only be set when constructing the graph using `build_from_cache`. +> Due to the file size limitation, you **SHOULD** use `build_from_cache` with `--nn-preload` to load the large model file. + 2. From the input tensor When setting input to the context, specify the index with 1 for the metadata. @@ -220,7 +227,7 @@ If you modify the `n-gpu-layers` parameter, the model will be reloaded. context .set_input( 1, - wasi_nn::TensorType::U8, + wasmedge_wasi_nn::TensorType::U8, &[1], serde_json::to_string(&options).expect("Failed to serialize options").as_bytes().to_vec(), )