Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Example] ggml: add Qwen2-VL example #166

Open
wants to merge 4 commits into
base: master
Choose a base branch
from
Open

[Example] ggml: add Qwen2-VL example #166

wants to merge 4 commits into from

Conversation

dm4
Copy link
Member

@dm4 dm4 commented Dec 24, 2024

Copy link
Member

juntao commented Dec 24, 2024

Hello, I am a code review agent on flows.network. Here are my reviews of changed source code files in this PR.


.github/workflows/llama.yml

Potential issues

N/A

Summary of changes

N/A

wasmedge-ggml/qwen2vl/Cargo.toml

Potential issues

  1. The edition field is set to "2021", which does not correspond to a valid Rust edition; the correct values are "2015" or "2018".
  2. The wasmedge-wasi-nn dependency version "0.7.1" may introduce compatibility issues if it has breaking changes not documented or expected by the rest of the codebase.
  3. Missing dependencies that are likely required for a Qwen2-VL example, such as ggml, should be listed in the [dependencies] section to ensure proper functionality.

Summary of changes

    • Added a new Cargo.toml file for the package wasmedge-ggml-qwen2vl.
  • Specified dependencies: serde_json version 1.0 and wasmedge-wasi-nn version 0.7.1.
  • Defined the Rust edition to be 2021.

wasmedge-ggml/qwen2vl/src/main.rs

Potential issues

  1. Error Handling in get_options_from_env: The use of unwrap() on the result of serde_json::from_str can cause a panic if the JSON is malformed, which should be handled gracefully.

  2. Redundant Metadata Output: In non-interactive mode, the number of input tokens is printed twice from the metadata, which is redundant and should be removed.

  3. Inconsistent Context Reset Logic: The context reset logic in interactive mode only clears saved_prompt without reinitializing the GraphExecutionContext, which might lead to inconsistent states if the model requires a fresh execution context each time.

Summary of changes

  • Key Changes Summary:
  • Added Qwen2-VL Example Code: Introduced a new example for running the Qwen2 Visual-Language (VL) model using wasmedge_wasi_nn for neural network inference.
  • Environment Variable Parsing: Implemented parsing of required and optional environment variables (mmproj, image, enable_log, ctx_size, n_gpu_layers) to configure the model's execution options.
  • Interactive Prompt Handling: Added interactive user input handling and management, including prompt formatting and context resetting mechanisms for continuous dialogue simulation.

wasmedge-ggml/qwen2vl/wasmedge-ggml-qwen2vl.wasm

Potential issues

N/A

Summary of changes

N/A

@dm4 dm4 marked this pull request as draft December 25, 2024 08:03
@dm4 dm4 marked this pull request as ready for review January 2, 2025 08:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants