Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Chore] Reorganize engine.ts #536

Merged
merged 1 commit into from
Aug 10, 2024
Merged

Conversation

CharlieFRuan
Copy link
Contributor

@CharlieFRuan CharlieFRuan commented Aug 10, 2024

This PR reorganizes engine.ts to keep it relatively more organized.

  • We move getToolCallFromOutputMessage() from engine.ts to support.ts since it does not depend on any engine-related fields and is stateless
  • We move asyncLoadTokenizer() from engine.ts to cache_util.ts for the same reason
  • In future, we can implement a common helper for chatCompletion() and completion()
  • For engine.ts, we break down the functions into sections:
  1. Setters and getters
  2. Model/pipeline loading and unloading: reload(), unload()
  3. Underlying auto-regressive generation functions: _generate(), asyncGenerate(), interruptGenerate()
  4. High-level generation APIs: generate() (deprecated), chatCompletion(), completion()
  5. WebGPU info-querying helpers. Arguably, this can be implemented in support.ts as well. Currently these methods are in the MLCEngineInterface, and goes through the WebWorkerMLCEngine - MLCEngine message passing workflow
  6. Low-level APIs that interact with pipeline

@CharlieFRuan CharlieFRuan merged commit d21f00e into mlc-ai:main Aug 10, 2024
1 check passed
CharlieFRuan added a commit that referenced this pull request Aug 12, 2024
### Changes
- Reorganized internal code, which should have no affect on behaviors
#536
- Bumped version of tokenizers-cpp to fix potential error `The global
thread pool has not been initialized.`
  - For more, see mlc-ai/tokenizers-cpp#42

### TVMjs
- Still compiled at
apache/tvm@1fcb620,
no change
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant