From 648571369d36d13089c0fe9910824093d14af00c Mon Sep 17 00:00:00 2001 From: Charlie Ruan <53290280+CharlieFRuan@users.noreply.github.com> Date: Fri, 22 Nov 2024 10:18:24 -0800 Subject: [PATCH] [Docs] Add portable API (#87) Add Portable API to How-To --- docs/how_to/engine_integration.rst | 4 ++-- docs/how_to/portable_api.rst | 25 +++++++++++++++++++++++++ docs/index.rst | 1 + 3 files changed, 28 insertions(+), 2 deletions(-) create mode 100644 docs/how_to/portable_api.rst diff --git a/docs/how_to/engine_integration.rst b/docs/how_to/engine_integration.rst index e1d8a44..1b6f3c1 100644 --- a/docs/how_to/engine_integration.rst +++ b/docs/how_to/engine_integration.rst @@ -97,8 +97,8 @@ as we will see in :ref:`how-to-engine-integration-batched`. # Instantiate grammar matcher with the compiled grammar matcher = xgr.GrammarMatcher(compiled_grammar) -Auto-regressive Generation with xgr.GrammarMatcher -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +Bitmasking Logits in Auto-regressive Generation +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Now we simulate a single-request auto-regressive generation. See later section for :ref:`how-to-engine-integration-batched`. diff --git a/docs/how_to/portable_api.rst b/docs/how_to/portable_api.rst new file mode 100644 index 0000000..103340c --- /dev/null +++ b/docs/how_to/portable_api.rst @@ -0,0 +1,25 @@ +.. _how-to-portable-api: + +Portable API +============ + +XGrammar is implemented with a lightweight C++ core that can be integrated into many platforms. +Besides the C++ backend, we also provide ready-to-use Python and JavaScript/TypeScript libraries. + +For the Python library, simply check out the :ref:`Python API Reference `. Below +we take a high-level view of the Javascript library. + +.. _how-to-portable-js: + +Javascript SDK for Web-based LLMs +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The JS SDK is designed to be used for LLMs that run in the browser, including +`WebLLM `__. WebLLM integrated with XGrammar's +JS SDK, ``web-xgrammar``. It uses `emscripten `__ to compile +the C++ code into WebAssembly. + +To use this SDK, simply run ``npm install @mlc-ai/web-xgrammar``. For more, see +`here `__. + + diff --git a/docs/index.rst b/docs/index.rst index 6ea3c72..60dc88d 100644 --- a/docs/index.rst +++ b/docs/index.rst @@ -30,6 +30,7 @@ Check out :ref:`quick-start` for quick start examples of using XGrammar. how_to/json_generation.rst how_to/ebnf_guided_generation.rst how_to/engine_integration.rst + how_to/portable_api.rst .. tutorials/web_sdk.rst .. TODO