Skip to content

Commit

Permalink
[Docs] Add portable API (#87)
Browse files Browse the repository at this point in the history
Add Portable API to How-To
  • Loading branch information
CharlieFRuan authored Nov 22, 2024
1 parent 8b47fa1 commit 6485713
Show file tree
Hide file tree
Showing 3 changed files with 28 additions and 2 deletions.
4 changes: 2 additions & 2 deletions docs/how_to/engine_integration.rst
Original file line number Diff line number Diff line change
Expand Up @@ -97,8 +97,8 @@ as we will see in :ref:`how-to-engine-integration-batched`.
# Instantiate grammar matcher with the compiled grammar
matcher = xgr.GrammarMatcher(compiled_grammar)
Auto-regressive Generation with xgr.GrammarMatcher
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Bitmasking Logits in Auto-regressive Generation
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Now we simulate a single-request auto-regressive generation. See later section for
:ref:`how-to-engine-integration-batched`.
Expand Down
25 changes: 25 additions & 0 deletions docs/how_to/portable_api.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
.. _how-to-portable-api:

Portable API
============

XGrammar is implemented with a lightweight C++ core that can be integrated into many platforms.
Besides the C++ backend, we also provide ready-to-use Python and JavaScript/TypeScript libraries.

For the Python library, simply check out the :ref:`Python API Reference <apixgrammar>`. Below
we take a high-level view of the Javascript library.

.. _how-to-portable-js:

Javascript SDK for Web-based LLMs
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

The JS SDK is designed to be used for LLMs that run in the browser, including
`WebLLM <https://github.com/mlc-ai/web-llm>`__. WebLLM integrated with XGrammar's
JS SDK, ``web-xgrammar``. It uses `emscripten <https://emscripten.org/>`__ to compile
the C++ code into WebAssembly.

To use this SDK, simply run ``npm install @mlc-ai/web-xgrammar``. For more, see
`here <https://github.com/mlc-ai/xgrammar/tree/main/web>`__.


1 change: 1 addition & 0 deletions docs/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,7 @@ Check out :ref:`quick-start` for quick start examples of using XGrammar.
how_to/json_generation.rst
how_to/ebnf_guided_generation.rst
how_to/engine_integration.rst
how_to/portable_api.rst
.. tutorials/web_sdk.rst .. TODO


Expand Down

0 comments on commit 6485713

Please sign in to comment.