[Bug] UnicodeDecodeError during inference w/ json schema

[AvivaTang]

I deployed llama3.1-8b based on sglang0.4.0+xgrammar0.1.7, and used json schema during inference. The following error message appeared and my service failed directly. How can I solve this problem?

Scheduler hit an exception: Traceback (most recent call last):
File "/sgl-workspace/sglang/python/sglang/srt/managers/scheduler.py", line 1500, in run_scheduler_process
scheduler.event_loop_normal()
File "/usr/local/lib/python3.10/dist-packages/torch/utils/_contextlib.py", line 116, in decorate_context
return func(*args, **kwargs)
File "/sgl-workspace/sglang/python/sglang/srt/managers/scheduler.py", line 384, in event_loop_normal
batch = self.get_next_batch_to_run()
File "/sgl-workspace/sglang/python/sglang/srt/managers/scheduler.py", line 740, in get_next_batch_to_run
self.running_batch = self.update_running_batch(self.running_batch)
File "/sgl-workspace/sglang/python/sglang/srt/managers/scheduler.py", line 890, in update_running_batch
jump_forward_reqs = batch.check_for_jump_forward(self.pad_input_ids_func)
File "/sgl-workspace/sglang/python/sglang/srt/managers/schedule_batch.py", line 908, in check_for_jump_forward
jump_helper = req.grammar.try_jump_forward(req.tokenizer)
File "/sgl-workspace/sglang/python/sglang/srt/constrained/xgrammar_backend.py", line 54, in try_jump_forward
s = self.matcher.find_jump_forward_string()
File "/usr/local/lib/python3.10/dist-packages/xgrammar/matcher.py", line 240, in find_jump_forward_string
return self._handle.find_jump_forward_string()
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x80 in position 0: invalid start byte

[Ubospica]

Now the integration of XGrammar in SGLang could not support sub-utf8 characters in jumpforward decoding. We are working to handle that issue, but you can disable jumpforward decoding for now.

[merrymercy]

Try the latest version. This problem should be fixed because we do not turn on jump forward by default in the new versions.

[AvivaTang]

Thanks, I'll try it.