Fix streaming hieroglyphs

[pavel-esir]

• When only until n last characters are printed then they are cut in the middle and we get invalid python utf8 byte sequence and they are corrupted to �.
• Print until n last tokens! This fixed this issue.

visual_language_chat.py ./tiny-random-minicpmv-2_6 ./images <<< $'Describe the images?'
��������������������룅 encouraging룅 encouraging룅 encouraging룅 encouraging룅 encouraging룅 encouraging룅 encouraging

After the fix

룅튜룅튜룅튜룅튜룅튜룅 encouraging룅 encouraging룅 encouraging룅 encouraging룅 encouraging룅 encouraging

CVS-159227

[ilya-lavrenov]

is it possible to add tests for streamer?
e.g. w/o model put some tokens and see how they are processed via callback (i.e. we don't receive broken symbols)

[pavel-esir]

is it possible to add tests for streamer? e.g. w/o model put some tokens and see how they are processed via callback (i.e. we don't receive broken symbols)

Unfortunately, it was quite difficult to add calling python from c++ and be sure that environment is correct with necessary packages to convert with optimum-cli to openivno_detokenizer.xml.

I upgraded existing tests for streamer and ensure that streamed results are the same as from generate.

[ilya-lavrenov]

I think we need to move this to

openvino.genai/tests/python_tests/common.py

Lines 328 to 359 in b62145b

	def run_llm_pipeline(
	models_path : Path,
	prompts: List[str],
	generation_config : GenerationConfig,
	use_cb : bool = False
	) -> List[GenerationResult]:
	properties = get_default_properties()
	if use_cb:
	properties['scheduler_config'] = SchedulerConfig()
	ov_pipe = LLMPipeline(models_path, device='CPU', **properties)
	generate_outputs : DecodedResults = ov_pipe.generate(inputs=prompts, generation_config=generation_config)
	index = 0
	generation_results = []
	for _ in prompts:
	generation_result = GenerationResult()
	generation_result.m_generation_ids = generate_outputs.texts[index : index + generation_config.num_return_sequences]
	# sequences_scores are available only for beam search case
	if generation_config.is_beam_search():
	generation_result.m_scores = generate_outputs.scores[index : index + generation_config.num_return_sequences]
	generation_results.append(generation_result)
	index += generation_config.num_return_sequences
	del ov_pipe
	shutil.rmtree(models_path)
	return generation_results

because it covers significantly more cases (but limit it to cases when we have a single batch and num_return_sequences is 1)

[pavel-esir]

I agree. But it's present only in master. This PR is for release branch.
I will do that when cherry-pick to master.

[ilya-lavrenov]

I think we don't need to duplicate it in VLM pipeline, because internally both VLM and LLM use the same TextStreamer

[pavel-esir]

This VLM model is a good example where this streamer was broken in the first place but since we were not comparing here we didn't catch it from the beginning. This assert comparisons costs us almost nothing, but can prevent from mistakes.