[Trt-llm] always perform verbose dump of defaults for forward compatibility #3790
Annotations
2 errors
test_config.test_to_dict_trtllm[False-True]:
.venv/lib/python3.9/site-packages/libcst/_parser/tests/test_config.py#L471
AssertionError: assert ({'build_comma...ata': {}, ...} == {'build_comma...ata': {}, ...}
Omitting 10 identical items, use -vv to show
Differing items:
{'trt_llm': {'build': {'base_model': 'llama', 'checkpoint_repository': {'repo': 'meta/llama4-500B', 'revision': None, ...ict', 'enable_chunked_context': True, 'kv_cache_free_gpu_mem_fraction': 0.9, 'kv_cache_host_memory_bytes': None, ...}}} != {'trt_llm': {'build': {'base_model': 'llama', 'checkpoint_repository': {'repo': 'meta/llama4-500B', 'source': 'HF'}, 'gather_all_token_logits': False, 'max_batch_size': 512, ...}, 'runtime': {}}}
Full diff:
{
'build_commands': [],
'environment_variables': {},
'external_package_dirs': [],
'model_metadata': {},
'model_name': None,
'python_version': 'py39',
'requirements': [],
'resources': {'accelerator': 'L4',
'cpu': '1',
'memory': '24Gi',
'use_gpu': True},
'secrets': {},
'system_packages': [],
'trt_llm': {'build': {'base_model': 'llama',
'checkpoint_repository': {'repo': 'meta/llama4-500B',
+ 'revision': None,
'source': 'HF'},
'gather_all_token_logits': False,
'max_batch_size': 512,
+ 'max_beam_width': 1,
+ 'max_num_tokens': 8192,
+ 'max_prompt_embedding_table_size': 0,
- 'max_seq_len': 2048},
? -
+ 'max_seq_len': 2048,
- 'runtime': {}},
+ 'num_builder_gpus': None,
+ 'pipeline_parallel_count': 1,
+ 'plugin_configuration': {'gemm_plugin': 'auto',
+ 'paged_kv_cache': True,
+ 'use_fp8_context_fmha': False,
+ 'use_paged_context_fmha': True},
+ 'quantization_type': 'no_quant',
+ 'speculator': None,
+ 'strongly_typed': False,
+ 'tensor_parallel_count': 1},
+ 'runtime': {'batch_scheduler_policy': 'guaranteed_no_evict',
+ 'enable_chunked_context': True,
+ 'kv_cache_free_gpu_mem_fraction': 0.9,
+ 'kv_cache_host_memory_bytes': None,
+ 'request_default_max_tokens': None,
+ 'total_token_limit': 500000}},
}) == True
|
test
Process completed with exit code 1.
|