fx error: symbolically traced variables cannot be used as inputs to control flow

[alexander-sony]

Issue Type

Bug

Source

pip (model-compression-toolkit)

MCT Version

2.2

OS Platform and Distribution

Ubuntu 20.04

Python version

3.8.10

Describe the issue

Trying to get https://github.com/HRNet/HRNet-Facial-Landmark-Detection though mct. ptq quantization step throws:

...
  File "/home/SONY/s1000328194/git/HRNet-Facial-Landmark-Detection/.venv/lib/python3.8/site-packages/model_compression_toolkit/logger.py", line 117, in critical
    raise Exception(msg)
Exception: Error parsing model with torch.fx
fx error: symbolically traced variables cannot be used as inputs to control flow

Understand that its probably a problem with conditionals in the forward pass - any ideas for how to proceed?

Expected behaviour

No response

Code to reproduce the issue

see above

Log output

see above

[Idan-BenAmi]

Hi @alexander-sony,
Usually those conditionals in the forward pass are static per block and can be modified to be torch.fx compatible. Can you send the full log, I'll try to be more specific.
Thanks
Idan

[alexander-sony]

Start Model Reading...
Error parsing model with torch.fx
fx error: symbolically traced variables cannot be used as inputs to control flow
Traceback (most recent call last):
  File "/home/SONY/s1000328194/git/HRNet-Facial-Landmark-Detection/.venv/lib/python3.8/site-packages/model_compression_toolkit/core/pytorch/reader/reader.py", line 90, in fx_graph_module_generation
    symbolic_traced = symbolic_trace(pytorch_model)
  File "/home/SONY/s1000328194/git/HRNet-Facial-Landmark-Detection/.venv/lib/python3.8/site-packages/torch/fx/_symbolic_trace.py", line 1070, in symbolic_trace
    graph = tracer.trace(root, concrete_args)
  File "/home/SONY/s1000328194/git/HRNet-Facial-Landmark-Detection/.venv/lib/python3.8/site-packages/torch/fx/_symbolic_trace.py", line 739, in trace
    (self.create_arg(fn(*args)),),
  File "/home/SONY/s1000328194/git/HRNet-Facial-Landmark-Detection/.venv/lib/python3.8/site-packages/torch/nn/parallel/data_parallel.py", line 161, in forward
    inputs, kwargs = self.scatter(inputs, kwargs, self.device_ids)
  File "/home/SONY/s1000328194/git/HRNet-Facial-Landmark-Detection/.venv/lib/python3.8/site-packages/torch/nn/parallel/data_parallel.py", line 178, in scatter
    return scatter_kwargs(inputs, kwargs, device_ids, dim=self.dim)
  File "/home/SONY/s1000328194/git/HRNet-Facial-Landmark-Detection/.venv/lib/python3.8/site-packages/torch/nn/parallel/scatter_gather.py", line 52, in scatter_kwargs
    inputs = scatter(inputs, target_gpus, dim) if inputs else []
  File "/home/SONY/s1000328194/git/HRNet-Facial-Landmark-Detection/.venv/lib/python3.8/site-packages/torch/fx/proxy.py", line 298, in __bool__
    return self.tracer.to_bool(self)
  File "/home/SONY/s1000328194/git/HRNet-Facial-Landmark-Detection/.venv/lib/python3.8/site-packages/torch/fx/proxy.py", line 174, in to_bool
    raise TraceError('symbolically traced variables cannot be used as inputs to control flow')
torch.fx.proxy.TraceError: symbolically traced variables cannot be used as inputs to control flow

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "quantize.py", line 138, in <module>
    main()
  File "quantize.py", line 132, in main
    quant_model = quantization(model, N_ITER, SHAPE)
  File "quantize.py", line 96, in quantization
    resource_utilization_data = mct.core.pytorch_resource_utilization_data(model,
  File "/home/SONY/s1000328194/git/HRNet-Facial-Landmark-Detection/.venv/lib/python3.8/site-packages/model_compression_toolkit/core/pytorch/resource_utilization_data_facade.py", line 83, in pytorch_resource_utilization_data
    return compute_resource_utilization_data(in_model,
  File "/home/SONY/s1000328194/git/HRNet-Facial-Landmark-Detection/.venv/lib/python3.8/site-packages/model_compression_toolkit/core/common/mixed_precision/resource_utilization_tools/resource_utilization_data.py", line 64, in compute_resource_utilization_data
    transformed_graph = graph_preparation_runner(in_model,
  File "/home/SONY/s1000328194/git/HRNet-Facial-Landmark-Detection/.venv/lib/python3.8/site-packages/model_compression_toolkit/core/graph_prep_runner.py", line 72, in graph_preparation_runner
    graph = read_model_to_graph(in_model,
  File "/home/SONY/s1000328194/git/HRNet-Facial-Landmark-Detection/.venv/lib/python3.8/site-packages/model_compression_toolkit/core/graph_prep_runner.py", line 207, in read_model_to_graph
    graph = fw_impl.model_reader(in_model,
  File "/home/SONY/s1000328194/git/HRNet-Facial-Landmark-Detection/.venv/lib/python3.8/site-packages/model_compression_toolkit/core/pytorch/pytorch_implementation.py", line 149, in model_reader
    return model_reader(_module, representative_data_gen, self.to_numpy, self.to_tensor)
  File "/home/SONY/s1000328194/git/HRNet-Facial-Landmark-Detection/.venv/lib/python3.8/site-packages/model_compression_toolkit/core/pytorch/reader/reader.py", line 153, in model_reader
    fx_model = fx_graph_module_generation(model, representative_data_gen, to_tensor)
  File "/home/SONY/s1000328194/git/HRNet-Facial-Landmark-Detection/.venv/lib/python3.8/site-packages/model_compression_toolkit/core/pytorch/reader/reader.py", line 92, in fx_graph_module_generation
    Logger.critical(f'Error parsing model with torch.fx\n'
  File "/home/SONY/s1000328194/git/HRNet-Facial-Landmark-Detection/.venv/lib/python3.8/site-packages/model_compression_toolkit/logger.py", line 117, in critical
    raise Exception(msg)
Exception: Error parsing model with torch.fx
fx error: symbolically traced variables cannot be used as inputs to control flow
make: *** [Makefile:2: quant] Error 1

[Idan-BenAmi]

Hi @alexander-sony ,
Thanks for sharing the log,
I've tried to find more specific info in the torch.fx error log, but couldn't. To proceed, I would suggest you to mask parts of the model's forward pass, and try to locate the incompatible torch.fx part of the model. Reviewing the model, I do see several conditionals in the forward pass so I would try to start there (https://github.com/HRNet/HRNet-Facial-Landmark-Detection/blob/master/lib/models/hrnet.py#L404).

Since torch.fx compatibility is out of the scope of MCT I'm closing the issue, let me know if you need further assistance.
Idan