undefined symbol: _ZNK6google8protobuf7Message13SpaceUsedLongEv

[lcskrishna]

I am trying to use this tool for converting a caffe2 model to onnx model using the example given #3

I am trying to convert resnet-101 model.

Below is my error log:

Traceback (most recent call last):
  File "conversion.py", line 1, in <module>
    import onnx_caffe2.frontend as c2_onnx
  File "/home/chaitanya/.local/lib/python2.7/site-packages/onnx_caffe2/frontend.py", line 8, in <module>
    from onnx import onnx_pb2, checker
  File "/home/chaitanya/.local/lib/python2.7/site-packages/onnx/__init__.py", line 7, in <module>
    from . import checker, helper
  File "/home/chaitanya/.local/lib/python2.7/site-packages/onnx/checker.py", line 14, in <module>
    from onnx import defs
  File "/home/chaitanya/.local/lib/python2.7/site-packages/onnx/defs/__init__.py", line 6, in <module>
    import onnx.onnx_cpp2py_export as C
ImportError: /home/chaitanya/.local/lib/python2.7/site-packages/onnx/onnx_cpp2py_export.so: undefined symbol: _ZNK6google8protobuf7Message13SpaceUsedLongEv

can someone help me out with the above issue.

[ezyang]

Hi @lcskrishna. Did you install using the binaries (conda install -c ezyang onnx) or from source?

[lcskrishna]

I have installed using source (pip install onnx)

[ezyang]

Please pip uninstall onnx and then try the binary install.

[bddppq]

@lcskrishna Did you do pip inside a conda virtual environment?

[lcskrishna]

@bddppq No, I don't have conda virtual environment setup, I'm trying directly using pip install.
Also, I have installed protobuf and protoc from github source and I have installed numpy using pip.

[bddppq]

@lcskrishna I see. Which OS are you using? Could you do
ldd /home/chaitanya/.local/lib/python2.7/site-packages/onnx/onnx_cpp2py_export.so and readelf -d /home/chaitanya/.local/lib/python2.7/site-packages/onnx/onnx_cpp2py_export.so and paste the output here? Also echo $LD_LIBRARY_PATH.

[lcskrishna]

@bddppq

I am using Ubuntu 16.04

Here are the outputs you asked for:

%ldd /home/chaitanya/.local/lib/python2.7/site-packages/onnx/onnx_cpp2py_export.so

 linux-vdso.so.1 =>  (0x00007ffd2ad53000)
        libprotobuf.so.9 => /usr/lib/x86_64-linux-gnu/libprotobuf.so.9 (0x00007f0f3edc6000)
        libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f0f3ebb0000)
        libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f0f3e992000)
        libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f0f3e5c8000)
        libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007f0f3e3ae000)
        libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007f0f3e02b000)
        libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f0f3dd22000)
        /lib64/ld-linux-x86-64.so.2 (0x00005563d2b0c000)

%readelf -d /home/chaitanya/.local/lib/python2.7/site-packages/onnx/onnx_cpp2py_export.so

Dynamic section at offset 0x67bd0 contains 27 entries:
  Tag        Type                         Name/Value
 0x0000000000000001 (NEEDED)             Shared library: [libprotobuf.so.9]
 0x0000000000000001 (NEEDED)             Shared library: [libgcc_s.so.1]
 0x0000000000000001 (NEEDED)             Shared library: [libpthread.so.0]
 0x0000000000000001 (NEEDED)             Shared library: [libc.so.6]
 0x000000000000000c (INIT)               0x13a48
 0x000000000000000d (FINI)               0x4feb4
 0x0000000000000019 (INIT_ARRAY)         0x2671f0
 0x000000000000001b (INIT_ARRAYSZ)       96 (bytes)
 0x000000000000001a (FINI_ARRAY)         0x267250
 0x000000000000001c (FINI_ARRAYSZ)       8 (bytes)
 0x000000006ffffef5 (GNU_HASH)           0x1f0
 0x0000000000000005 (STRTAB)             0x5be0
 0x0000000000000006 (SYMTAB)             0x18d8
 0x000000000000000a (STRSZ)              41221 (bytes)
 0x000000000000000b (SYMENT)             24 (bytes)
 0x0000000000000003 (PLTGOT)             0x268000
 0x0000000000000002 (PLTRELSZ)           4896 (bytes)
 0x0000000000000014 (PLTREL)             RELA
 0x0000000000000017 (JMPREL)             0x12728
 0x0000000000000007 (RELA)               0x102e0
 0x0000000000000008 (RELASZ)             9288 (bytes)
 0x0000000000000009 (RELAENT)            24 (bytes)
 0x000000006ffffffe (VERNEED)            0x10280
 0x000000006fffffff (VERNEEDNUM)         2
 0x000000006ffffff0 (VERSYM)             0xfce6
 0x000000006ffffff9 (RELACOUNT)          186
 0x0000000000000000 (NULL)               0x0

% echo $LD_LIBRARY_PATH

/usr/local/lib

[bddppq]

@lcskrishna Hmm...everything looks normal to me. Could you also do nm -C /usr/lib/x86_64-linux-gnu/libprotobuf.so.9 | grep SpaceUsedLong?

[lcskrishna]

I am getting the following output :

nm: /usr/lib/x86_64-linux-gnu/libprotobuf.so.9: no symbols

[bddppq]

@lcskrishna Not sure whether it's your protobuf installation being broken. Adding "-D" flag to the nm command might help debugging. In the meanwhile, since you are using ubuntu, could you use "sudo apt-get install libprotobuf-dev protobuf-compiler" to install protobuf?

[bddppq]

@lcskrishna Have you been able to resolve the issue?

[lcskrishna]

I tried a fresh installation of caffe2, protobuf, onnx and onnx-caffe2
Also, I have used conda installation for onnx.
Now the above error doesn't show up, however I am getting the following error while running the conversion:

WARNING:root:This caffe2 python run does not have GPU support. Will run in CPU only mode.
WARNING:root:Debug message: No module named caffe2_pybind11_state_gpu
Traceback (most recent call last):
  File "../../caffe2-conv/conversion.py", line 9, in <module>
    c2_net.ParseFromString(f.read())
google.protobuf.message.DecodeError: Error parsing message

Here is my script:

import onnx_caffe2.frontend as c2_onnx
from caffe2.proto import caffe2_pb2
import os

c2_net = caffe2_pb2.NetDef()
model_path = '/home/chaitu/work/caffe2_models/model/'
c2_model_file = os.path.join(model_path, 'resnet101_init_net.pb')
with open(c2_model_file, 'rb') as f:
    c2_net.ParseFromString(f.read())
onnx_graph = c2_onnx.caffe2_net_to_onnx_graph(c2_net)

[bddppq]

@lcskrishna What's the size of your pb file? I suspect it's hitting the 64mb limit. Could you try 'export PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python' in your terminal and then run your code snippet again?

[lcskrishna]

@bddppq the size of the file is around 4mb and i forgot to export.
However, I have tried by using a simple CIFAR 10 network.

I took a caffemodel of CIFAR10 and using the conversion tool in caffe2 and translated into caffe2 model and tried to perform conversion as mentioned above, still i get the following error.
I'm not sure what's the issue is:

WARNING:root:This caffe2 python run does not have GPU support. Will run in CPU only mode.
WARNING:root:Debug message: No module named caffe2_pybind11_state_gpu
Unrecognized attribute: legacy_pad
Traceback (most recent call last):
  File "conversion.py", line 10, in <module>
    onnx_graph = c2_onnx.caffe2_net_to_onnx_graph(c2_net)
  File "/home/chaitu/.local/lib/python2.7/site-packages/onnx_caffe2/frontend.py", line 254, in caffe2_net_to_onnx_graph
    caffe2_op_to_node_def(op, name_map) for op in net_def.op)
  File "/home/chaitu/.local/lib/python2.7/site-packages/onnx_caffe2/frontend.py", line 254, in <genexpr>
    caffe2_op_to_node_def(op, name_map) for op in net_def.op)
  File "/home/chaitu/.local/lib/python2.7/site-packages/onnx_caffe2/frontend.py", line 205, in caffe2_op_to_node_def
    checker.check_node(node_def)
  File "/home/chaitu/.local/lib/python2.7/site-packages/onnx/checker.py", line 38, in check_node
    'NodeProto of type {} did not pass defs schema check.'.format(str(node.op_type)))
ValueError: NodeProto of type MaxPool did not pass defs schema check.

[jerryzh168]

@lcskrishna Please add option --remove_legacy_pad when you do the translation from caffe model to caffe2 model.