Panic with 'Unable to get layout information?'

[virtualritz]

I'm trying to create a wrapper around OpenVDB C++ library. OpenVDB is a volume data file format used in blockbuster visual effects.

There is a repo with instructions to replicate on macOS. The panic happens pretty late during binding generation.

Bindgen Invocation

    let openvdb_bindings = bindgen::Builder::default()
        .header("wrapper.hpp")
        .clang_arg("-std=c++11")
        .clang_arg(format!("-I/{}", open_vdb.join("include").display()))
        .clang_arg("-I/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/System/Library/Frameworks/CoreFoundation.framework/Versions/A/Headers/")
        .clang_arg("-I/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/System/Library/Frameworks/CoreServices.framework/Versions/A/Headers/")
        .clang_arg("-I/Library/Developer/CommandLineTools/usr/include/c++/v1/")
        .clang_arg("-I/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/")
        .clang_arg("-F/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/System/Library/Frameworks/")
        .generate()
        .expect("Unable to generate OpenVDB bindings.");

Actual Results

thread 'main' panicked at 'Unable to get layout information?', /Users/moritz/.cargo/registry/src/github.com-1ecc6299db9ec823/bindgen-0.53.2/src/codegen/mod.rs:1682:26
stack backtrace:
   0: <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt
   1: core::fmt::write
   2: std::io::Write::write_fmt
   3: std::panicking::default_hook::{{closure}}
   4: std::panicking::default_hook
   5: std::panicking::rust_panic_with_hook
   6: rust_begin_unwind
   7: core::panicking::panic_fmt
   8: core::option::expect_failed
   9: core::option::Option<T>::expect
  10: <bindgen::ir::comp::CompInfo as bindgen::codegen::CodeGenerator>::codegen
  11: <bindgen::ir::ty::Type as bindgen::codegen::CodeGenerator>::codegen
  12: <bindgen::ir::item::Item as bindgen::codegen::CodeGenerator>::codegen
  13: <bindgen::ir::comp::CompInfo as bindgen::codegen::CodeGenerator>::codegen
  14: <bindgen::ir::ty::Type as bindgen::codegen::CodeGenerator>::codegen
  15: <bindgen::ir::item::Item as bindgen::codegen::CodeGenerator>::codegen
  16: <bindgen::ir::module::Module as bindgen::codegen::CodeGenerator>::codegen::{{closure}}
  17: <bindgen::ir::module::Module as bindgen::codegen::CodeGenerator>::codegen
  18: <bindgen::ir::item::Item as bindgen::codegen::CodeGenerator>::codegen
  19: <bindgen::ir::module::Module as bindgen::codegen::CodeGenerator>::codegen::{{closure}}
  20: <bindgen::ir::module::Module as bindgen::codegen::CodeGenerator>::codegen
  21: <bindgen::ir::item::Item as bindgen::codegen::CodeGenerator>::codegen
  22: <bindgen::ir::module::Module as bindgen::codegen::CodeGenerator>::codegen::{{closure}}
  23: <bindgen::ir::module::Module as bindgen::codegen::CodeGenerator>::codegen
  24: <bindgen::ir::item::Item as bindgen::codegen::CodeGenerator>::codegen
  25: bindgen::codegen::codegen::{{closure}}
  26: bindgen::ir::context::BindgenContext::gen
  27: bindgen::codegen::codegen
  28: bindgen::Bindings::generate
  29: bindgen::Builder::generate
  30: build_script_build::main
  31: std::rt::lang_start::{{closure}}
  32: std::rt::lang_start_internal
  33: std::rt::lang_start
  34: main

[emilio]

I'm on Linux unfortunately :(

The panic is here:

rust-bindgen/src/codegen/mod.rs

Line 1682 in f568d09

let layout = layout.expect("Unable to get layout information?");

[emilio]

(Sorry, pressed enter too early)

Would be good to try to get a standalone test-case, there are instructions for that in the CONTRIBUTING file. Alternatively printing some information about the union in there may help figure out which type is the one causing havoc.

Thanks for reporting this!

[Neo-Zhixing]

I was able to reproduce this issue on my mac. The same issue seems to exist for Linux as well.

I have produced a build script that is supposed to work on both mac and linux. https://github.com/Neo-Zhixing/openvdb-sys

[Neo-Zhixing]

The problematic item, when printed, looks like this:

Item {
  id: ItemId(67559),
  local_id: LazyCell { inner: UnsafeCell },
  next_child_local_id: Cell { value: 1 },
  canonical_name: LazyCell { inner: UnsafeCell },
  path_for_whitelisting: LazyCell { inner: UnsafeCell },
  comment: None,
  annotations: Annotations {
    opaque: false,
    hide: false,
    use_instead_of: None,
    disallow_copy: false,
    private_fields: None,
    accessor_kind: None,
    constify_enum_variant: false, derives: [] },
    parent_id: ItemId(67548),
    kind: Type(Type {
      name: Some("ptr_converter"),
      layout: None,
      kind: Comp(CompInfo {
        kind: Union,
        fields: AfterComputingBitfieldUnits { fields: [], has_bitfield_units: false },
        template_params: [TypeId(ItemId(67560))],
        methods: [],
        constructors: [],
        destructor: None,
       base_members: [],
       inner_types: [],
       inner_vars: [],
       has_own_virtual_method: false,
       has_destructor: false,
       has_nonempty_base: false,
       has_non_type_template_params: false,
       packed_attr: false,
       found_unknown_attr: false,
       is_forward_declaration: false,
    }),
    is_const: false
  })
}

[Neo-Zhixing]

I was able to produce a test case that breaks bindgen.

struct atomic_impl {
    template<typename value_t>
    union ptr_converter;
};

Errors:

neo@Mac-Pro openvdb-sys % RUST_BACKTRACE=1 bindgen test.cpp
thread 'main' panicked at 'Unable to get layout information?', /Users/neo/.cargo/registry/src/github.com-1ecc6299db9ec823/bindgen-0.54.1/src/codegen/mod.rs:1778:26
stack backtrace:
   0: <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt
   1: core::fmt::write
   2: std::io::Write::write_fmt
   3: std::panicking::default_hook::{{closure}}
   4: std::panicking::default_hook
   5: std::panicking::rust_panic_with_hook
   6: rust_begin_unwind
   7: core::panicking::panic_fmt
   8: core::option::expect_failed
   9: <bindgen::ir::comp::CompInfo as bindgen::codegen::CodeGenerator>::codegen
  10: <bindgen::ir::ty::Type as bindgen::codegen::CodeGenerator>::codegen
  11: <bindgen::ir::item::Item as bindgen::codegen::CodeGenerator>::codegen
  12: <bindgen::ir::comp::CompInfo as bindgen::codegen::CodeGenerator>::codegen
  13: <bindgen::ir::ty::Type as bindgen::codegen::CodeGenerator>::codegen
  14: <bindgen::ir::item::Item as bindgen::codegen::CodeGenerator>::codegen
  15: <bindgen::ir::module::Module as bindgen::codegen::CodeGenerator>::codegen
  16: <bindgen::ir::item::Item as bindgen::codegen::CodeGenerator>::codegen
  17: bindgen::ir::context::BindgenContext::gen
  18: bindgen::Builder::generate
  19: std::panicking::try
  20: bindgen::main
  21: std::rt::lang_start::{{closure}}
  22: std::rt::lang_start_internal
  23: main

The breaking item came from <tbb/atomic.h>

The problem boils down to is_forward_declaration.

struct atomic_impl2 {
    template<typename value_t>
    union ptr_converter;
};

For this one, item.is_forward_declaration is false.

@virtualritz Since this was brought in from tbb, a good workaround would be disabling whitelist_recursively and whitelist all types manually.

[emilio]

Thanks for the reduced test-case, that's awesome! Yeah, that should be considered a forward declaration but it's not, that is very odd.

[luke-titley]

Hey. I'm also getting this issue (wrapping another lib that depends on tbb). As a work around if I'm not bothered about that type.
Can I change the expect to be an 'if let' ? on my local bindgen checkout ? Would that cause a major issue?

Edit: Wait. I missed your work around above. Sorry will try that.

[adi-g15]

@emilio Here's an even smaller test case:

variant.hpp

#include <variant>

main.rs

    builder().header("variant.hpp")
        .clang_arg("--std=c++17")
        .generate();

This one truly surprised me, while i was trying to find what caused it... 😅/😢

It causes this panic:

thread 'main' panicked at 'Unable to get layout information?', /home/ag15035/.cargo/registry/src/github.com-1ecc6299db9ec823/bindgen-0.59.1/src/codegen/mod.rs:1903:33
note: run with RUST_BACKTRACE=1 environment variable to display a backtrace

The backtrace doesn't have more than what @Neo-Zhixing posted.

[safasofuoglu]

I have tracked down another case from Intel TBB atomic.h causing this panic:

template<typename value_t>
union ptr_converter;            //Primary template declared, but never defined.

template<typename value_t>
union ptr_converter<value_t *> {
    ptr_converter(){}
    ptr_converter(value_t* a_value) : value(a_value) {}
    value_t* value;
    uintptr_t bits;
};

Empty union, not forward declaration.

[wmmc88]

It would be useful if bindgen emitted error information about which file it failed to process.