How to build a submodule clone of a 3rd party repo?

[embcla]

Hello

I'm using meson for my build system, and everything works with my sources.

The issue I have is because I target an ST chip, so I want to have a submodule pointing at their device support repo
https://github.com/STMicroelectronics/STM32CubeG0

so I have
source

\......meson.build
\......src
        \......meson.build
\......subprojects
        \......library
                   \......meson.build
        \......STM32CubeG0

Ideally I'd keep the STM32 submodule as a pure clone of the STM code.
So I started adding hardcoded paths in the main project meson file, and it complains

WARNING: include_directories sandbox violation!
The project is trying to access the directory subprojects/STM32CubeG0/Drivers/STM32G0xx_HAL_Driver/Inc which belongs to a different
subproject. This is a problem as it hardcodes the relative paths of these two projeccts.
This makes it impossible to compile the project in any other directory layout and also
prevents the subproject from changing its own directory layout.

Instead of poking directly at the internals the subproject should be executed and
it should set a variable that the caller can then use. Something like:

# In subproject
some_dep = declare_dependency(include_directories: include_directories('include'))

# In subproject wrap file
[provide]
some = some_dep

# In parent project
some_dep = dependency('some')
executable(..., dependencies: [some_dep])

This warning will become a hard error in a future Meson release.


meson.build:79:0: ERROR: Sandbox violation: Tried to grab directory Templates from a nested subproject.

What's the suggested solution for doing this kind of thing?

[eli-schwartz]

https://mesonbuild.com/Wrap-dependency-system-manual.html

[embcla]

Thank you Eli, I tried this way

here is my wrap file STM32CubeG0.wrap

[wrap-file]
directory = STM32CubeG0
url = https://github.com/STMicroelectronics/STM32CubeG0.git
revision = 1.5.0

[provide]
dependency_names = STM32CubeG0-1.5.0

and in my main meson file I have
stm_bsp_proj = subproject('STM32CubeG0')

but meson still complains
meson.build:61:0: ERROR: Subproject exists but has no meson.build file

[mscofield0]

The situation isn't as simple as that.

Look at the wraps in the official WrapDB repository here.

e.g. fmt

fmt.wrap

[wrap-file]
directory = fmt-8.0.1
source_url = https://github.com/fmtlib/fmt/archive/8.0.1.tar.gz
source_filename = fmt-8.0.1.tar.gz
source_hash = b06ca3130158c625848f3fb7418f235155a4d389b2abc3a6245fb01cb0eb1e01
patch_directory = fmt

[provide]
fmt = fmt_dep # <---- This will allow users to include `fmt` with `dependency('fmt')`
                       # Although, that only works if the library is already on the system.
                       # The foolproof way is with: `dependency('fmt', fallback: ['fmt', 'fmt_dep'])`,
                       # as noted [here](https://mesonbuild.com/Dependencies.html#building-dependencies-as-subprojects)

The above (in particular, the fmt = fmt_dep) is only possible because of the following packagefile.

packagefiles/fmt/meson.build

project('fmt', 'cpp',
    version : '8.0.1',
    license : 'BSD',
    default_options : ['cpp_std=c++14']
)

fmt_private_cpp_args = [ ]
fmt_interface_cpp_args = [ ]
libtype = get_option('default_library')
if libtype == 'shared'
    fmt_private_cpp_args += [ '-DFMT_EXPORT' ]
    fmt_interface_cpp_args += [ '-DFMT_SHARED' ]
endif

fmt_inc = include_directories('include')
fmt_lib = library('fmt',
    sources : [
        'src/format.cc',
        'src/os.cc'
    ],
    cpp_args : fmt_private_cpp_args,
    include_directories : fmt_inc
)

# We create the fmt dependency here
fmt_dep = declare_dependency(
    include_directories : fmt_inc,
    compile_args : fmt_interface_cpp_args,
    link_with : fmt_lib
)

fmt_header_only_dep = declare_dependency(
    include_directories : fmt_inc,
    compile_args : '-DFMT_HEADER_ONLY'
)

A packagefile is just a custom meson.build that sets the dependency up for the wrap to provide it.

To be honest, I have no idea how to do it locally, without WrapDB, maybe someone else can touch on that subject.
If you want to add a new package to WrapDB, you can submit an issue here or submit a PR to add it.

[eli-schwartz]

You can add subprojects/packagefiles/fmt/meson.build to any repository and use it as an alternative to the WrapDB. The WrapDB repository is a working, example implementation of using wraps without using official WrapDB packages (which are generated after the fact based on that repo).

The key is this line:

patch_directory = fmt

[mscofield0]

Ah, that makes sense.

[embcla]

I'm not sure I totally understand what's going on here.

I browsed the various WrapDB existing configurations and from what I can see, there are two cases:
A) header-only library
B) separately built library

Unfortunately my case falls in a third area: non-built sources.
You see, the way STM Cube packages work is that each one supports a family of MCUs, so you take the one for your family and add that to your project, then use your main project build system that's already configured for your specific chip, to traverse these additional sources to provide low level peripheral functionality.
The firmware support package is non the wiser about your chosen MCU, so I can't specify build options and flags.

So ultimately I want my project to have a dependency on a firmware support package that declares both sources and headers, but doesn't build them.
Actually it should inherit the cross build configuration defined elsewhere as other subprojects do.

So what I did so far is this

root
  \......meson.build
  \......src
          \......meson.build
  \......subprojects
          \......stm-cube-support.wrap
          \......library
                     \......meson.build
          \......packagefiles
                     \......STM32CubeG0
                                \......meson.build

so in my main meson.build I now have
stm_bsp_proj = subproject('stm-cube-support')

in the stm-cube-support.wrap file I have

[wrap-git]
url = https://github.com/STMicroelectronics/STM32CubeG0.git
revision = v1.5.0
patch_directory = STM32CubeG0

[provide]
stm-cube-support = stm-cube-support_dep

and in the packagefile meson.build I have

project('stm-cube-support', 'c',
  version: '1.5.0'
)



stm_inc = include_directories: include_directories('Drivers/CMSIS/Device7ST/STM32G0xx/Include',
                                                'Drivers/CMSIS/Include',
                                                'Drivers/CMSIS/Device',
                                                'Drivers/CMSIS/Device/ST/STM32G0xx/Include',
                                                'Drivers/STM32G0xx_HAL_Driver/Inc',
                                                'Drivers/STM32G0xx_HAL_Driver/Inc/Legacy')


stm_sources = files('Drivers/STM32G0xx_HAL_Driver/Srcstm32g0xx_hal.c',
                'Drivers/STM32G0xx_HAL_Driver/stm32g0xx_hal_cortex.c',
                'Drivers/STM32G0xx_HAL_Driver/stm32g0xx_hal_dma.c',
                'Drivers/STM32G0xx_HAL_Driver/stm32g0xx_hal_dma_ex.c',
                'Drivers/STM32G0xx_HAL_Driver/stm32g0xx_hal_exti.c',
                'Drivers/STM32G0xx_HAL_Driver/stm32g0xx_hal_flash.c',
                'Drivers/STM32G0xx_HAL_Driver/stm32g0xx_hal_flash_ex.c',
                'Drivers/STM32G0xx_HAL_Driver/stm32g0xx_hal_gpio.c',
                'Drivers/STM32G0xx_HAL_Driver/stm32g0xx_hal_i2c.c',
                'Drivers/STM32G0xx_HAL_Driver/stm32g0xx_hal_i2c_ex.c',
                'Drivers/STM32G0xx_HAL_Driver/stm32g0xx_hal_iwdg.c',
                'Drivers/STM32G0xx_HAL_Driver/stm32g0xx_hal_pwr.c',
                'Drivers/STM32G0xx_HAL_Driver/stm32g0xx_hal_pwr_ex.c',
                'Drivers/STM32G0xx_HAL_Driver/stm32g0xx_hal_rcc.c',
                'Drivers/STM32G0xx_HAL_Driver/stm32g0xx_hal_rcc_ex.c',
                'Drivers/STM32G0xx_HAL_Driver/stm32g0xx_hal_tim.c',
                'Drivers/STM32G0xx_HAL_Driver/stm32g0xx_hal_tim_ex.c',
                'Drivers/STM32G0xx_HAL_Driver/stm32g0xx_hal_uart.c',
                'Drivers/STM32G0xx_HAL_Driver/stm32g0xx_hal_uart_ex.c',
                'Drivers/STM32G0xx_HAL_Driver/stm32g0xx_ll_dma.c',
                'Drivers/STM32G0xx_HAL_Driver/stm32g0xx_ll_rcc.c')

stm_lib = library('STM32CubeG0',
    sources : stm_sources,
    include_directories : stm_inc
)

stm-cube-support_dep = declare_dependency(
    include_directories : stm_inc,
    link_with : stm_lib
)

which I don't like because it's attempting to build it.
Regardless, meson gives me an error:
meson.build:61:0: ERROR: Subproject exists but has no meson.build file

As I understand it, my packagefile meson.build is not being picked up. How is that one linked with the wrap file?
I thought it was the patch_directory line, but maybe it's something else?

[eli-schwartz]

• what's the difference, to you, between the subproject building the files, and the main project building the files? Either way it gets built and linked into your own code.

  so I can't specify build options and flags.

  Is this the sort of situation where you have global CFLAGS? Do you need -DDEFINEs? Can options be specified through meson_options.txt? Some of these questions matter more if you want the firmware support repo to support meson out of the box whenever anyone wants to add it to a meson project.

• If you already had the subproject cloned, then meson won't try to init it using the packagefiles you specified. You can use meson subprojects packagefiles --apply to sync them, or delete the clone and meson will re-clone it on the next run -- and in the process of re-cloning, it will apply the packagefiles.

[embcla]

Thanks Eli-schwartz
I think I fixed the configuration so it passes the configuration generation now, but it's attempting to compile it.

So the point is that this dependency is not meant to be built at all. There is no main function, some objects don't have their configuration because that lives somewhere else, the actual platform is not defined.
It's just a bunch of sources you need when targeting STM32 microcontrollers, not a standalone project (not even a library).

The reason for going for a subproject is two fold:

• STM releases updates on their own schedule and we want to be able to easily update to the latest
• we support multiple boards that can have different configurations but use the same support package

So at a higher level we have

main project
  \......subprojects
            \......BSP
                     \......board1
                                \......board1.cross
                     \......board2
                                \......board2.cross
                     \......subprojects
                                \......stm_support_package

in this configuration, what I've been doing so far in this thread, is trying to get the stm_support_package to work inside the BSP as additional sources

[eli-schwartz]

In that case, you can add files() objects as sources to stm_dep = declare_dependency(sources: ...) and they will be forwarded to the target that links to the stm_dep. It will then get rebuilt once per each "board" target ( I assume using different c_args?) the same as if you specified it as a direct source file for the "board" target.

[eli-schwartz]

I'm assuming here that you don't want an object library of compiled .o files which are compiled once and linked into multiple boards (the discussion about main functions is irrelevant to object libraries).

[embcla]

The order of inclusion is my main issue here: according to STM themselves, this is included into the project from other files which are then themselves included in the application.
Application <- configuration <- support package
Arrows represent inclusions.

Defines like the platform (#DEFINE STM32G071RB for example which sets register overlays and other things) are set at a higher level and pushed down, so I really don't see how I could properly compile this autonomously when all relevant properties are set elsewhere. The only way I see is by setting all relevant properties again for this project, but this would be a copy of some other properties available elsewhere.

I'll try using the sources in the dependency declaration as you suggested and report back

[embcla]

So here is my update: I was able to compile using the dependency declaration with sources.

This is a great achievement, thank you both @eli-schwartz and @mscofield0

I want to contribute this wrap, it seems there is no pull request process thought, right?

[embcla]

-Wunused-parameter is supposed to add those warnings, not suppress them. Did you mean to use -Wno-*?

You're right but it doesn't apply cause we have -Wall in our cross configuration so nothing I can add can override that.
That's why I removed that part of my comment :)

[eli-schwartz]

Ah, I must have seen your comment before it got edited... :)

[embcla]

You're just so fast! :)

[eli-schwartz]

By the way according to the GCC manual:

Some options, such as -Wall and -Wextra, turn on other options, such as -Wunused, which may turn on further options, such as -Wunused-value. The combined effect of positive and negative forms is that more specific options have priority over less specific ones, independently of their position in the command-line. For options of the same specificity, the last one takes effect. Options enabled or disabled via pragmas (see Diagnostic Pragmas) take effect as if they appeared at the end of the command-line.

So you should be able to use -Wall -Wno-unused-parameter to disable that one specific warning type...

[embcla]

Thanks!

[embcla]

Hi @eli-schwartz , I noticed one thing.
When I build the main project, the wrap stuff is bubbled up to the top level

main project
  \......subprojects
            \......stm_support_package <-------
            \......stm_support_package.wrap <------
            \......BSP
                     \......board1
                                \......board1.cross
                     \......board2
                                \......board2.cross
                     \......subprojects
                                \......stm_support_package.wrap

Is there a way to avoid this?
It clutters the source and requires adding ignore entries to git