Selecting different header files based on configuration

[oscarbenjamin]

This comes from attempting to make a meson build configuration for the Flint C library (flintlib/flint#1910).

I need to select one of several different header files depending on the configuration. This seems like a common thing to do but I can't quite find the right way to do it. Right now I have it working but meson is giving me a deprecation warning about the copy argument to configure_file.

Currently I do this with e.g.:

if gmp_long_long_limb
  gmpcompat_h_in = 'gmpcompat-longlong.h.in'
else
  gmpcompat_h_in = 'gmpcompat.h.in'
endif

gmp_compat_h = configure_file(
    input: gmpcompat_h_in,
    output: 'gmpcompat.h',
    copy: true
)

headers_all += [gmp_compat_h]

install_headers(headers_all, subdir: 'flint')

This works but I get a warning:

$ meson setup build.meson
...
include/flint/meson.build:28: WARNING: Project targets '>= 1.1.0'
but uses feature deprecated since '0.64.0': copy arg in configure_file.
Use fs.copyfile instead

I tried changing to copyfile:

gmp_compat_h = fs.copyfile(
    gmpcompat_h_in,
    'gmpcompat.h',
)

headers_all += [gmp_compat_h]

install_headers(headers_all, subdir: 'flint')

This gives an error:

$ meson setup build.meson
...
meson.build:50:0: ERROR: install_headers argument 172 was of type
"CustomTarget" but should have been one of: "str", "File"

Instead of using install_headers I tried using install: true:

gmp_compat_h = fs.copyfile(
    gmpcompat_h_in,
    'gmpcompat.h',
    install: true,
    install_dir: 'flint',
)

This installs in prefix/flint/gmpcompat.h rather than prefix/include/flint/gmpcompat.h.

I feel like I must be missing something obvious. What is the typical way to accomplish this?

[oscarbenjamin]

ight now I have it working but meson is giving me a deprecation warning about the copy argument to configure_file.

Actually it is not working. I hadn't actually tested running install:

$ meson install -C build.meson
...
ERROR: Tried to install something that isn't a file:
'.../flint/build.meson/include/flint/gmpcompat.h'

[oscarbenjamin]

This installs in prefix/flint/gmpcompat.h rather than prefix/include/flint/gmpcompat.h

That one is easy to fix with get_option('includedir') so that I have it working I think now.

I feel like I am misunderstanding something about how this all works or what is the typical way to do this though.

Why is it that the output of configure_file can be passed to install_headers but the output of fs.copyfile cannot?

Is there a better analogue of configure_file for selecting between two or more files like select_file or something?

[eli-schwartz]

Configured files have the same output type as files(), so meson thinks it's a source file.

Perhaps it shouldn't, but it does, and therefore it needs to be supported at minimum for backwards compatibility.

BTW: those are .in files but they don't have any templates to replace?

[oscarbenjamin]

Configured files have the same output type as files(), so meson thinks it's a source file.

Perhaps it shouldn't,

Why shouldn't it?

I think I'm not quite grasping conceptually how meson's semantics work here. I thought that part of the purpose of having a "file" type as distinct from "string" was so that you could represent generated files as files as well and meson would understand the dependency relationships. In that sense it seems natural to me that configure_file returns a "file" but I would also have expected that fs.copyfile returns a "file" as well and that install_headers would know how to install any "file".

Is it actually intended that a "file" is only a file in the source tree? Then the purpose of the "file" type is just supposed to be that when you call files meson checks that the file exists and then stores its absolute path so that you can refer to the file from meson.build files in other directories without needing to give a relative path?

BTW: those are .in files but they don't have any templates to replace?

Yes, that's correct. The autotools build just makes a symbolic link like:

gmpcompat.h -> ../src/gmpcompat.h.in

[eli-schwartz]

I thought that part of the purpose of having a "file" type as distinct from "string" was so that you could represent generated files as files as well and meson would understand the dependency relationships.

If an on-disk file is created by running ninja -- such as fs.copyfile(), executable(), custom_target() -- then it needs to track dependency relationships in both directions.

If it exists in the source tree, or if it's created as a side effect of meson setup like configure_file(), then there is no dependency relationship to track, because no matter how many times you run ninja the file will always exist with the same contents. Well, there is one type of dependency relationship and that is that build rules may depend on it (but not recursively).

That's why the return type of configure_file() is the same as the return type of files() -- because both represent an item without an associated build rule. In contrast, the return type of fs.copfile() is the same as custom_target() or gnome.generate_gir(), because they all create ninja build rules.

Is it actually intended that a "file" is only a file in the source tree? Then the purpose of the "file" type is just supposed to be that when you call files meson checks that the file exists and then stores its absolute path so that you can refer to the file from meson.build files in other directories without needing to give a relative path?

"A file" is a generic term. But a mesonlib.File class instance is a file that can be referred to from any directory without needing to give a relative path. Meson uses them internally to represent the difference between paths to files , vs strings that may not have anything to do with files at all. It performs this conversion in the preamble of any meson DSL function that accepts "string or mesonlib.File, as names of input files".

Invoking the files() function allows you to perform that translation under your control, ahead of time, with the specific intention of supporting cross-directory file handling. It's just not the only reason a mesonlib.File might exist.

The real question here is a more practical one: meson doesn't support using custom_target() to produce an output file (as a ninja rule) on one line, and then on another line use install_data() or similar to install it on a second line. But it does allow that for configure_file(), which is a bit inconsistent and leads to the question, "should we allow it?" - it's something that has come up before and I'm not sure what to do about it. It is inconsistent and confusing, and violated the meson idea that all information about a build directory file should be defined in one place, but changing this is a backwards compatibility break...

[oscarbenjamin]

it's created as a side effect of meson setup like configure_file()

Okay I see the difference now. Would it make sense to have something like select_file that is analogous to configure_file but just for selecting between different files?

Or why is it that the copy argument to configure_file is deprecated and could there be a way to have configure_file do what I wanted?

I suppose I could just "configure" the file with an empty dict:

configure_file(
    input: gmp_compat_h_in,
    output: 'gmpcompat.h',
    configuration: {},
)

Conceptually what I am doing here is part of the same stage as processing other .h.in files that are configured. All of this takes place during ./configure in the autotools build. You can see what I've ended up with now here:
https://github.com/flintlib/flint/blob/3ca5c2e46cc2bde4f49c8226fefc47df2844747e/_meson_build/include/flint/meson.build#L1-L62

Everything works fine if I pass these arguments everywhere:

  install_dir: get_option('includedir') / 'flint',
  install: true,

It just seemed cleaner to collect a list of headers and then have a single call to install_headers like:

install_headers(headers_all, subdir: 'flint')

The real question here is a more practical one: meson doesn't support using custom_target() to produce an output file (as a ninja rule) on one line, and then on another line use install_data() or similar to install it on a second line. But it does allow that for configure_file(), which is a bit inconsistent and leads to the question, "should we allow it?"

I assume that your "should we allow it" refers to wanting to disallow doing this with configure_file. What about resolving the inconsistency in the other direction by allowing custom targets to be passed to install_headers (and other functions)?

It seemed natural to me that if I could pass install: true to fs.copyfile then I should alternatively be able to pass the copied "file" to install_headers.

Actually copying all of the headers to the build directory does seem to take a surprising amount of time during the build step and I don't really want to have to do that if possible. The reason I did it that way is just so that you could do gcc -Ibuild-dir/include to compile an ad-hoc test program. Ideally build-dir/include would just be a bunch of symlinks back to the source directory but maybe that complicates things for meson.

[eli-schwartz]

The reason I did it that way is just so that you could do gcc -Ibuild-dir/include to compile an ad-hoc test program. Ideally build-dir/include would just be a bunch of symlinks back to the source directory but maybe that complicates things for meson.

This would also make it more complicated to link an ad-hoc test program to the library. Another option would be to use gcc $(pkg-config --cflags --libs libgmp) mytest.c -o mytest.

Using meson devenv should make that pkg-config call work with the uninstalled copy of libgmp. Under the hood it's powered by builddir/meson-uninstalled/*-uninstalled.pc, which is a pkg-config feature.

[oscarbenjamin]

I'll try out meson devenv. Maybe that is a better way than copying the headers.

[dcbaker]

The history of this is that in the old days configure_file(input : 'file', output : 'file') would do a copy, which it was never meant to do, it just happened to work, so the copy argument was added. When I added fs.copyfile I was running into cases where a lengthy re-configure was required every time the input files were touched, deferring to build time made things a lot faster.

But yeah, the fact that you then need to put the copyfile target in the sources of your declare_dependency() is annoying, so I understand why some people don't like it.

[oscarbenjamin]

I don't mind what is the "right" way to do this but it would be good to document it. I imagine that a lot of projects need to select between files like foo-v1.h and foo-v2.h as part of configuration.

Right now from the docs it looks like copy: true is the way to do this but that is apparently deprecated. Initially I did not see the deprecation warnings: they only appeared once I added a meson_version >= x.y constraint.

[dcbaker]

Yeah, that's another thing that there's been long disagreement on :) I'd really like to treat an unset meson_version as >= 1, but there are projects that don't ever want warning for some reason.

[oscarbenjamin]

Note that this actually does do what I wanted:

project('foo', 'c')

foo_h = configure_file(
  input: 'foo-v1.h',
  output: 'foo.h',
  configuration: {},
)

install_headers([foo_h])

I don't know if that is the expected way to do this but it would be good to demonstrate the expected way here:
https://mesonbuild.com/Configuration.html
I just went from there to the configure_file docs and saw the copy parameter which looked like what I wanted. There is no mention in the docs of it being deprecated:
https://mesonbuild.com/Reference-manual_functions.html#configure_file