Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Compiler wrapper #352

Merged
merged 378 commits into from
Dec 19, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
378 commits
Select commit Hold shift + click to select a range
be36931
#3 Removed debug print, fixed python 3.7 typing information.
hiker May 13, 2024
8c790ce
#3 Updated comments.
hiker May 13, 2024
ade5f67
Modified the get_optimisation_script function examples and updated th…
May 13, 2024
15212ae
#3 Add Rsync tool.
hiker May 13, 2024
48c7543
#3 Removed now unused function.
hiker May 13, 2024
664c89c
#3 Added test for rsync.
hiker May 13, 2024
44756f7
#3 Fixed all mypy warnings about functions not checked.
hiker May 13, 2024
064ea35
#3 Replace all mock-tests to use subprocess so the name of the execut…
hiker May 13, 2024
22a1b91
#3 Remove duplicated flags.
hiker May 13, 2024
2e1b0a1
#3 Fixed changed order of linking.
hiker May 13, 2024
921daa8
#3 Removed run_command function.
hiker May 13, 2024
fc668cd
#3 Fixed 3.8 typing error.
hiker May 13, 2024
47a0a70
#3 Fixed unused imports.
hiker May 13, 2024
4389374
#3 Move flags checksum into Flags, and remove now unused tools.py file.
hiker May 14, 2024
3a015d2
#3 Renamed newtools to tools.
hiker May 14, 2024
202e274
#3 Made custom function for all git functions called (instead of just…
hiker May 14, 2024
3667030
#3 Updated and fixed comments.
hiker May 14, 2024
860d5a3
#3 Fixed errors in comments.
hiker May 14, 2024
25a1f58
Fixed minor errors in documentation.
hiker May 15, 2024
2562c22
Merge pull request #19 from hiker/18_optimisation_script
hiker May 15, 2024
72f7ab5
#3 Make it easier to create wrapper around standard compiler.
hiker May 15, 2024
53aeb62
#3 Added documentation for all tool related classes and their usage.
hiker May 15, 2024
8723a4e
#3 Added MISC category.
hiker May 21, 2024
358ae23
Addressed reviewer's comments.
hiker May 29, 2024
71e0b2f
Updated cli to properly use ToolBox etc, removing hard-coded gnu comm…
hiker May 29, 2024
b33b8b5
Merge remote-tracking branch 'origin/bom_master' into 3_better_compil…
hiker May 29, 2024
1109a88
Fixed mypy failures, including changes to import statement to avoid c…
hiker May 29, 2024
8e303f5
#3 Fix circular import.
hiker May 30, 2024
393f98a
Added #TODO so that this can be removed once fparser supports sentinels.
hiker Jun 2, 2024
2464e99
Fix typing problems by ignoring fparser.
hiker Jun 2, 2024
cce68e9
Merge pull request #21 from hiker/3_better_compiler_support
jasonjunweilyu Jun 3, 2024
8d9b5c5
Merge remote-tracking branch 'origin/bom_master' into 13_omp_sentinels
hiker Jun 3, 2024
572055a
Merge pull request #24 from hiker/13_omp_sentinels
jasonjunweilyu Jun 3, 2024
01558a2
Merge branch 'bom_master' into add_standard_artefacts
hiker Jun 3, 2024
4d85e70
Replaced more string names for artefacts with enums.
hiker Jun 4, 2024
10067a1
Removed EXECUTABLES from constants.
hiker Jun 4, 2024
137eb84
Moved Artefact class out of ArtefactStore and renamed it to ArtefactSet.
hiker Jun 4, 2024
b95611f
Moved OBJECT_FILES from constants into ArtefactSet.
hiker Jun 4, 2024
5f6625c
Moved OBJECT_ARCHIVES from constants to ArtefactSet.
hiker Jun 4, 2024
3d58184
Moved PRAGMAD_C from constants to ArtefactSet.
hiker Jun 4, 2024
f2942b1
Turned 'all_source' into an enum.
hiker Jun 4, 2024
d005c47
Allow integer as revision.
hiker Jun 4, 2024
73f91d4
Fixed flake8 error.
hiker Jun 4, 2024
6d5e82e
Removed specific functions to add/get fortran source files etc.
hiker Jun 5, 2024
37f7adc
Removed non-existing and unneccessary collections.
hiker Jun 5, 2024
ac93579
Try to fix all run_configs.
hiker Jun 4, 2024
2f633b9
Fixed rebase issues.
hiker Jun 6, 2024
03c6076
Added replace functionality to ArtefactStore, updated test_artefacts …
hiker Jun 6, 2024
499c9b5
Started to replace artefacts when files are pre-processed.
hiker Jun 6, 2024
7570696
Removed linker argument from linking step in all examples.
hiker Jun 7, 2024
41d9273
Try to get jules to link.
hiker Jun 7, 2024
56f001a
Fixed build_jules.
hiker Jun 7, 2024
8fb2be1
Fixed other issues raised in reviews.
hiker Jun 7, 2024
71cd141
Merge pull request #26 from hiker/3_better_compiler_support
jasonjunweilyu Jun 7, 2024
5b959c3
Merge branch '3_better_compiler_support' into add_standard_artefacts
hiker Jun 7, 2024
0f6c1a5
Try to get jules to link.
hiker Jun 7, 2024
588f647
Fixed other issues raised in reviews.
hiker Jun 7, 2024
cd81ce3
Merge branch 'bom_master' into add_standard_artefacts
hiker Jun 7, 2024
e857c94
Simplify handling of X90 files by replacing the X90 with x90, meaning…
hiker Jun 7, 2024
e024824
Make OBJECT_ARCHIVES also a dict, migrate more code to replace/add fi…
hiker Jun 11, 2024
a1346cf
Merge branch '3_better_compiler_support' into improve_artefact_workflow
hiker Jun 11, 2024
a0c2b98
Fixed some examples.
hiker Jun 11, 2024
169ff69
Merge branch 'master' into 3_better_compiler_support
hiker Jun 11, 2024
a34febc
Fix flake8 error.
hiker Jun 11, 2024
df893d6
Fixed failing tests.
hiker Jun 11, 2024
a2a8a54
Support empty comments.
hiker Jun 11, 2024
f03bc37
Fix preprocessor to not unnecessary remove and add files that are alr…
hiker Jun 12, 2024
7632775
Allow find_soure_files to be called more than once by adding files (n…
hiker Jun 12, 2024
e73179f
Updated lfric_common so that files created by configurator are writte…
hiker Jun 12, 2024
f7919ce
Use c_build_files instead of pragmad_c.
hiker Jun 12, 2024
50f4a42
Removed unnecessary str.
hiker Jun 12, 2024
36fa57b
Documented the new artefact set handling.
hiker Jun 13, 2024
77b2b97
Fixed typo.
hiker Jun 13, 2024
07e86d4
Make the PSyclone API configurable.
hiker Jun 13, 2024
5c895ba
Fixed formatting of documentation, properly used ArtefactSet names.
hiker Jun 13, 2024
e556115
Merge branch 'improve_artefact_workflow' into generalisation
hiker Jun 13, 2024
8a552f0
Support .f and .F Fortran files.
hiker Jun 14, 2024
f0e660d
Removed setter for tool.is_available, which was only used for testing.
hiker Jun 17, 2024
9f1ffc3
#3 Fix documentation and coding style issues from review.
hiker Jun 17, 2024
a0531a2
Renamed Categories into Category.
hiker Jun 17, 2024
4ff887c
Minor coding style cleanup.
hiker Jun 17, 2024
8a3b02d
Removed more unnecessary ().
hiker Jun 17, 2024
10ae159
Re-added (invalid) grab_pre_build call.
hiker Jun 17, 2024
622d0bd
Fixed typo.
hiker Jun 17, 2024
3f48703
Renamed set_default_vendor to set_default_compiler_suite.
hiker Jun 18, 2024
ba9245d
Renamed VendorTool to CompilerSuiteTool.
hiker Jun 18, 2024
f8c4418
Also accept a Path as exec_name specification for a tool.
hiker Jun 18, 2024
6a7aef8
Move the check_available function into the base class.
hiker Jun 18, 2024
0b73089
Fixed some types and documentation.
hiker Jun 18, 2024
7796246
Fix typing error.
hiker Jun 18, 2024
412a136
Added explanation for meta-compiler.
hiker Jun 18, 2024
8ec2227
Improved error handling and documentation.
hiker Jun 18, 2024
6d1a1ee
Replace mpiifort with mpifort to be a tiny bit more portable.
hiker Jun 18, 2024
8d4c66f
Use classes to group tests for git/svn/fcm together.
hiker Jun 18, 2024
46bc4ef
Fixed issue in get_transformation script, and moved script into lfric…
hiker Jun 18, 2024
ffdfd1a
Merge remote-tracking branch 'upstream/master' into 3_better_compiler…
hiker Jun 18, 2024
4f436f3
Code improvement as suggested by review.
hiker Jun 19, 2024
52d5751
Fixed run config
hiker Jun 19, 2024
562d56f
Merge branch 'add_standard_artefacts' into improve_artefact_workflow
hiker Jun 20, 2024
ca7e477
Merge pull request #28 from hiker/add_standard_artefacts
jasonjunweilyu Jun 20, 2024
1150318
Merge branch 'improve_artefact_workflow' into generalisation
hiker Jun 20, 2024
516ec34
Added reference to ticket.
hiker Jun 20, 2024
c17545a
Merge branch 'bom_master' into improve_artefact_workflow
hiker Jun 20, 2024
19005e3
Merge branch 'improve_artefact_workflow' into generalisation
hiker Jun 20, 2024
ee2fc85
Updated type information.
hiker Jun 20, 2024
5912e59
More typing fixes.
hiker Jun 20, 2024
83ba6a1
Fixed typing warnings.
hiker Jun 21, 2024
7e66eca
As requested by reviewer removed is_working_copy functionality.
hiker Jun 21, 2024
ddd3cf2
Issue a warning (which can be silenced) when a tool in a toolbox is r…
hiker Jun 21, 2024
68bc218
Fixed flake8.
hiker Jun 21, 2024
2bdcb8c
Merge remote-tracking branch 'upstream/master' into 3_better_compiler…
hiker Jun 21, 2024
c02729c
Fixed flake8.
hiker Jun 21, 2024
a29be95
Merge remote-tracking branch 'upstream/master' into 3_better_compiler…
hiker Jun 21, 2024
03bed8e
Merge branch 'master' into 3_better_compiler_support
hiker Jun 24, 2024
9ccdefd
Merge branch '3_better_compiler_support' into bom_master
hiker Jun 24, 2024
00fa8b2
Merge branch '3_better_compiler_support' into improve_artefact_workflow
hiker Jun 24, 2024
c21d34f
Merge branch '3_better_compiler_support' into add_standard_artefacts
hiker Jun 24, 2024
d8fe42b
Merge branch 'add_standard_artefacts' into improve_artefact_workflow
hiker Jun 24, 2024
2f069c8
Merge branch 'bom_master' into add_standard_artefacts
hiker Jun 24, 2024
9abfc1d
Merge branch 'add_standard_artefacts' into improve_artefact_workflow
hiker Jun 24, 2024
772e79c
Merge branch 'improve_artefact_workflow' into generalisation
hiker Jun 24, 2024
c04ab9e
Fixed failing test.
hiker Jun 24, 2024
87b8012
Addressed issues raised in review.
hiker Jun 24, 2024
523e876
Removed now unnecessary operations.
hiker Jun 24, 2024
538e4da
Updated some type information.
hiker Jun 24, 2024
e3114c7
Merge pull request #29 from hiker/improve_artefact_workflow
jasonjunweilyu Jun 24, 2024
88cf3ea
Merge branch 'bom_master' into generalisation
hiker Jun 24, 2024
0f1ead3
Fixed all references to APIs to be consistent with PSyclone 2.5.
hiker Jun 25, 2024
8ab8c40
Added api to the checksum computation.
hiker Jun 25, 2024
674575e
Fixed type information.
hiker Jun 25, 2024
c980e7e
Added test to verify that changing the api changes the checksum.
hiker Jun 28, 2024
0a61359
Merge pull request #30 from hiker/generalisation
jasonjunweilyu Jul 1, 2024
ff306da
Make compiler version a tuple of integers
lukehoffmann Jul 16, 2024
c5b1b02
Update some tests to use tuple versions
lukehoffmann Jul 16, 2024
ccf4ad8
Explicitly test handling of bad version format
lukehoffmann Jul 16, 2024
fd02c8c
Fix formatting
lukehoffmann Jul 16, 2024
10eb726
Tidying up
lukehoffmann Jul 16, 2024
ea18256
Make compiler raise an error for any invalid version string
lukehoffmann Jul 18, 2024
6e6579a
Merge remote-tracking branch 'origin/master' into bom_master
hiker Jul 19, 2024
932f401
Merge remote-tracking branch 'upstream/master' into bom_master
hiker Jul 19, 2024
120116c
Check compiler version string for compiler name
lukehoffmann Jul 22, 2024
5a24c24
Fix formatting
lukehoffmann Jul 23, 2024
dbc3d73
Add compiler.get_version_string() method
lukehoffmann Jul 24, 2024
90adcca
Add mpi and openmp settings to BuildConfig, made compiler MPI aware.
hiker Jul 24, 2024
150dc37
Looks like the circular dependency has been fixed.
hiker Jul 24, 2024
db34597
Revert "Looks like the circular dependency has been fixed." ...
hiker Jul 24, 2024
7c347c8
Don't even try to find a C compiler if no C files are to be compiled.
hiker Jul 24, 2024
ff71393
Updated gitignore to ignore (recently renamed) documentation.
hiker Jul 25, 2024
5dc01f3
Fixed failing test.
hiker Jul 25, 2024
2b5c4bd
Return from compile Fortran early if there are no files to compiles. …
hiker Jul 25, 2024
46706c5
Add MPI enables wrapper for intel and gnu compiler.
hiker Jul 25, 2024
d55344f
Fixed test.
hiker Jul 25, 2024
d00c187
Automatically add openmp flag to compiler and linker based on BuildCo…
hiker Jul 25, 2024
e9333fd
Removed enforcement of keyword parameters, which is not supported in …
hiker Jul 25, 2024
a85bc7a
Fixed failing test.
hiker Jul 25, 2024
6c8e8df
Support more than one tool of a given suite by sorting them.
hiker Jul 26, 2024
3bb726e
Use different version checkout for each compiler vendor with mixins
lukehoffmann Jul 26, 2024
286d457
Refactoring, remove unittest compiler class
lukehoffmann Jul 26, 2024
dae9102
Fix some mypy errors
lukehoffmann Jul 28, 2024
cdcc023
Use 'Union' type hint to fix build checks
lukehoffmann Jul 28, 2024
493f09d
Added option to add flags to a tool.
hiker Jul 29, 2024
be9e97d
Introduce proper compiler wrapper, used this to implement properly wr…
hiker Jul 29, 2024
cbde919
Fixed typo in types.
hiker Jul 29, 2024
0bdb734
Return run_version_command to base Compiler class
lukehoffmann Jul 29, 2024
b97ec50
Add a missing type hint
lukehoffmann Jul 29, 2024
b1b4057
Merge remote-tracking branch 'upstream/master' into bom_master
hiker Jul 29, 2024
93c6810
Added (somewhat stupid) 'test' to reach 100% coverage of PSyclone tool.
hiker Jul 30, 2024
50a891a
Simplified MPI support in wrapper.
hiker Jul 30, 2024
5b51aa1
Merge pull request #13 from hiker/update_bom_master_to_upstream_master
jasonjunweilyu Jul 30, 2024
6b05755
More compiler wrapper coverage.
hiker Jul 31, 2024
4d03ed6
Removed duplicated function.
hiker Jul 31, 2024
ada8a8f
Removed debug print.
hiker Jul 31, 2024
c3d0e6e
Removed permanently changing compiler attributes, which can cause tes…
hiker Jul 31, 2024
1abdd53
More test for C compiler wrapper.
hiker Aug 1, 2024
e921a3d
More work on compiler wrapper tests.
hiker Aug 1, 2024
81136b1
Fixed version and availability handling, added missing tests for 100%…
hiker Aug 1, 2024
84a25b9
Fixed typing error.
hiker Aug 1, 2024
13da4fe
Try to fix python 3.7.
hiker Aug 1, 2024
590dc75
Tried to fix failing tests.
hiker Aug 1, 2024
19eab9f
Remove inheritance from mixins and use protocol
lukehoffmann Aug 2, 2024
cbe1cb8
Simplify compiler inheritance
lukehoffmann Aug 2, 2024
bf81dee
Merge pull request #9 from hiker/version_tuples
hiker Aug 2, 2024
196e034
Merge branch 'bom_master' into mpi_omp_support
hiker Aug 2, 2024
a375a34
Merge branch 'mpi_omp_support' into compiler_wrapper
hiker Aug 2, 2024
83a3045
Updated wrapper and tests to handle error raised in get_version.
hiker Aug 7, 2024
becb483
Simplified regular expressions (now tests cover detection of version …
hiker Aug 7, 2024
19576b2
Test for missing mixin.
hiker Aug 7, 2024
01e5480
Use the parsing mixing from the compiler in a compiler wrapper.
hiker Aug 7, 2024
cd6d9b0
Use setattr instead of assignment to make mypy happy.
hiker Aug 7, 2024
540b544
Simplify usage of compiler-specific parsing mixins.
hiker Aug 7, 2024
08ac3b5
Minor code cleanup.
hiker Aug 7, 2024
07d84db
Updated documentation.
hiker Aug 7, 2024
3076715
Simplify usage of compiler-specific parsing mixins.
hiker Aug 7, 2024
7a50859
Test for missing mixin.
hiker Aug 7, 2024
749fa2d
Fixed test.
hiker Aug 7, 2024
20fef2c
Merge branch 'version_tuples' into mpi_omp_support
hiker Aug 8, 2024
6c438dc
Merge branch 'mpi_omp_support' into compiler_wrapper
hiker Aug 8, 2024
86612ad
Added missing openmp_flag property to compiler wrapper.
hiker Aug 8, 2024
8b7ea63
Don't use isinstance for consistency check, which does not work for C…
hiker Aug 8, 2024
51b8324
Fixed isinstance test for C compilation which doesn't work with a Com…
hiker Aug 8, 2024
c8c8956
Use a linker's compiler to determine MPI support. Removed mpi propert…
hiker Aug 8, 2024
9598c09
Added more tests for invalid version numbers.
hiker Aug 9, 2024
5c6f99f
Added more test cases for invalid version number, improved regex to w…
hiker Aug 9, 2024
c1eb158
Fixed typo in test.
hiker Aug 9, 2024
1524777
Merge branch 'version_tuples' into mpi_omp_support
hiker Aug 9, 2024
3731719
Merge branch 'mpi_omp_support' into compiler_wrapper
hiker Aug 9, 2024
7fa4340
Fixed flake/mypy errors.
hiker Aug 9, 2024
f99145a
Combine wrapper flags with flags from wrapped compiler.
hiker Aug 10, 2024
f9a467d
Made mypy happy.
hiker Aug 10, 2024
02fbb7b
Fixed test.
hiker Aug 12, 2024
955befc
Merge remote-tracking branch 'upstream/master' into bom_master
hiker Aug 12, 2024
552e4cc
Merge pull request #18 from hiker/version_tuples
lukehoffmann Aug 12, 2024
b38cdb8
Merge remote-tracking branch 'upstream/master' into mpi_omp_support
hiker Aug 12, 2024
247eeb5
Merge branch 'mpi_omp_support' into compiler_wrapper
hiker Aug 12, 2024
6826c85
Merge remote-tracking branch 'upstream/master' into version_tuples
hiker Aug 13, 2024
e5aca3b
Merge branch 'version_tuples' into mpi_omp_support
hiker Aug 13, 2024
c6b4ef2
Merge branch 'mpi_omp_support' into compiler_wrapper
hiker Aug 13, 2024
ccbe810
Split tests into smaller individual ones, fixed missing asssert in test.
hiker Aug 14, 2024
f8f8ab0
Parameterised compiler version tests to also test wrapper.
hiker Aug 14, 2024
1543841
Added missing MPI parameter when getting the compiler.
hiker Aug 14, 2024
cd8ec62
Fixed comments.
hiker Aug 14, 2024
13935c0
Order parameters to be in same order for various compiler classes.
hiker Aug 14, 2024
de80c44
Merge branch 'bom_master' into mpi_omp_support
hiker Aug 14, 2024
009478f
Merge remote-tracking branch 'origin/bom_master' into mpi_omp_support
hiker Aug 14, 2024
304d6e9
Merge branch 'mpi_omp_support' into compiler_wrapper
hiker Aug 14, 2024
7887d24
Remove stray character
lukehoffmann Aug 16, 2024
83836e7
Merge pull request #14 from hiker/mpi_omp_support
lukehoffmann Aug 16, 2024
2f9beab
Merge remote-tracking branch 'origin/bom_master' into compiler_wrapper
hiker Aug 16, 2024
a79b41d
Merge remote-tracking branch 'MetOffice/master' into bom_master
lukehoffmann Aug 21, 2024
420fa5e
Added getter for wrapped compiler.
hiker Aug 21, 2024
07a50a6
Fixed small error that would prevent nested compiler wrappers from be…
hiker Aug 21, 2024
282df65
Merge branch 'bom_master' into compiler_wrapper
hiker Aug 21, 2024
e5dd7c9
Added a cast to make mypy happy.
hiker Aug 21, 2024
0a6f374
Fixed spelling mistake in option.
hiker Sep 4, 2024
09dceb5
Clarified documentation.
hiker Sep 4, 2024
ef17154
Removed unnecessary functions in CompilerWrapper.
hiker Sep 4, 2024
2617322
Made mpi and openmp default to False in the BuildConfig constructor.
hiker Oct 21, 2024
f165e46
Removed white space.
hiker Oct 21, 2024
df5f63a
Support compilers that do not support OpenMP.
hiker Nov 8, 2024
788e75a
Added documentation for openmp parameter.
hiker Nov 8, 2024
ebe3f09
Mpi omp support (#343)
hiker Nov 11, 2024
755ad9a
Merge branch 'mpi_omp_support' into compiler_wrapper
hiker Nov 11, 2024
17119fe
Mpi omp support (#346)
hiker Nov 12, 2024
23d2f8e
Merge branch 'dev' into compiler_wrapper
hiker Nov 12, 2024
519b8fb
Compiler wrapper (#347)
hiker Nov 12, 2024
2a15f0e
Merge remote-tracking branch 'upstream/dev' into compiler_wrapper
hiker Nov 12, 2024
6fd7a4d
Merge remote-tracking branch 'upstream/master' into mpi_omp_support
hiker Nov 19, 2024
852dd48
Merge branch 'mpi_omp_support' into compiler_wrapper
hiker Nov 19, 2024
68ccc4c
Minor updates to documentation and coding style.
hiker Nov 20, 2024
600f9bf
Addressed documentation issues raised in review.
hiker Dec 3, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
63 changes: 63 additions & 0 deletions Documentation/source/site-specific-config.rst
Original file line number Diff line number Diff line change
Expand Up @@ -161,6 +161,64 @@ On the other hand, if no MPI is requested, an MPI-enabled compiler
might be returned, which does not affect the final result, since
an MPI compiler just adds include- and library-paths.


Compiler Wrapper
================
Fab supports the concept of a compiler wrapper, which is typically
a script that calls the actual compiler. An example for a wrapper is
`mpif90`, which might call a GNU or Intel based compiler (with additional
parameter for accessing the MPI specific include and library paths.).
An example to create a `mpicc` wrapper (note that this wrapper is already
part of Fab, there is no need to explicitly add this yourself):

.. code-block::
:linenos:
:caption: Defining an mpicc compiler wrapper

class Mpicc(CompilerWrapper):
def __init__(self, compiler: Compiler):
super().__init__(name=f"mpicc-{compiler.name}",
exec_name="mpicc",
compiler=compiler, mpi=True)

The tool system allows several different tools to use the same name
for the executable, as long as the Fab name is different, i.e. the
`mpicc-{compiler.name}`. The tool
repository will automatically add compiler wrapper for `mpicc` and
`mpif90` for any compiler that is added by Fab. If you want to add
a new compiler, which can also be invoked using `mpicc`, you need
to add a compiler wrapper as follows:

.. code-block::
:linenos:
:caption: Adding a mpicc wrapper to the tool repository

my_new_compiler = MyNewCompiler()
ToolRepository().add_tool(my_new_compiler)
my_new_mpicc = Mpicc(MyNewCompiler)
ToolRepository().add_tool(my_new_mpicc)

When creating a completely new compiler and compiler wrapper
as in the example above, it is strongly recommended to add
the new compiler instance to the tool repository as well. This will
allow the wrapper and the wrapped compiler to share flags. For example,
a user script can query the ToolRepository to get the original compiler
and modify its flags. These modification will then automatically be
applied to the wrapper as well:

.. code-block::
:linenos:
:caption: Sharing flags between compiler and compiler wrapper

tr = ToolRepository()
my_compiler = tr.get_tool(Category.C_COMPILER, "my_compiler")
my_mpicc = tr.get_tool(Category.C_COMPILER, "mpicc-my_compiler")

my_compiler.add_flags(["-a", "-b"])

assert my_mpicc.flags == ["-a", "-b"]


TODO
====
At this stage compiler flags are still set in the corresponding Fab
Expand All @@ -169,3 +227,8 @@ definition in the compiler objects.
This will allow a site to define their own set of default flags to
be used with a certain compiler by replacing or updating a compiler
instance in the Tool Repository

Also, a lot of content in this chapter is not actually about site-specific
configuration. This should likely be renamed or split (once we
have details about the using site-specific configuration, which might be
once the Baf base script is added to Fab).
13 changes: 8 additions & 5 deletions source/fab/steps/compile_c.py
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@
import logging
import os
from dataclasses import dataclass
from typing import List, Dict, Optional, Tuple
from typing import cast, Dict, List, Optional, Tuple

from fab import FabException
from fab.artefacts import (ArtefactsGetter, ArtefactSet, ArtefactStore,
Expand All @@ -19,7 +19,7 @@
from fab.metrics import send_metric
from fab.parse.c import AnalysedC
from fab.steps import check_for_errors, run_mp, step
from fab.tools import Category, CCompiler, Flags
from fab.tools import Category, Compiler, Flags
from fab.util import CompiledFile, log_or_dot, Timer, by_type

logger = logging.getLogger(__name__)
Expand Down Expand Up @@ -124,9 +124,12 @@ def _compile_file(arg: Tuple[AnalysedC, MpCommonArgs]):
analysed_file, mp_payload = arg
config = mp_payload.config
compiler = config.tool_box[Category.C_COMPILER]
if not isinstance(compiler, CCompiler):
raise RuntimeError(f"Unexpected tool '{compiler.name}' of type "
f"'{type(compiler)}' instead of CCompiler")
if compiler.category != Category.C_COMPILER:
raise RuntimeError(f"Unexpected tool '{compiler.name}' of category "
f"'{compiler.category}' instead of CCompiler")
# Tool box returns a Tool, in order to make mypy happy, we need
# to cast it to be a Compiler.
compiler = cast(Compiler, compiler)
MatthewHambley marked this conversation as resolved.
Show resolved Hide resolved
with Timer() as timer:
flags = Flags(mp_payload.flags.flags_for_path(path=analysed_file.fpath,
config=config))
Expand Down
22 changes: 14 additions & 8 deletions source/fab/steps/compile_fortran.py
Original file line number Diff line number Diff line change
Expand Up @@ -14,15 +14,15 @@
from dataclasses import dataclass
from itertools import chain
from pathlib import Path
from typing import List, Set, Dict, Tuple, Optional, Union
from typing import cast, Dict, List, Optional, Set, Tuple, Union

from fab.artefacts import (ArtefactsGetter, ArtefactSet, ArtefactStore,
FilterBuildTrees)
from fab.build_config import BuildConfig, FlagsConfig
from fab.metrics import send_metric
from fab.parse.fortran import AnalysedFortran
from fab.steps import check_for_errors, run_mp, step
from fab.tools import Category, Compiler, Flags, FortranCompiler
from fab.tools import Category, Compiler, Flags
from fab.util import (CompiledFile, log_or_dot_finish, log_or_dot, Timer,
by_type, file_checksum)

Expand Down Expand Up @@ -133,9 +133,12 @@ def handle_compiler_args(config: BuildConfig, common_flags=None,

# Command line tools are sometimes specified with flags attached.
compiler = config.tool_box[Category.FORTRAN_COMPILER]
if not isinstance(compiler, FortranCompiler):
raise RuntimeError(f"Unexpected tool '{compiler.name}' of type "
f"'{type(compiler)}' instead of FortranCompiler")
if compiler.category != Category.FORTRAN_COMPILER:
raise RuntimeError(f"Unexpected tool '{compiler.name}' of category "
f"'{compiler.category}' instead of FortranCompiler")
# The ToolBox returns a Tool. In order to make mypy happy, we need to
# cast this to become a Compiler.
compiler = cast(Compiler, compiler)
MatthewHambley marked this conversation as resolved.
Show resolved Hide resolved
logger.info(
f'Fortran compiler is {compiler} {compiler.get_version_string()}')

Expand Down Expand Up @@ -263,10 +266,13 @@ def process_file(arg: Tuple[AnalysedFortran, MpCommonArgs]) \
config = mp_common_args.config
compiler = config.tool_box.get_tool(Category.FORTRAN_COMPILER,
config.mpi)
if not isinstance(compiler, FortranCompiler):
raise RuntimeError(f"Unexpected tool '{compiler.name}' of type "
f"'{type(compiler)}' instead of "
if compiler.category != Category.FORTRAN_COMPILER:
raise RuntimeError(f"Unexpected tool '{compiler.name}' of "
f"category '{compiler.category}' instead of "
f"FortranCompiler")
# The ToolBox returns a Tool, but we need to tell mypy that
# this is a Compiler
compiler = cast(Compiler, compiler)
MatthewHambley marked this conversation as resolved.
Show resolved Hide resolved
flags = Flags(mp_common_args.flags.flags_for_path(
path=analysed_file.fpath, config=config))

Expand Down
11 changes: 5 additions & 6 deletions source/fab/tools/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -11,8 +11,8 @@
from fab.tools.category import Category
from fab.tools.compiler import (CCompiler, Compiler, FortranCompiler, Gcc,
Gfortran, GnuVersionHandling, Icc, Ifort,
IntelVersionHandling, MpiGcc, MpiGfortran,
MpiIcc, MpiIfort)
IntelVersionHandling)
from fab.tools.compiler_wrapper import CompilerWrapper, Mpicc, Mpif90
from fab.tools.flags import Flags
from fab.tools.linker import Linker
from fab.tools.psyclone import Psyclone
Expand All @@ -29,6 +29,7 @@
"CCompiler",
"Compiler",
"CompilerSuiteTool",
"CompilerWrapper",
"Cpp",
"CppFortran",
"Fcm",
Expand All @@ -43,10 +44,8 @@
"Ifort",
"IntelVersionHandling",
"Linker",
"MpiGcc",
"MpiGfortran",
"MpiIcc",
"MpiIfort",
"Mpif90",
"Mpicc",
"Preprocessor",
"Psyclone",
"Rsync",
Expand Down
Loading
Loading