Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Compiler wrapper #352

Merged
merged 378 commits into from
Dec 19, 2024
Merged

Compiler wrapper #352

merged 378 commits into from
Dec 19, 2024

Conversation

hiker
Copy link
Collaborator

@hiker hiker commented Nov 20, 2024

Introduces proper compiler wrapper, e.g. for mpif90.

hiker and others added 30 commits May 13, 2024 14:36
Allowing file-specific transformation scripts
lukehoffmann and others added 24 commits August 16, 2024 15:20
Add support for specifying MPI and OpenMP in the config file
* #3 Introduce fixtures for Fortran compiler and tool_box.

* #3 Remove explicit compiler information from MpCommonArgs (since it's already part of the config toolbox).

* #3 Added linker as tool.

* #3 Added test for linking shared libraries.

* #3 Pass compiler flags to the linker if a compiler was specified.

* #3 Remove unused function.

* #3 Removed more unused code.

* #3 Automatically add a linker for each compiler.

* #3 Fixed typo.

* #3 Support vendor for compiler and linker.

* #3 Make linker having a vendor, too.

* #3 Add set_default_vendor method to tool repository.

* Ignore build directory for git.

* Updated test.

* # Fix some mypy errors and warnings.

* Avoid using get() for singleton, instead use __new__ which makes mypy happier.

* Changed the transformation_script parameter of function psyclone to accept a function that can return file-specific transformation scripts

* Make mypy happy by using patch.object.

* Remove more comments and confusion about mypy :)

* Try to make mypy happy on older python versions.

* Make flake8 happy.

* Sort imported name alphabetically.

* Try to fix failing hash test (and add some additional improvements in the test).

* Removed fpath= for input transformation_script function to pass mypy test for Python 3.7; Moved transformation_script_hash test to unit test from system test

* Fix mypy typing check errors for psyclone unit test

* Fix config typing issue with mypy in psyclone unit test

* Fix flake8 issues; Revert Config mypy typing fix

* Add comment to ignore typing check for fpath parameter of input transformation_script function

* Fix assert check after transformation_script function is changed from being called twice to once

* Filter out 'no transformation script' warning for psyclone system test

* Replace 'ignore' typing of fpath of transformation_script with removing keyword argument

* #3 Support proper tests to check if tools are available.

* 1. Updated transformation_script description; 2. Modified mock_transformation_script; 3.Removed redundant _analysis_for_prebuilds

* Updated lfric/atm.py and lfric/gungho.py examples to pass in transformation_script functions

* Added description for the psyclone step to instructions on writing a config

* #3 Added git as a tool.

* #3 Fix incorrect | usage in typing.

* #3 Added unit tests for git.

* #3 Renamed git.py to versioning.py, to avoid name clash with the corresponding test_git.py tests.

* #3 Converted svn and fcm to tools.

* #3 Fixed missing whitespace.

* Modified the documentation for writing a config with PSyclone

* Add config as a parameter for run_psyclone for the transformation_script to use;Updated the related functions and tests;
Changed the logic of the transformation_script examples

* #3 Replaced ar with tool object.

* #3 Added tests for ar.py.

* #3 Removed debug output.

* #3 Converted PSyclone to be a tool.

* #3 Removed debug print, fixed python 3.7 typing information.

* #3 Updated comments.

* Modified the get_optimisation_script function examples and updated the doc formatting

* #3 Add Rsync tool.

* #3 Removed now unused function.

* #3 Added test for rsync.

* #3 Fixed all mypy warnings about functions not checked.

* #3 Replace all mock-tests to use subprocess so the name of the executable is tested as well.

* #3 Remove duplicated flags.

* #3 Fixed changed order of linking.

* #3 Removed run_command function.

* #3 Fixed 3.8 typing error.

* #3 Fixed unused imports.

* #3 Move flags checksum into Flags, and remove now unused tools.py file.

* #3 Renamed newtools to tools.

* #3 Made custom function for all git functions called (instead of just calling run).

* #3 Updated and fixed comments.

* #3 Fixed errors in comments.

* Fixed minor errors in documentation.

* #3 Make it easier to create wrapper around standard compiler.

* #3 Added documentation for all tool related classes and their usage.

* #3 Added MISC category.

* Addressed reviewer's comments.

* Updated cli to properly use ToolBox etc, removing hard-coded gnu command linker option.

* Fixed mypy failures, including changes to import statement to avoid cyclic imports :(.

* #3 Fix circular import.

* Added #TODO so that this can be removed once fparser supports sentinels.

* Fix typing problems by ignoring fparser.

* Replaced more string names for artefacts with enums.

* Removed EXECUTABLES from constants.

* Moved Artefact class out of ArtefactStore and renamed it to ArtefactSet.

* Moved OBJECT_FILES from constants into ArtefactSet.

* Moved OBJECT_ARCHIVES from constants to ArtefactSet.

* Moved PRAGMAD_C from constants to ArtefactSet.

* Turned 'all_source' into an enum.

* Allow integer as revision.

* Fixed flake8 error.

* Removed specific functions to add/get fortran source files etc.

* Removed non-existing and unneccessary collections.

* Try to fix all run_configs.

* Fixed rebase issues.

* Added replace functionality to ArtefactStore, updated test_artefacts to cover all lines in that file.

* Started to replace artefacts when files are pre-processed.

* Removed linker argument from linking step in all examples.

* Try to get jules to link.

* Fixed build_jules.

* Fixed other issues raised in reviews.

* Try to get jules to link.

* Fixed other issues raised in reviews.

* Simplify handling of X90 files by replacing the X90 with x90, meaning only one artefact set is involved when running PSyclone.

* Make OBJECT_ARCHIVES also a dict, migrate more code to replace/add files to the default build artefact collections.

* Fixed some examples.

* Fix flake8 error.

* Fixed failing tests.

* Support empty comments.

* Fix preprocessor to not unnecessary remove and add files that are already in the output directory.

* Allow find_soure_files to be called more than once by adding files (not replacing artefact).

* Updated lfric_common so that files created by configurator are written in build (not source).

* Use c_build_files instead of pragmad_c.

* Removed unnecessary str.

* Documented the new artefact set handling.

* Fixed typo.

* Make the PSyclone API configurable.

* Fixed formatting of documentation, properly used ArtefactSet names.

* Support .f and .F Fortran files.

* Removed setter for tool.is_available, which was only used for testing.

* #3 Fix documentation and coding style issues from review.

* Renamed Categories into Category.

* Minor coding style cleanup.

* Removed more unnecessary ().

* Re-added (invalid) grab_pre_build call.

* Fixed typo.

* Renamed set_default_vendor to set_default_compiler_suite.

* Renamed VendorTool to CompilerSuiteTool.

* Also accept a Path as exec_name specification for a tool.

* Move the check_available function into the base class.

* Fixed some types and documentation.

* Fix typing error.

* Added explanation for meta-compiler.

* Improved error handling and documentation.

* Replace mpiifort with mpifort to be a tiny bit more portable.

* Use classes to group tests for git/svn/fcm together.

* Fixed issue in get_transformation script, and moved script into lfric_common to remove code duplication.

* Code improvement as suggested by review.

* Fixed run config

* Added reference to ticket.

* Updated type information.

* More typing fixes.

* Fixed typing warnings.

* As requested by reviewer removed is_working_copy functionality.

* Issue a warning (which can be silenced) when a tool in a toolbox is replaced.

* Fixed flake8.

* Fixed flake8.

* Fixed failing test.

* Addressed issues raised in review.

* Removed now unnecessary operations.

* Updated some type information.

* Fixed all references to APIs to be consistent with PSyclone 2.5.

* Added api to the checksum computation.

* Fixed type information.

* Added test to verify that changing the api changes the checksum.

* Make compiler version a tuple of integers

* Update some tests to use tuple versions

* Explicitly test handling of bad version format

* Fix formatting

* Tidying up

* Make compiler raise an error for any invalid version string

Assume these compilers don't need to be hashed.
Saves dealing with empty tuples.

* Check compiler version string for compiler name

* Fix formatting

* Add compiler.get_version_string() method

Includes other cleanup from PR comments

* Add mpi and openmp settings to BuildConfig, made compiler MPI aware.

* Looks like the circular dependency has been fixed.

* Revert "Looks like the circular dependency has been fixed." ...
while it works with the tests, a real application still triggered it.

This reverts commit 150dc37.

* Don't even try to find a C compiler if no C files are to be compiled.

* Updated gitignore to ignore (recently renamed) documentation.

* Fixed failing test.

* Return from compile Fortran early if there are no files to compiles. Fixed coding style.

* Add MPI enables wrapper for intel and gnu compiler.

* Fixed test.

* Automatically add openmp flag to compiler and linker based on BuildConfig.

* Removed enforcement of keyword parameters, which is not supported in python 3.7.

* Fixed failing test.

* Support more than one tool of a given suite by sorting them.

* Use different version checkout for each compiler vendor with mixins

* Refactoring, remove unittest compiler class

* Fix some mypy errors

* Use 'Union' type hint to fix build checks

* Return run_version_command to base Compiler class

Provides default version command that can be overridden for other compilers.
Also fix some incorrect tests
Other tidying

* Add a missing type hint

* Remove inheritance from mixins and use protocol

* Simplify compiler inheritance

Mixins have static methods with unique names,
overrides only happen in concrete classes

* Simplify usage of compiler-specific parsing mixins.

* Test for missing mixin.

* Fixed test.

* Added more tests for invalid version numbers.

* Added more test cases for invalid version number, improved regex to work as expected.

* Fixed typo in test.

* Fixed test.

* Split tests into smaller individual ones, fixed missing asssert in test.

* Parameterised compiler version tests to also test wrapper.

* Added missing MPI parameter when getting the compiler.

* Fixed comments.

* Order parameters to be in same order for various compiler classes.

* Remove stray character

* Made mpi and openmp default to False in the BuildConfig constructor.

* Removed white space.

* Support compilers that do not support OpenMP.

* Added documentation for openmp parameter.

---------

Co-authored-by: Junwei Lyu <[email protected]>
Co-authored-by: Junwei Lyu <[email protected]>
Co-authored-by: Junwei Lyu <[email protected]>
Co-authored-by: Junwei Lyu <[email protected]>
Co-authored-by: jasonjunweilyu <[email protected]>
Co-authored-by: Luke Hoffmann <[email protected]>
Co-authored-by: Luke Hoffmann <[email protected]>
* #3 Introduce fixtures for Fortran compiler and tool_box.

* #3 Remove explicit compiler information from MpCommonArgs (since it's already part of the config toolbox).

* #3 Added linker as tool.

* #3 Added test for linking shared libraries.

* #3 Pass compiler flags to the linker if a compiler was specified.

* #3 Remove unused function.

* #3 Removed more unused code.

* #3 Automatically add a linker for each compiler.

* #3 Fixed typo.

* #3 Support vendor for compiler and linker.

* #3 Make linker having a vendor, too.

* #3 Add set_default_vendor method to tool repository.

* Ignore build directory for git.

* Updated test.

* # Fix some mypy errors and warnings.

* Avoid using get() for singleton, instead use __new__ which makes mypy happier.

* Changed the transformation_script parameter of function psyclone to accept a function that can return file-specific transformation scripts

* Make mypy happy by using patch.object.

* Remove more comments and confusion about mypy :)

* Try to make mypy happy on older python versions.

* Make flake8 happy.

* Sort imported name alphabetically.

* Try to fix failing hash test (and add some additional improvements in the test).

* Removed fpath= for input transformation_script function to pass mypy test for Python 3.7; Moved transformation_script_hash test to unit test from system test

* Fix mypy typing check errors for psyclone unit test

* Fix config typing issue with mypy in psyclone unit test

* Fix flake8 issues; Revert Config mypy typing fix

* Add comment to ignore typing check for fpath parameter of input transformation_script function

* Fix assert check after transformation_script function is changed from being called twice to once

* Filter out 'no transformation script' warning for psyclone system test

* Replace 'ignore' typing of fpath of transformation_script with removing keyword argument

* #3 Support proper tests to check if tools are available.

* 1. Updated transformation_script description; 2. Modified mock_transformation_script; 3.Removed redundant _analysis_for_prebuilds

* Updated lfric/atm.py and lfric/gungho.py examples to pass in transformation_script functions

* Added description for the psyclone step to instructions on writing a config

* #3 Added git as a tool.

* #3 Fix incorrect | usage in typing.

* #3 Added unit tests for git.

* #3 Renamed git.py to versioning.py, to avoid name clash with the corresponding test_git.py tests.

* #3 Converted svn and fcm to tools.

* #3 Fixed missing whitespace.

* Modified the documentation for writing a config with PSyclone

* Add config as a parameter for run_psyclone for the transformation_script to use;Updated the related functions and tests;
Changed the logic of the transformation_script examples

* #3 Replaced ar with tool object.

* #3 Added tests for ar.py.

* #3 Removed debug output.

* #3 Converted PSyclone to be a tool.

* #3 Removed debug print, fixed python 3.7 typing information.

* #3 Updated comments.

* Modified the get_optimisation_script function examples and updated the doc formatting

* #3 Add Rsync tool.

* #3 Removed now unused function.

* #3 Added test for rsync.

* #3 Fixed all mypy warnings about functions not checked.

* #3 Replace all mock-tests to use subprocess so the name of the executable is tested as well.

* #3 Remove duplicated flags.

* #3 Fixed changed order of linking.

* #3 Removed run_command function.

* #3 Fixed 3.8 typing error.

* #3 Fixed unused imports.

* #3 Move flags checksum into Flags, and remove now unused tools.py file.

* #3 Renamed newtools to tools.

* #3 Made custom function for all git functions called (instead of just calling run).

* #3 Updated and fixed comments.

* #3 Fixed errors in comments.

* Fixed minor errors in documentation.

* #3 Make it easier to create wrapper around standard compiler.

* #3 Added documentation for all tool related classes and their usage.

* #3 Added MISC category.

* Addressed reviewer's comments.

* Updated cli to properly use ToolBox etc, removing hard-coded gnu command linker option.

* Fixed mypy failures, including changes to import statement to avoid cyclic imports :(.

* #3 Fix circular import.

* Added #TODO so that this can be removed once fparser supports sentinels.

* Fix typing problems by ignoring fparser.

* Replaced more string names for artefacts with enums.

* Removed EXECUTABLES from constants.

* Moved Artefact class out of ArtefactStore and renamed it to ArtefactSet.

* Moved OBJECT_FILES from constants into ArtefactSet.

* Moved OBJECT_ARCHIVES from constants to ArtefactSet.

* Moved PRAGMAD_C from constants to ArtefactSet.

* Turned 'all_source' into an enum.

* Allow integer as revision.

* Fixed flake8 error.

* Removed specific functions to add/get fortran source files etc.

* Removed non-existing and unneccessary collections.

* Try to fix all run_configs.

* Fixed rebase issues.

* Added replace functionality to ArtefactStore, updated test_artefacts to cover all lines in that file.

* Started to replace artefacts when files are pre-processed.

* Removed linker argument from linking step in all examples.

* Try to get jules to link.

* Fixed build_jules.

* Fixed other issues raised in reviews.

* Try to get jules to link.

* Fixed other issues raised in reviews.

* Simplify handling of X90 files by replacing the X90 with x90, meaning only one artefact set is involved when running PSyclone.

* Make OBJECT_ARCHIVES also a dict, migrate more code to replace/add files to the default build artefact collections.

* Fixed some examples.

* Fix flake8 error.

* Fixed failing tests.

* Support empty comments.

* Fix preprocessor to not unnecessary remove and add files that are already in the output directory.

* Allow find_soure_files to be called more than once by adding files (not replacing artefact).

* Updated lfric_common so that files created by configurator are written in build (not source).

* Use c_build_files instead of pragmad_c.

* Removed unnecessary str.

* Documented the new artefact set handling.

* Fixed typo.

* Make the PSyclone API configurable.

* Fixed formatting of documentation, properly used ArtefactSet names.

* Support .f and .F Fortran files.

* Removed setter for tool.is_available, which was only used for testing.

* #3 Fix documentation and coding style issues from review.

* Renamed Categories into Category.

* Minor coding style cleanup.

* Removed more unnecessary ().

* Re-added (invalid) grab_pre_build call.

* Fixed typo.

* Renamed set_default_vendor to set_default_compiler_suite.

* Renamed VendorTool to CompilerSuiteTool.

* Also accept a Path as exec_name specification for a tool.

* Move the check_available function into the base class.

* Fixed some types and documentation.

* Fix typing error.

* Added explanation for meta-compiler.

* Improved error handling and documentation.

* Replace mpiifort with mpifort to be a tiny bit more portable.

* Use classes to group tests for git/svn/fcm together.

* Fixed issue in get_transformation script, and moved script into lfric_common to remove code duplication.

* Code improvement as suggested by review.

* Fixed run config

* Added reference to ticket.

* Updated type information.

* More typing fixes.

* Fixed typing warnings.

* As requested by reviewer removed is_working_copy functionality.

* Issue a warning (which can be silenced) when a tool in a toolbox is replaced.

* Fixed flake8.

* Fixed flake8.

* Fixed failing test.

* Addressed issues raised in review.

* Removed now unnecessary operations.

* Updated some type information.

* Fixed all references to APIs to be consistent with PSyclone 2.5.

* Added api to the checksum computation.

* Fixed type information.

* Added test to verify that changing the api changes the checksum.

* Make compiler version a tuple of integers

* Update some tests to use tuple versions

* Explicitly test handling of bad version format

* Fix formatting

* Tidying up

* Make compiler raise an error for any invalid version string

Assume these compilers don't need to be hashed.
Saves dealing with empty tuples.

* Check compiler version string for compiler name

* Fix formatting

* Add compiler.get_version_string() method

Includes other cleanup from PR comments

* Add mpi and openmp settings to BuildConfig, made compiler MPI aware.

* Looks like the circular dependency has been fixed.

* Revert "Looks like the circular dependency has been fixed." ...
while it works with the tests, a real application still triggered it.

This reverts commit 150dc37.

* Don't even try to find a C compiler if no C files are to be compiled.

* Updated gitignore to ignore (recently renamed) documentation.

* Fixed failing test.

* Return from compile Fortran early if there are no files to compiles. Fixed coding style.

* Add MPI enables wrapper for intel and gnu compiler.

* Fixed test.

* Automatically add openmp flag to compiler and linker based on BuildConfig.

* Removed enforcement of keyword parameters, which is not supported in python 3.7.

* Fixed failing test.

* Support more than one tool of a given suite by sorting them.

* Use different version checkout for each compiler vendor with mixins

* Refactoring, remove unittest compiler class

* Fix some mypy errors

* Use 'Union' type hint to fix build checks

* Return run_version_command to base Compiler class

Provides default version command that can be overridden for other compilers.
Also fix some incorrect tests
Other tidying

* Add a missing type hint

* Remove inheritance from mixins and use protocol

* Simplify compiler inheritance

Mixins have static methods with unique names,
overrides only happen in concrete classes

* Simplify usage of compiler-specific parsing mixins.

* Test for missing mixin.

* Fixed test.

* Added more tests for invalid version numbers.

* Added more test cases for invalid version number, improved regex to work as expected.

* Fixed typo in test.

* Fixed test.

* Split tests into smaller individual ones, fixed missing asssert in test.

* Parameterised compiler version tests to also test wrapper.

* Added missing MPI parameter when getting the compiler.

* Fixed comments.

* Order parameters to be in same order for various compiler classes.

* Remove stray character

* Made mpi and openmp default to False in the BuildConfig constructor.

* Removed white space.

* Support compilers that do not support OpenMP.

* Added documentation for openmp parameter.

---------

Co-authored-by: Junwei Lyu <[email protected]>
Co-authored-by: Junwei Lyu <[email protected]>
Co-authored-by: Junwei Lyu <[email protected]>
Co-authored-by: Junwei Lyu <[email protected]>
Co-authored-by: jasonjunweilyu <[email protected]>
Co-authored-by: Luke Hoffmann <[email protected]>
Co-authored-by: Luke Hoffmann <[email protected]>
* #3 Converted svn and fcm to tools.

* #3 Fixed missing whitespace.

* Modified the documentation for writing a config with PSyclone

* Add config as a parameter for run_psyclone for the transformation_script to use;Updated the related functions and tests;
Changed the logic of the transformation_script examples

* #3 Replaced ar with tool object.

* #3 Added tests for ar.py.

* #3 Removed debug output.

* #3 Converted PSyclone to be a tool.

* #3 Removed debug print, fixed python 3.7 typing information.

* #3 Updated comments.

* Modified the get_optimisation_script function examples and updated the doc formatting

* #3 Add Rsync tool.

* #3 Removed now unused function.

* #3 Added test for rsync.

* #3 Fixed all mypy warnings about functions not checked.

* #3 Replace all mock-tests to use subprocess so the name of the executable is tested as well.

* #3 Remove duplicated flags.

* #3 Fixed changed order of linking.

* #3 Removed run_command function.

* #3 Fixed 3.8 typing error.

* #3 Fixed unused imports.

* #3 Move flags checksum into Flags, and remove now unused tools.py file.

* #3 Renamed newtools to tools.

* #3 Made custom function for all git functions called (instead of just calling run).

* #3 Updated and fixed comments.

* #3 Fixed errors in comments.

* Fixed minor errors in documentation.

* #3 Make it easier to create wrapper around standard compiler.

* #3 Added documentation for all tool related classes and their usage.

* #3 Added MISC category.

* Addressed reviewer's comments.

* Updated cli to properly use ToolBox etc, removing hard-coded gnu command linker option.

* Fixed mypy failures, including changes to import statement to avoid cyclic imports :(.

* #3 Fix circular import.

* Added #TODO so that this can be removed once fparser supports sentinels.

* Fix typing problems by ignoring fparser.

* Replaced more string names for artefacts with enums.

* Removed EXECUTABLES from constants.

* Moved Artefact class out of ArtefactStore and renamed it to ArtefactSet.

* Moved OBJECT_FILES from constants into ArtefactSet.

* Moved OBJECT_ARCHIVES from constants to ArtefactSet.

* Moved PRAGMAD_C from constants to ArtefactSet.

* Turned 'all_source' into an enum.

* Allow integer as revision.

* Fixed flake8 error.

* Removed specific functions to add/get fortran source files etc.

* Removed non-existing and unneccessary collections.

* Try to fix all run_configs.

* Fixed rebase issues.

* Added replace functionality to ArtefactStore, updated test_artefacts to cover all lines in that file.

* Started to replace artefacts when files are pre-processed.

* Removed linker argument from linking step in all examples.

* Try to get jules to link.

* Fixed build_jules.

* Fixed other issues raised in reviews.

* Try to get jules to link.

* Fixed other issues raised in reviews.

* Simplify handling of X90 files by replacing the X90 with x90, meaning only one artefact set is involved when running PSyclone.

* Make OBJECT_ARCHIVES also a dict, migrate more code to replace/add files to the default build artefact collections.

* Fixed some examples.

* Fix flake8 error.

* Fixed failing tests.

* Support empty comments.

* Fix preprocessor to not unnecessary remove and add files that are already in the output directory.

* Allow find_soure_files to be called more than once by adding files (not replacing artefact).

* Updated lfric_common so that files created by configurator are written in build (not source).

* Use c_build_files instead of pragmad_c.

* Removed unnecessary str.

* Documented the new artefact set handling.

* Fixed typo.

* Make the PSyclone API configurable.

* Fixed formatting of documentation, properly used ArtefactSet names.

* Support .f and .F Fortran files.

* Removed setter for tool.is_available, which was only used for testing.

* #3 Fix documentation and coding style issues from review.

* Renamed Categories into Category.

* Minor coding style cleanup.

* Removed more unnecessary ().

* Re-added (invalid) grab_pre_build call.

* Fixed typo.

* Renamed set_default_vendor to set_default_compiler_suite.

* Renamed VendorTool to CompilerSuiteTool.

* Also accept a Path as exec_name specification for a tool.

* Move the check_available function into the base class.

* Fixed some types and documentation.

* Fix typing error.

* Added explanation for meta-compiler.

* Improved error handling and documentation.

* Replace mpiifort with mpifort to be a tiny bit more portable.

* Use classes to group tests for git/svn/fcm together.

* Fixed issue in get_transformation script, and moved script into lfric_common to remove code duplication.

* Code improvement as suggested by review.

* Fixed run config

* Added reference to ticket.

* Updated type information.

* More typing fixes.

* Fixed typing warnings.

* As requested by reviewer removed is_working_copy functionality.

* Issue a warning (which can be silenced) when a tool in a toolbox is replaced.

* Fixed flake8.

* Fixed flake8.

* Fixed failing test.

* Addressed issues raised in review.

* Removed now unnecessary operations.

* Updated some type information.

* Fixed all references to APIs to be consistent with PSyclone 2.5.

* Added api to the checksum computation.

* Fixed type information.

* Added test to verify that changing the api changes the checksum.

* Make compiler version a tuple of integers

* Update some tests to use tuple versions

* Explicitly test handling of bad version format

* Fix formatting

* Tidying up

* Make compiler raise an error for any invalid version string

Assume these compilers don't need to be hashed.
Saves dealing with empty tuples.

* Check compiler version string for compiler name

* Fix formatting

* Add compiler.get_version_string() method

Includes other cleanup from PR comments

* Add mpi and openmp settings to BuildConfig, made compiler MPI aware.

* Looks like the circular dependency has been fixed.

* Revert "Looks like the circular dependency has been fixed." ...
while it works with the tests, a real application still triggered it.

This reverts commit 150dc37.

* Don't even try to find a C compiler if no C files are to be compiled.

* Updated gitignore to ignore (recently renamed) documentation.

* Fixed failing test.

* Return from compile Fortran early if there are no files to compiles. Fixed coding style.

* Add MPI enables wrapper for intel and gnu compiler.

* Fixed test.

* Automatically add openmp flag to compiler and linker based on BuildConfig.

* Removed enforcement of keyword parameters, which is not supported in python 3.7.

* Fixed failing test.

* Support more than one tool of a given suite by sorting them.

* Use different version checkout for each compiler vendor with mixins

* Refactoring, remove unittest compiler class

* Fix some mypy errors

* Use 'Union' type hint to fix build checks

* Added option to add flags to a tool.

* Introduce proper compiler wrapper, used this to implement properly wrapper MPI compiler.

* Fixed typo in types.

* Return run_version_command to base Compiler class

Provides default version command that can be overridden for other compilers.
Also fix some incorrect tests
Other tidying

* Add a missing type hint

* Added (somewhat stupid) 'test' to reach 100% coverage of PSyclone tool.

* Simplified MPI support in wrapper.

* More compiler wrapper coverage.

* Removed duplicated function.

* Removed debug print.

* Removed permanently changing compiler attributes, which can cause test failures later.

* More test for C compiler wrapper.

* More work on compiler wrapper tests.

* Fixed version and availability handling, added missing tests for 100% coverage.

* Fixed typing error.

* Try to fix python 3.7.

* Tried to fix failing tests.

* Remove inheritance from mixins and use protocol

* Simplify compiler inheritance

Mixins have static methods with unique names,
overrides only happen in concrete classes

* Updated wrapper and tests to handle error raised in get_version.

* Simplified regular expressions (now tests cover detection of version numbers with only a major version).

* Test for missing mixin.

* Use the parsing mixing from the compiler in a compiler wrapper.

* Use setattr instead of assignment to make mypy happy.

* Simplify usage of compiler-specific parsing mixins.

* Minor code cleanup.

* Updated documentation.

* Simplify usage of compiler-specific parsing mixins.

* Test for missing mixin.

* Fixed test.

* Added missing openmp_flag property to compiler wrapper.

* Don't use isinstance for consistency check, which does not work for CompilerWrappers.

* Fixed isinstance test for C compilation which doesn't work with a CompilerWrapper.

* Use a linker's compiler to determine MPI support. Removed mpi property from CompilerSuite.

* Added more tests for invalid version numbers.

* Added more test cases for invalid version number, improved regex to work as expected.

* Fixed typo in test.

* Fixed flake/mypy errors.

* Combine wrapper flags with flags from wrapped compiler.

* Made mypy happy.

* Fixed test.

* Split tests into smaller individual ones, fixed missing asssert in test.

* Parameterised compiler version tests to also test wrapper.

* Added missing MPI parameter when getting the compiler.

* Fixed comments.

* Order parameters to be in same order for various compiler classes.

* Remove stray character

* Added getter for wrapped compiler.

* Fixed small error that would prevent nested compiler wrappers from being used.

* Added a cast to make mypy happy.

* Fixed spelling mistake in option.

* Clarified documentation.

* Removed unnecessary functions in CompilerWrapper.

* Made mpi and openmp default to False in the BuildConfig constructor.

* Removed white space.

* Support compilers that do not support OpenMP.

* Added documentation for openmp parameter.

---------

Co-authored-by: Junwei Lyu <[email protected]>
Co-authored-by: Junwei Lyu <[email protected]>
Co-authored-by: jasonjunweilyu <[email protected]>
Co-authored-by: Luke Hoffmann <[email protected]>
Co-authored-by: Luke Hoffmann <[email protected]>
Copy link
Collaborator

@MatthewHambley MatthewHambley left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've repeated the issue I raised by e-mail so you can post your response here for posterity.

source/fab/steps/compile_c.py Show resolved Hide resolved
source/fab/steps/compile_fortran.py Show resolved Hide resolved
source/fab/steps/compile_fortran.py Show resolved Hide resolved
source/fab/tools/compiler.py Outdated Show resolved Hide resolved
source/fab/tools/compiler.py Outdated Show resolved Hide resolved
source/fab/tools/compiler_wrapper.py Show resolved Hide resolved
source/fab/tools/compiler_wrapper.py Show resolved Hide resolved
source/fab/tools/tool.py Show resolved Hide resolved
tests/unit_tests/tools/test_compiler_wrapper.py Outdated Show resolved Hide resolved
@hiker hiker requested a review from MatthewHambley December 3, 2024 07:05
Copy link
Collaborator

@MatthewHambley MatthewHambley left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lets move things forward a bit and get to the next change.

@MatthewHambley MatthewHambley merged commit e7ae9fe into main Dec 19, 2024
11 checks passed
@MatthewHambley MatthewHambley deleted the compiler_wrapper branch December 19, 2024 11:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants