- while working on NMODL + LLVM, we don't worry that much
    about Python bindings by default
  - so lets disable them by default

  * added NMODL_ENABLE_LLVM option to enable/disable
    llvm support in nmodl
  * LLVMHelper.cmake added to help with linking LLVM libraries
     - clang might need to use libstdc++ or libc++ linking
     - on BB5, using GCC with LLVM libraries is fine. But using
       clang results into lots of link error. Adding -stdlib=libstd++
       solves the issue
     - use check_cxx_source_compiles to find out which cxx flag is needed

 - added llvm dir under codegen where LLVM code generation
   work will live
 - llvm codegen visitor created that can be used as template
   for initial work
 - cmake adapted to enable llvm codegen based on CMake option
 - simple procedure.mod added that can be initial target for
   testing
 - new CLI option --llvm that runs LLVM codegen visitor
 - Enable CXX 14 because new LLVM versions require it

 - install llvm via brew
 - set LLV_DIR variable so that CMake can find llvm-config

  - print table with different build options, flags and paths
    used that can be helpful for debugging
  - fix git revision date for older git version
  - update INSTALL.md with correct brew paths for flex and bison

  - test/unit/codegen/llvm.cpp added for unit testing
    LLVM code generation visitor
  - ./bin/testcodegen binary can be used to launch
    LLVM codegen specific tests
  - multiple llvm_map_components_to_libnames removed
  - update procedure.mod with simple examples for IR generation

* Added LLVM code generation for `ProcedureBlock`.
* Added code generation routines for double, integer and
   boolean variable types.
* Added binary and unary operator code generation:
     - Supported binary operators: +, -, *, /.
     - Supported unary operators: -.
     - Assignment (=) is also supported.
* Added regex matching unit tests for LLVM code generation.
* Fixed Travis CI/builds.

fixes #451, fixes #452, fixes #456

Co-authored-by: Pramod Kumbhar <[email protected]>

* LLVM code generation for `FunctionBlock` is now supported.
* Terminators in function or procedure blocks are enforced:
      - Every procedure must have `ret void` instruction.
      - Every function returns a double, specified by `ret_<function_name>`.
* For local symbol table, code generation now uses LLVM's builtin
`llvm::ValueSymbolTable`.


fixes #454, fixes #469

* Add option to run LLVM optimisation passes
  - update CLI argument from --llvm to llvm --ir --opt
  - --ir runs CodegenLLVMVicitor and emits LLVM IR
  - if --opt is passed, we run basic LLVM optimisation passes
  - update simple test to check optimisation passes
* Add function example in procedure.mod
* Add test for LLVM optimisation passes and dead code removal

This patch adds support for function call code generation, particularly:

- User-defined procedures and functions can now lowered to LLVM IR.
- A framework for external method calls (e.g. sin, exp, etc.) has been created, currently `exp` and `pow` are supported.
- Corresponding tests added.

fixes #472

LLVM code generation for `IndexedName`s.

- Added code generation for initialising arrays in LOCAL blocks (with both integer constants and macros).
- Added support for indexing arrays.

fixes #467

NMODL AST needs various transformation to generate C++
code or LLVM IR. This PR is begining of AST transformations
to simplify code generation backends.

* New CodegenLLVMHelperVisitor to perform various AST
  transformations to simplify code generation for various
  backends and simulators.
* CodegenLLVMHelperVisitor is currently limited to LLVM
  backend to simplify initial implementation and keep
  C++ based backends working.
* CodegenLLVMHelperVisitor now handles FUNCTIONS and
  PROCEDURES blocks
  -  Replace LocalListStatement with CodegenVarStatement
  - Added new AST types for code generation
    - CodegenVar to represent variable used for code generation
    - CodegenVarType to represent codegen variable
    - CodegenVarListStatement to represent list of CodegenVar
    - CodegenStruct will be used in future to represent struct
      like NrnThread or Mechanism class

See #474

* Added new BinaryOp for += and -=
  * Added string_to_binaryop function
  * Added Void node type to represent void return type
  * Added CodegenAtomicStatement for ion write statements
  * llvm helper started handling visit_nrn_state_block
    - NrnStateBlock is being converted into CodegenFunction
    - for loop body with solution blocks created
    - voltage and node index initialization code added
    - read and write ion statements are handled
  * Some of the functions are now moved into CodegenInfo

Co-authored-by: Ioannis Magkanaris <[email protected]>

This commit introduces a functionality to execute functions from MOD file via LLVM jit.

For that, there is now:
- `JITDriver` class that, given a LLVM IR module, set ups the JIT compiler and is able to look up a function and execute it.
- `Runner` class that wraps around JIT driver. It helps to initialise JIT with LLVM IR module only once, and then run multiple functions from it. 

To execute functions, `nmodl_llvm_runner` executable is used. It takes a single mod file and a specified entry-point function, and runs it via LLVM code generation pipeline and JIT driver. Only functions with double result types are supported at the moment.

For example, for MOD file `foo.mod`:
```
FUNCTION one() {
    one = 1
}

FUNCTION bar() {
    bar = one() + exp(1)
}
```
running `nmodl_llvm_runner -f foo.mod -e bar` gives
```
Result: 3.718282
```

Tests for execution of generated  IR have been added as well.

fixes #482

Co-authored-by: Pramod Kumbhar <[email protected]>

* Added more bin ops and refactored code
   - Now, there are code generation functions for all comparison
      and logical operators.
   - Code generation functions are now split based on the expression "type"
      (assignment, arithmetic, comparison, logical). Moreover, the lhs and rhs
      expression results can be both double and integer. This is important for
      control flow code generation and for the new AST node CodegenVarType.
* Added support for NOT op
* Added default type flag to switch between float and double
* Added tests for single precision
* Renames LLVM test file to codegen_llvm_ir.cpp to follow convention.
* NOTE : Tests for new operators will be added when the first control
                flow node (most likely FOR node) will land.

fixes #453

* visit_statement_block of all FUNCTION and PROCEDURE
    blocks was called resulting in changing LOCAL
    statement to DOUBLE statement
  * As statement block doesn't need to be visited for this
    purpose, rename function to convert_local_statement
  * Call convert_local_statement only when required i.e.
    only when codegen function creation time.

fixes #491

* Handle CodegenVarType type in JSON printer
  - As AstNodeType is enum type and node itself,
    we need to print that explicitly
* Indent json visitor jinja template
 - initially template was not indented as code generated
   was not looking good
 - now all generated code is automatically clang-formatted
   so it's less of a concern. Readability is important.

fixes #493

* LLVM Helper visitor now can return a vector of `CodegenFunction`s.
* LLVM Helper visitor has been integrated into LLVM visitor:
   - The type of variables is still double by default, but can also be inferred from `CodegenVarType` node.
   - Procedure's return type changed to int (so that error codes can be returned in the future). 
   - New visitor functions added: for `CodegenReturn`, `CodegenFunction`, `CodegenVarList` and `CodegenVarType`.

* Added a new code generation function for conditional statements (`if`, `else if`, `else` and their nested variations).
* Added tests for the new code generation:
   - IR unit tests.
   - Execution tests.
* Fixed FP and integer comparison ordering in macros.

fixes #468

Added error handling when a non-scope value is looked up. Before, such a lookup would yield a nullptr, therefore leading to a segmentation fault. This PR adds a lookup function that wraps around value symbol lookup, and throws an error with a message if nullptr is returned.

Added support for WHILE statement code generation. Corresponding tests for IR generation and execution were also added.

Additional visitor for StatementBlock was added to reduce code duplication.

fixes #500

* Moved info related function to codegen_info
  - Moved get_float_variables, codegen_int_variables,
     codegen_global_variables, codegen_shadow_variables
     into CodegenHelper
  - Move small utility functions from CodegenCVisitor to codeged_utils
* Add proper variables to the mech_Instance
* Adding LLVMStructBlock
* Added test and visitor
* Fix llvm codegen tests with x[0-9].*
* Fixes after rebasing llvm branch on master (15.9.2022)

- Added support for string function arguments. These are
   converted into global `i8` array values.
- Added support for `printf` function call with variable number 
   of arguments.
- Refactored function/procedure call argument processing into
   a separate function.

fixes #510

* Move code gen specific InstanceStruct node to codegen.yaml
  - nmodl.yaml file is more for language constructs
  - InstanceStruct is specific for code generation and hence
    move it to codegen.yaml
* Update CI scripts
* fix cmake-format with v==0.6.13

 - instance structure now contains all global variables
 - instance structure now contains index variables for ions
 - nrn_state kernel now has all variables converted to instance
 - InstanceVarHelper added to query variable and it's location
* Support for codegen variable with type
* Add nmodl_to_json helper added in main.cpp
* Added --vector-width CLI option
* Add instance struct argument to nrn_state_hh
* Add comments as TODOs to support LLVM IR generation

Note that this commit and next commit (Part II) are required to
make LLVM IR code generation working. Vector IR generation is
working except indirect indexes. See comment in #531.

  - remove undefined visit_codegen_instance_var
  - Improved member creation for instance struct
  - Instance struct type generation for kernel arguments
  - Proper integration of instance struct
  - Added scalar code generation for the kernel
  - Removed instance test since it is not created explicitly anymore
  - Fixed ordering for precision and width in LLVM Visitor
  - Added vector induction variable
  - Vectorised code for compute with direct loads fully functional
  - Instance naming fixed
  - (LLVM IR) Fixed compute vector code generation types
  -  refactoring : improve coversion of double to int for
    the loop expressions

This PR adds a unit test to check LLVM instructions generated for
the scalar kernel, particularly:

- FOR loop blocks.

- Induction variable increments and comparisons.

- Correct loads through GEPs from the struct.

Test for vectorised code generation would be added in a separate
PR or when full vectorisation support (indirect indexing) would
land.

Improved index code generation within the LLVM pipeline.
The following issues were addressed:

Array indices are i64 per LLVM's addressing convention.
This means that if the value is not a constant, an additional
sext instruction must be created.

Bounds check is removed since it requires a certain analysis
on the index value. This can be addressed in a separate PR.

`IndexedName` code generation is separated into 2 functions
The first, `get_array_length()` is responsible for array initialisation,
the second, `get_array_index()`, for indexing. In latter case, we
support the following cases:
```
...
// Indexing with an integer constant
k[0] = ...

// Indexing with an integer expression
k[10 - 10] 

// Indexing with a `Name` AST node that is an integer
// (in our case a FOR loop induction variable or a variable
// with `CodegenVarType` == `Integer`
k[id] = ...      
k[ena_id] = ...
```
Note that the case:
```
// id := loop integer induction variable
k[id + 1] = ... 
```
is not supported for 2 reasons:

On the AST level, as per #545 the expression would
contain a Name and not VarName node that fails the
code generation.

The case only arises in the kernel functions like state_update,
where indexing is "artificially" created with indexing by a Name
only.

fixes #541

* CodegenLLVMHelperVisitor improved without hardcoded parameters
* Added get_instance_struct_ptr to get instance structure for variable information
* test/unit/codegen/codegen_data_helper.cpp : first draft implementation
   of codegen data helper
* Added test for typecasting to the proper struct type

Co-authored-by: Pramod Kumbhar <[email protected]>

* Update hpc-coding-convention (#836)
* Run clang-format with clang-format-13
* Fix gitlab ci NMODL spack variants

Co-authored-by: Nicolas Cornu <[email protected]>

… calls (#835)

Added an LLVM pass that replaces math intrinsics
with calls to math library. In particular:

* Functionality of replacement with SIMD functions is factored
out into a separate file and LLVM version dependencies are
dropped (LLVM 13 is already used anyway).

* A pass to replace intrinsics with libdevice calls when targeting
CUDA platforms has been added. So far only `exp` and `pow` are
 supported (single and double precision).

* Added a test to check the replacement

Co-authored-by: Ioannis Magkanaris <[email protected]>

- Created CodegenDriver class to factor out ast preparation
- Created pybind wrappers for Jit and Codegen configuration options
- Updated benchmark runner to return runtime stats
- Return benchmark results to python
- Addressed @iomaganaris' comments.
- Add a PyJIT integration test
* Re-enable python bindings by default

* Fixes issue with debug printing of the various stages of code generation in files
* Small fix and comment addition

* Support for Breakpoint block (nrn_cur) for code generation
* similar to DERIVATIVE (nrn_state), handle BREAKPOINT (nrn_cur)
   blocks with AST level transformation
* Move common code from CodegenCVisitor to CodegenInfo
* Add tests

fixes #644

Co-authored-by: George Mitenkov <[email protected]>

- Added CUDADriver to compile LLVM IR string generated from CodegenLLVMVisitor to PTX string and then execute it using CUDA API
- Ability to select the compilation GPU architecture and then set the proper GPU architecture based on the GPU that is going to be used
- Link `libdevice` math library with GPU LLVM module
- Handles kernel and wrapper functions attributes properly for GPU execution (wrapper function is `kernel` and kernel attribute is `device`)
- Small fixes in InstanceStruct declaration and setup to allocate the pointer variables properly, including the shadow variables
- Adds tests in the CI that run small benchmarks in CPU and GPU on BB5
- Adds replacement of `log` math function for SLEEF and libdevice, `pow` and `fabs` for libdevice
- Adds GPU execution ability in PyJIT
- Small improvement in PyJIT benchmark python script to handle arguments and GPU execution
- Separated benchmark info from benchmark driver
- Added hh and expsyn mod files in benchmarking tests

1. Helper visitor generates atomic statements with += and -=
once again.
2. LLVM visitor now knows how to lower atomic statements for
CPUs (trivially) and GPUs.
3. A corresponding IR  test was added. `expsyn` test on GPU
enabled

Co-authored-by: Ioannis Magkanaris <[email protected]>

* Fixed float AST in helper
* Fixed InstanceStruct test to generate struct with doubles

#868)

* Use named structs instead of functions, pass instances to solvers.
* Goes with BlueBrain/CoreNeuron#809.
* Add fmt_line, fmt_start_block and restart_block methods to CodePrinter.
* gitlab-ci: support CVF_BRANCH variable.

Co-authored-by: Olli Lupton <[email protected]>

* Makes sure that llvm branch compiles with LLVM codegen and Python bindings disabled or enabled
* Only compile benchmark folder when LLVM backend is enabled
* Selectively enables options of Python Codegen and Python JIT Driver based on LLVM backend being enabled or disabled
* Fixes the order of linkage of static variables for the generation of nmodl binary

* Squash commit when rebasing with master

* [CP-870] fixup! CI errors with sympy 1.9 and 1.10 (#870)

* fixup! CI errors with sympy 1.9 and 1.10

- bin is a function in python: renamed variable in test to not get this error
- sympy now can solve z'=a/z+b/z/z. It is getting hard to find simple equations
that it cannot solve.

Warning: this is a downgrade of the testing capabilities. I could not find
an ode with simple functions (no sin, exp, log etc. Only simple multiplications
and additions) that cannot be solved by sympy. Best case the test hangs waiting
for sympy to solve a very long equation. For this reason, with this PR, the
fact that the code should return the equation untouched if it cannot be processed
is untested

* fixup! clang-format

* remove limit for sympy 1.9

* Relax constraints of GPU nodes to allocate in gitlab CI

Co-authored-by: Alessandro Cattabiani <[email protected]>
Co-authored-by: Alessandro Cattabiani <[email protected]>

* Adds code generation functionality for atomic writes on SIMD platforms
* Adds both execution and IR tests
* Refactors common GPU/SIMD atomics code

Co-authored-by: Ioannis Magkanaris <[email protected]>

- Changed g RANGE variable name to g_var to avoid issue with confusing g local variable for current with the instance variable g
- Corrected expected code

* New `Annotator` class to specialise how NMODL compute kernels (llvm::Functions) are annotated
* New `ReplacePass` class that replaces the standard math function calls of the kernels with the optimised math libraries passed to NMODL
* Replace kernel wrappers with generation of the compute functions with a `void*` parameter which is then casted to the proper struct type internally by the compute function
* Identify compute kernels `CodegenFunction` with a new `is_kernel` flag
  - `is_kernel` is mapped to LLVM metadata node `nmodl.compute-kernel` that allows any to easily identify compute kernels in LLVM IR
* Removes restriction for auto-vectorising loops by external compiler using certain LLVM IR loop metadata when the vector length is set to 1 in NMODL

…ug symbols options (#912)

* Added options for fast math and debug symbols on CodegenLLVMVisitor instantiated by NMODL PyJIT
* Disable debug symbols in GPU PyJIT test

Co-authored-by: Ioannis Magkanaris <[email protected]>