Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

new backend #140

Merged
merged 179 commits into from
Sep 4, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
179 commits
Select commit Hold shift + click to select a range
359314c
Initial commit
hexaeder Jul 23, 2021
9054999
Files generated by PkgTemplates
hexaeder Jul 23, 2021
6711aea
first draft
hexaeder Jul 24, 2021
6962af6
start edge coloring algorithm
hexaeder Jul 24, 2021
318cff1
give up on coloring algo. in fav. of greedy algo
hexaeder Jul 25, 2021
51919ea
construction works
hexaeder Jul 25, 2021
c7839cb
coreloop works
hexaeder Jul 25, 2021
0c3ca76
unroll loop, allocation free inhomogeneous network
hexaeder Jul 25, 2021
ecf5ffc
support coupling types
hexaeder Jul 26, 2021
b2a582e
several tweaks
hexaeder Jul 26, 2021
fdaac8f
work on mt
hexaeder Jul 26, 2021
40eaa76
clean up coreloop
hexaeder Jul 26, 2021
3091979
baseline bm results of ND
hexaeder Jul 26, 2021
09e19f9
CompatHelper: add new compat entry for "Unrolled" at version "0.1"
github-actions[bot] Jul 27, 2021
046f0e7
fix sync for mt
hexaeder Jul 27, 2021
38602cd
fix auto accdim in constructor
hexaeder Jul 28, 2021
2ded9d7
compare to nd for differen julia verions
hexaeder Jul 28, 2021
715165b
Merge pull request #3 from hexaeder/compathelper/new_version/2021-07-…
hexaeder Jul 29, 2021
648373c
updates
hexaeder Aug 7, 2021
cf0cce5
reenable MT
hexaeder Aug 9, 2021
ac0a0ca
use Arrays instead of Tuples
hexaeder Aug 9, 2021
e5d4611
experiemnt with floops
hexaeder Aug 13, 2021
dbfd481
introduce execution typeparameter
hexaeder Aug 13, 2021
4fbd833
update benchmark scripts
hexaeder Aug 13, 2021
88747d5
improve and simplify edge coloring
hexaeder Aug 14, 2021
021343e
cleanup
hexaeder Aug 16, 2021
0cbf25a
Merge branch 'main' of github.com:hexaeder/NDPrototype.jl
hexaeder Aug 16, 2021
78d31ef
update packages
hexaeder Apr 17, 2024
1d0e0ad
use ArgCheck and prealloc tools
hexaeder Apr 17, 2024
349cbcd
update data structures for scatter/gather design
hexaeder Apr 17, 2024
2a3581b
wip, work on gather and scatter methods
hexaeder Apr 19, 2024
e7f93c8
format
hexaeder Apr 26, 2024
a9fdf71
renam layer->nl
hexaeder Apr 26, 2024
2c473b4
edge coreloop
hexaeder Apr 26, 2024
1cb2574
vertex coreloop
hexaeder Apr 26, 2024
167b2bc
further improvements
hexaeder Apr 29, 2024
7a67d84
implement new aggregation scheme
hexaeder Apr 29, 2024
01ae577
compare to old nd and sequential accumulator
hexaeder Apr 29, 2024
75a4c43
separate common aggregator structure to own struct
hexaeder Apr 30, 2024
fdc65ae
add buffered sequential execution strategy
hexaeder Apr 30, 2024
973e6c2
add polyester aggregator
hexaeder Apr 30, 2024
4a3c775
introduce BatchStride Objects
hexaeder May 1, 2024
8154ea1
fix line endings and format files
hexaeder May 1, 2024
05f4ec9
minimal example works on metal
hexaeder May 1, 2024
5ea668c
wip
hexaeder May 17, 2024
b3fc9b6
wip push to cont on new laptop
hexaeder May 23, 2024
4fcb20e
wip
hexaeder May 23, 2024
c2c4a32
basic tests running
hexaeder May 24, 2024
3022c86
add mass matrix construction
hexaeder May 24, 2024
bdb1b71
start work on symbolic indexing
hexaeder May 27, 2024
431ed7c
basic implementaion of symbolic indexing interface
hexaeder May 27, 2024
590d11b
rename types
hexaeder May 27, 2024
e0782a5
fix tests
hexaeder May 27, 2024
40a51eb
access multiple observables instantanious
hexaeder May 27, 2024
d25fe2b
fix order of symbols/parameters
hexaeder May 28, 2024
800da9f
indexing without wrapping in ODEFunction
hexaeder May 28, 2024
d65b1ec
add proxy functiosn for state and parameter indexing
hexaeder May 28, 2024
caf2cd1
improve show methods
hexaeder May 28, 2024
a641358
work on symbolic indexes
hexaeder May 28, 2024
576a86d
make colon indices work
hexaeder May 30, 2024
66ff95b
add broadcast interface
hexaeder May 30, 2024
c081e21
improve type stability
hexaeder May 30, 2024
12eb7a4
fix type stablity and performance
hexaeder May 30, 2024
2e56a64
make :t independent variable
hexaeder May 30, 2024
023f4de
clean up repo and deps
hexaeder May 30, 2024
19c2da8
move to separat test env
hexaeder May 31, 2024
676552a
Merge remote-tracking branch 'prototype/main' into newbackend
hexaeder May 31, 2024
76b794d
cleanup tests
hexaeder May 31, 2024
5aeb138
improve symbolic indexing, intruduce NWParameter object
hexaeder Jun 3, 2024
feeaff4
some work on old autodiff tests
hexaeder Jun 3, 2024
96b9c91
further work
hexaeder Jun 3, 2024
1938421
move diffusion test
hexaeder Jun 3, 2024
b3a3c6a
move inhomogeneous tests
hexaeder Jun 3, 2024
787f1c3
move massmatrix tests
hexaeder Jun 3, 2024
1c93c28
updates
hexaeder Jun 4, 2024
3b3a582
delete examples which are duplicated in docs
hexaeder Jun 4, 2024
93fcc46
improve benchmarking
hexaeder Jun 5, 2024
b213029
further improved benchmarking
hexaeder Jun 5, 2024
394d8be
updated benchmarks
hexaeder Jun 6, 2024
c233275
add benchmark CI
hexaeder Jun 6, 2024
06d9d35
improve bm printing
hexaeder Jun 6, 2024
728be8a
Merge pull request #1 from hexaeder/newbackend
hexaeder Jun 6, 2024
e1f5403
transfered first example
hexaeder Jun 6, 2024
387b8a5
mv directed and weighted graphs example
hexaeder Jun 13, 2024
1b7c839
rework keyword construction of component functions
hexaeder Jun 13, 2024
51f6dc1
allow syms or number as arguments
hexaeder Jun 13, 2024
02e5694
move indexing helpers to correct file
hexaeder Jun 13, 2024
73208ec
add new methods to generate symbolic idxs
hexaeder Jun 14, 2024
1f0a771
restrict print length of NWState and NWParameters
hexaeder Jun 14, 2024
237ebbf
check for oob acces in component constructor
hexaeder Jun 14, 2024
ddfc2a3
remove comment
hexaeder Jun 17, 2024
babeaf5
mv examples and fix FitzHugh-Nagumo example
hexaeder Jun 17, 2024
df639a5
prepare port of heterogenous example
hexaeder Jun 17, 2024
5f9419c
update chek functions
hexaeder Jun 17, 2024
6e4ddc2
nicer show methods
hexaeder Jun 17, 2024
c04ef20
improve symbolic indexing
hexaeder Jun 17, 2024
f3f62c5
allow :s=>1 defaults in component constructor
hexaeder Jun 17, 2024
91d1493
default to signaling NaN instead of nothing
hexaeder Jun 17, 2024
5efb10e
better show
hexaeder Jun 17, 2024
8b6c8df
adapt heterogenous example
hexaeder Jun 17, 2024
3d5dfba
mv SDE example
hexaeder Jun 18, 2024
71be856
add find_fixpoint methods
hexaeder Jun 18, 2024
8523f96
adapt StochasticSystem example
hexaeder Jun 18, 2024
3d02097
fix tests
hexaeder Jun 18, 2024
9fea366
mv delay example
hexaeder Jun 18, 2024
527f8b3
test docs/examples in runtest
hexaeder Jun 18, 2024
5e5dec2
compere execution style outputs
hexaeder Jun 19, 2024
6d2d65b
skip delay example in tests
hexaeder Jun 19, 2024
79f6462
revert SSI version (was yanked)
hexaeder Jun 20, 2024
5158b89
mv cascading failure example
hexaeder Jun 20, 2024
24dc9e7
allow for single symbols/pairs in syms/psysm
hexaeder Jun 20, 2024
1a9852d
allow more flexible NWPara/State construction
hexaeder Jun 20, 2024
c5e2f15
rename filelocal util function
hexaeder Jun 20, 2024
6e21426
adapt cascading failure example
hexaeder Jun 20, 2024
b30dfd8
work on docs
hexaeder Jun 20, 2024
4c98db9
include some examples
hexaeder Jun 20, 2024
22259b3
fix examples in docs
hexaeder Jun 21, 2024
f7afe05
push previews
hexaeder Jun 21, 2024
849e818
Merge pull request #4 from hexaeder/mvexamples
hexaeder Jun 21, 2024
08f6f8f
update CI according to PkgTemplates.jl
hexaeder Jun 21, 2024
0bf85bd
add styled strings dependency
hexaeder Jun 21, 2024
6afe6a9
extend coreloop to handle s and u differently
hexaeder Jun 21, 2024
38a82a1
force specialisation and encurage inlining
hexaeder Jun 21, 2024
7e55992
remove some @inline
hexaeder Jun 21, 2024
3352b47
more type specialization in KAExecution
hexaeder Jun 21, 2024
5135642
attempt to fix StyledStrings on 1.10 and 1.11
hexaeder Jun 24, 2024
14e26a1
remove anotations on julia < 1.11
hexaeder Jun 24, 2024
d497169
add threaded execution model
hexaeder Jul 8, 2024
4d1ec7e
improve benchmarks
hexaeder Jul 8, 2024
8140678
fix benchmark ci
hexaeder Jul 8, 2024
c78a3d5
attempt to fix tests
hexaeder Jul 8, 2024
2717594
extend benchmarks
hexaeder Jul 8, 2024
b9a474f
reduce bm size
hexaeder Jul 8, 2024
6b23bbc
fix benchmark comment
hexaeder Jul 9, 2024
5b275c8
unroll by recursion on tuple
hexaeder Jul 9, 2024
8c022c7
update gitignore
hexaeder Jul 9, 2024
2c322b4
fixup! fix benchmark comment
hexaeder Jul 9, 2024
534497e
revert introduction of s
hexaeder Jul 9, 2024
c1910fb
Merge pull request #5 from hexaeder/split_dyn_static
hexaeder Jul 9, 2024
a3c37e0
mv functions back to benchmark file
hexaeder Jul 10, 2024
2e4b6a3
make dispatch types concrete
hexaeder Jul 10, 2024
6eacc6c
unify comptype and compT
hexaeder Jul 10, 2024
408147d
rename compT to dispatchT
hexaeder Jul 11, 2024
6f18d9f
test GPU if CUDA avialable
hexaeder Jul 11, 2024
862a59d
compare Thrs-Seq against old parallel=true
hexaeder Jul 11, 2024
6139590
Merge pull request #7 from hexaeder/fixgpu
hexaeder Jul 11, 2024
13ff3bd
get u stacked for faster observables
hexaeder Jul 15, 2024
ad46592
reverse order of aggregation
hexaeder Jul 16, 2024
b7dc2a8
improve benchmarking
hexaeder Jul 17, 2024
f3fbcdb
improve aggregator setup time
hexaeder Jul 17, 2024
8847930
add SparseAggregator
hexaeder Jul 17, 2024
e09396b
improve benchmarks
hexaeder Jul 17, 2024
05c6a3d
rollback seq aggregator
hexaeder Jul 18, 2024
118b081
test sparse aggregation scheme
hexaeder Jul 18, 2024
d163363
Merge pull request #8 from hexaeder/getstaticbuf
hexaeder Jul 18, 2024
11529fa
fix gpu test
hexaeder Jul 18, 2024
46ac872
Merge branch 'main' into newbackend
hexaeder Jul 19, 2024
2dd96fa
ODEVertex from ODESystem
hexaeder Jul 24, 2024
412d318
StaticEdge from ODESystem
hexaeder Jul 24, 2024
24ec52e
use name for component
hexaeder Jul 25, 2024
0e084a3
bump pkg versions
hexaeder Aug 5, 2024
83f5628
automaticially test all CUDA compatible executions
hexaeder Aug 5, 2024
f082e4f
test package quality with Aqua and ExplicitImports
hexaeder Aug 5, 2024
b86eec7
Merge remote-tracking branch 'origin/main' into hw/mtk_integration
hexaeder Aug 5, 2024
80c1942
fix version specifier
hexaeder Aug 5, 2024
67db0f9
Merge branch 'main' into hw/mtk_integration
hexaeder Aug 5, 2024
44a3e37
reduce tolerance on test
hexaeder Aug 5, 2024
2a82b57
include MTK tests
hexaeder Aug 5, 2024
dff9ba5
Merge pull request #9 from hexaeder/hw/mtk_integration
hexaeder Aug 5, 2024
0eb6e87
start testing different AD backends
hexaeder Aug 8, 2024
59f71b8
add simple jac and paramjac tests for ForwardDiff and ReverseDiff
hexaeder Aug 12, 2024
c21b459
Merge pull request #10 from hexaeder/hw/adtests
hexaeder Aug 12, 2024
1ce954e
rm unparsable scratch
hexaeder Aug 8, 2024
82f6bf0
adjust styling (not enforce (yet))
hexaeder Aug 12, 2024
b9d7d7b
implement parameter timeseries interface
hexaeder Aug 13, 2024
97f9fef
cleanup
hexaeder Sep 3, 2024
15355f1
rais compat of styled strings
hexaeder Sep 4, 2024
b39a117
try fix tests on 1.10
hexaeder Sep 4, 2024
610645e
improve MTK ext
hexaeder Sep 4, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
105 changes: 42 additions & 63 deletions .JuliaFormatter.toml
Original file line number Diff line number Diff line change
@@ -1,74 +1,53 @@
# base style
# full list: https://domluna.github.io/JuliaFormatter.jl/dev/#Formatting-Options
# style = "sciml"
style = "yas"

# number of spaces
indent = 4

# maximum lenght of line
margin = 120

# replace for i = [1:2] or i ∈ [1:2] with for i in [1:2]
always_for_in = true

# if true, add whitespace to typedef between , and <:
normalize_line_endings = "unix"
whitespace_in_kwargs = false
whitespace_typedefs = false

# f true, whitespace is added for binary operations in indices. Make this true
# if you prefer arr[a + b] to arr[a+b]. Additionally, if there's a colon :
# involved, parenthesis will be added to the LHS and RHS.
whitespace_ops_in_indices = false

# If true, superflous newlines will be removed. For example:
format_markdown = false
format_docstrings = true
join_lines_based_on_source = true
remove_extra_newlines = false

# import A, B, C -> using A:A, Using B:B ...
import_to_using = false

# transform pipes |> to function calls
pipe_to_function_call = false

# transform f(x) = x definitons do full function... end blocks if line limit is exeded
short_to_long_function_def = true

# allways put return keyword at the end of functions
always_use_return = false

# (; a = 1) -> (; a=1)
whitespace_in_kwargs = false

# add ::Any to untyped fields
annotate_untyped_fields_with_any = false

# format docstrings with the same rules as the codebase
format_docstrings = true

# f the conditional E ? A : B exceeds the maximum margin converts it into the
# equivalent if block:
conditional_to_if = true

# One of "unix" (normalize all \r\n to \n), "windows" (normalize all \n to
# \r\n), "auto" (automatically choose based on which line ending is more common
# in the file).
normalize_line_endings = "auto"

# comma after last element in multi line list?
# trailing_comma = "nothing"

# if there are linebreaks in argument lists in the source keep them like this
join_lines_based_on_source = true

# When set to true, submodule(s) appearing in the same file will be indented.
indent_submodule = false

# shot detials related to formatting
verbose = true

# also format markdown files?
format_markdown = true

# alignment rules
# foo = 1
# foobar = 2
# stays like this
align_assignment = true
align_struct_field = true
align_conditional = true
align_pair_arrow = true
align_matrix = true

# ignore = ["docs/examples", "oldexamples", "test/oldtests"]

# indent = 4,
# margin = 92,
# always_for_in = false,
# for_in_replacement = "in",
# whitespace_typedefs = false,
# whitespace_ops_in_indices = false,
# remove_extra_newlines = false,
# import_to_using = false,
# pipe_to_function_call = false,
# short_to_long_function_def = false,
# long_to_short_function_def = false,
# always_use_return = false,
# whitespace_in_kwargs = true,
# annotate_untyped_fields_with_any = true,
# format_docstrings = false,
# align_struct_field = false,
# align_conditional = false,
# align_assignment = false,
# align_pair_arrow = false,
# conditional_to_if = false,
# normalize_line_endings = "auto",
# align_matrix = false,
# trailing_comma = false,
# trailing_zero = true,
# indent_submodule = false,
# separate_kwargs_with_semicolon = false,
# surround_whereop_typeparameters = true,
# variable_call_indent = []
# short_circuit_to_if = false,
2 changes: 1 addition & 1 deletion .github/workflows/CompatHelper.yml
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
name: CompatHelper
on:
schedule:
- cron: '00 00 * * *'
- cron: 0 0 * * *
workflow_dispatch:
jobs:
CompatHelper:
Expand Down
36 changes: 26 additions & 10 deletions .github/workflows/Documentation.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -6,19 +6,35 @@ on:
- main
tags: '*'
pull_request:

workflow_dispatch:
jobs:
build:
runs-on: ubuntu-latest
permissions:
actions: write # needed to allow julia-actions/cache to proactively delete old caches that it has created
contents: write
statuses: write
steps:
- uses: actions/checkout@v2
- uses: julia-actions/setup-julia@latest
- uses: actions/checkout@v4
- uses: julia-actions/setup-julia@v2
with:
version: 1.6
- name: Install dependencies
run: julia --project=docs/ -e 'using Pkg; Pkg.develop(PackageSpec(path=pwd())); Pkg.instantiate()'
- name: Build and deploy
version: 1
- uses: julia-actions/cache@v2
- name: Configure doc environment
shell: julia --project=docs --color=yes {0}
run: |
using Pkg
Pkg.develop(PackageSpec(path=pwd()))
Pkg.instantiate()
- uses: julia-actions/julia-buildpkg@v1
- uses: julia-actions/julia-docdeploy@v1
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} # For authentication with GitHub Actions token
DOCUMENTER_KEY: ${{ secrets.DOCUMENTER_KEY }} # For authentication with SSH deploy key
run: julia --project=docs/ docs/make.jl
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
DOCUMENTER_KEY: ${{ secrets.DOCUMENTER_KEY }}
- name: Run doctests
shell: julia --project=docs --color=yes {0}
run: |
using Documenter: DocMeta, doctest
using NetworkDynamics
DocMeta.setdocmeta!(NetworkDynamics, :DocTestSetup, :(using NetworkDynamics); recursive=true)
doctest(NetworkDynamics)
48 changes: 48 additions & 0 deletions .github/workflows/benchmark.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
name: Benchmark a pull request

on:
pull_request:

permissions:
pull-requests: write
actions: write
contents: read

jobs:
run_benchmarks:
runs-on: ubuntu-latest

steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0 # fetch all history
- uses: julia-actions/setup-julia@v2
- uses: julia-actions/cache@v2
- uses: julia-actions/julia-buildpkg@v1
- name: Run benchmarks
env:
JULIA_NUM_THREADS: auto
working-directory: benchmark
run: |
julia --color=yes ./run_benchmarks.jl -t ${{github.event.pull_request.head.ref}} -b ${{github.event.pull_request.base.ref}} --no-data-export --prefix ""
echo '### Benchmark Results' > body.md
echo '' >> body.md
echo '```' >> body.md
cat comparison.txt >> body.md
echo '```' >> body.md
echo '' >> body.md
- name: Upload plot
uses: actions/upload-artifact@v4
id: artifact-upload-step
with:
name: comparison.pdf
path: benchmark/comparison.pdf
- name: Link to graphic
run: |
echo '${{ steps.artifact-upload-step.outputs.artifact-url }}' >> benchmark/body.md
- uses: int128/hide-comment-action@v1 # hide all previous comments
- name: Comment on PR
uses: peter-evans/create-or-update-comment@v3
with:
issue-number: ${{ github.event.pull_request.number }}
body-path: benchmark/body.md
47 changes: 22 additions & 25 deletions .github/workflows/tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,52 +4,49 @@ on:
branches:
- main
pull_request:
workflow_dispatch:
jobs:
test:
name: Julia ${{ matrix.version }} - ${{ matrix.os }} - ${{ matrix.arch }}
runs-on: ${{ matrix.os }}
permissions: # needed to allow julia-actions/cache to proactively delete old caches that it has created
actions: write
contents: read
strategy:
fail-fast: false
matrix:
version:
- '1.6'
# - '1.6' TODO: replace with new LTS (1.10 probably)
- '1'
- 'pre'
os:
- ubuntu-latest
- macos-latest
- windows-latest
arch:
- x64
steps:
- uses: actions/checkout@v2
- uses: julia-actions/setup-julia@v1
- uses: actions/checkout@v4
- uses: julia-actions/setup-julia@v2
with:
version: ${{ matrix.version }}
arch: ${{ matrix.arch }}
- uses: actions/cache@v1
env:
cache-name: cache-artifacts
with:
path: ~/.julia/artifacts
key: ${{ runner.os }}-test-${{ env.cache-name }}-${{ hashFiles('**/Project.toml') }}
restore-keys: |
${{ runner.os }}-test-${{ env.cache-name }}-
${{ runner.os }}-test-
${{ runner.os }}-
- uses: julia-actions/julia-buildpkg@latest
- run: |
git config --global user.name Tester
git config --global user.email [email protected]
- uses: julia-actions/julia-runtest@latest
continue-on-error: ${{ matrix.version == '1.2' }}
- uses: julia-actions/cache@v2
- uses: julia-actions/julia-buildpkg@v1
- uses: julia-actions/julia-runtest@v1
- uses: julia-actions/julia-processcoverage@v1
- uses: codecov/codecov-action@v1
- uses: codecov/codecov-action@v4
with:
file: lcov.info
files: lcov.info
token: ${{ secrets.CODECOV_TOKEN }}
fail_ci_if_error: false

pd-test:
name: PowerDynamics.jl test - Julia ${{ matrix.version }} - ${{ matrix.os }} - ${{ matrix.arch }}
runs-on: ${{ matrix.os }}
permissions:
actions: write # needed to allow julia-actions/cache to proactively delete old caches that it has created
contents: write
strategy:
fail-fast: false
matrix:
Expand All @@ -60,13 +57,13 @@ jobs:
arch:
- x64
steps:
- uses: actions/checkout@v2
- uses: julia-actions/setup-julia@latest
- uses: actions/checkout@v4
- uses: julia-actions/setup-julia@v2
with:
version: ${{ matrix.version }}
arch: ${{ matrix.arch }}
- uses: julia-actions/julia-buildpkg@latest
- uses: actions/checkout@v2
- uses: julia-actions/julia-buildpkg@v1
- uses: actions/checkout@v4
with:
repository: JuliaEnergy/PowerDynamics.jl
path: PowerDynamics.jl
Expand Down
5 changes: 4 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -3,11 +3,14 @@
*.jl.mem
deps/deps.jl
.idea
Manifest.toml
Manifest*.toml
scratch.jl
docs/build
bugs
design_experiments
benchmark/tune.json
benchmark/*.md
benchmark/*.data
benchmark/*.txt
/benchmark/*.pdf
/docs/src/generated/
Loading
Loading