Skip to content

Commit

Permalink
release 2.8.0
Browse files Browse the repository at this point in the history
  • Loading branch information
@danielhrisca
danielhrisca committed Dec 4, 2017
1 parent 7d9f953 commit ea0df7c
Show file tree
Hide file tree
Showing 9 changed files with 571 additions and 279 deletions.
8 changes: 4 additions & 4 deletions README.md
View file
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
[![PyPI version](https://badge.fury.io/py/asammdf.svg)](https://badge.fury.io/py/asammdf) [![Documentation Status](http://readthedocs.org/projects/asammdf/badge/?version=stable)](http://asammdf.readthedocs.io/en/stable/?badge=stable) [![Codacy Badge](https://api.codacy.com/project/badge/Grade/a3da21da90ca43a5b72fc24b56880c99)](https://www.codacy.com/app/danielhrisca/asammdf?utm_source=github.com&utm_medium=referral&utm_content=danielhrisca/asammdf&utm_campaign=badger)
[![PyPI version](https://badge.fury.io/py/asammdf.svg)](https://badge.fury.io/py/asammdf) [![Documentation Status](http://readthedocs.org/projects/asammdf/badge/?version=master)](http://asammdf.readthedocs.io/en/master/?badge=stable) [![Codacy Badge](https://api.codacy.com/project/badge/Grade/a3da21da90ca43a5b72fc24b56880c99)](https://www.codacy.com/app/danielhrisca/asammdf?utm_source=github.com&utm_medium=referral&utm_content=danielhrisca/asammdf&utm_campaign=badger)

*asammdf* is a fast parser/editor for ASAM (Associtation for Standardisation of Automation and Measuring Systems) MDF (Measurement Data Format) files.

Expand Down Expand Up @@ -91,11 +91,11 @@ Usage
```

Check the *examples* folder for extended usage demo, or the documentation
http://asammdf.readthedocs.io/en/latest/examples.html
http://asammdf.readthedocs.io/en/master/examples.html

Documentation
=============
http://asammdf.readthedocs.io/en/latest
http://asammdf.readthedocs.io/en/master

Installation
============
Expand Down Expand Up @@ -127,5 +127,5 @@ optional dependencies needed for exports
Benchmarks
==========

http://asammdf.readthedocs.io/en/latest/benchmarks.html
http://asammdf.readthedocs.io/en/master/benchmarks.html

218 changes: 100 additions & 118 deletions README.rst
View file

Large diffs are not rendered by default.

30 changes: 9 additions & 21 deletions benchmarks/bench.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -189,7 +189,7 @@ def convert_v3_v4(path, output, fmt, memory):
memory,
),
fmt) as timer:
x.convert('4.10')
x.convert('4.10', memory=memory)
output.send([timer.output, timer.error])


Expand All @@ -202,7 +202,7 @@ def convert_v4_v3(path, output, fmt, memory):
memory,
),
fmt) as timer:
y = x.convert('3.30')
y = x.convert('3.30', memory=memory)
y.close()
output.send([timer.output, timer.error])

Expand Down Expand Up @@ -423,18 +423,12 @@ def merge_reader_v3(path, output, fmt):
x2 = MDFreader(files[1])
x2.resample(0.01)
x1.mergeMdf(x2)
x2 = MDFreader(files[1])
x2.resample(0.01)
x1.mergeMdf(x2)
x2 = MDFreader(files[1])
x2.resample(0.01)
x1.mergeMdf(x2)
output.send([timer.output, timer.error])


def merge_reader_v3_compress(path, output, fmt):
os.chdir(path)
files = [r'test.mdf', ] * 3
files = [r'test.mdf', ] * 2
with Timer('Merge files',
'mdfreader {} compress v3'.format(mdfreader_version),
fmt) as timer:
Expand All @@ -443,12 +437,6 @@ def merge_reader_v3_compress(path, output, fmt):
x2 = MDFreader(files[1], compression='blosc')
x2.resample(0.01)
x1.mergeMdf(x2)
x2 = MDFreader(files[1], compression='blosc')
x2.resample(0.01)
x1.mergeMdf(x2)
x2 = MDFreader(files[1], compression='blosc')
x2.resample(0.01)
x1.mergeMdf(x2)
output.send([timer.output, timer.error])


Expand Down Expand Up @@ -573,13 +561,13 @@ def main(path, text_output, fmt):
partial(save_mdf3, memory='minimum'),
save_reader3,
save_reader3_nodata,
# save_reader3_compression,
save_reader3_compression,
partial(save_mdf4, memory='full'),
partial(save_mdf4, memory='low'),
partial(save_mdf4, memory='minimum'),
save_reader4,
save_reader4_nodata,
# save_reader4_compression,
save_reader4_compression,
)

if tests:
Expand All @@ -599,13 +587,13 @@ def main(path, text_output, fmt):
partial(get_all_mdf3, memory='minimum'),
get_all_reader3,
get_all_reader3_nodata,
# get_all_reader3_compression,
get_all_reader3_compression,
partial(get_all_mdf4, memory='full'),
partial(get_all_mdf4, memory='low'),
partial(get_all_mdf4, memory='minimum'),
get_all_reader4,
get_all_reader4_nodata,
# get_all_reader4_compression,
get_all_reader4_compression,
)

if tests:
Expand Down Expand Up @@ -644,12 +632,12 @@ def main(path, text_output, fmt):
partial(merge_v3, memory='low'),
partial(merge_v3, memory='minimum'),
merge_reader_v3,
# merge_reader_v3_compress,
merge_reader_v3_compress,
partial(merge_v4, memory='full'),
partial(merge_v4, memory='low'),
partial(merge_v4, memory='minimum'),
merge_reader_v4,
# merge_reader_v4_compress,
merge_reader_v4_compress,
)

if tests:
Expand Down
100 changes: 100 additions & 0 deletions benchmarks/results/x64_asammdf_2.8.0_mdfreader_2.7.2.txt
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,100 @@
Benchmark environment

* 3.6.1 (v3.6.1:69c0db5, Mar 21 2017, 18:41:36) [MSC v.1900 64 bit (AMD64)]
* Windows-10-10.0.14393-SP0
* Intel64 Family 6 Model 94 Stepping 3, GenuineIntel
* 16GB installed RAM

Notations used in the results

* full = asammdf MDF object created with memory=full (everything loaded into RAM)
* low = asammdf MDF object created with memory=low (raw channel data not loaded into RAM, but metadata loaded to RAM)
* minimum = asammdf MDF object created with memory=full (lowest possible RAM usage)
* compress = mdfreader mdf object created with compression=blosc
* noDataLoading = mdfreader mdf object read with noDataLoading=True

Files used for benchmark:

* 183 groups
* 36424 channels



================================================== ========= ========
Open file Time [ms] RAM [MB]
================================================== ========= ========
asammdf 2.8.0 full mdfv3 772 319
asammdf 2.8.0 low mdfv3 656 165
asammdf 2.8.0 minimum mdfv3 441 77
mdfreader 2.7.2 mdfv3 1783 428
mdfreader 2.7.2 compress mdfv3 3330 127
mdfreader 2.7.2 noDataLoading mdfv3 699 167
asammdf 2.8.0 full mdfv4 1903 381
asammdf 2.8.0 low mdfv4 1783 216
asammdf 2.8.0 minimum mdfv4 1348 88
mdfreader 2.7.2 mdfv4 4849 442
mdfreader 2.7.2 compress mdfv4 6347 138
mdfreader 2.7.2 noDataLoading mdfv4 3425 176
================================================== ========= ========


================================================== ========= ========
Save file Time [ms] RAM [MB]
================================================== ========= ========
asammdf 2.8.0 full mdfv3 359 321
asammdf 2.8.0 low mdfv3 415 172
asammdf 2.8.0 minimum mdfv3 993 86
mdfreader 2.7.2 mdfv3 8402 456
mdfreader 2.7.2 compress mdfv3 8364 424
asammdf 2.8.0 full mdfv4 497 387
asammdf 2.8.0 low mdfv4 507 228
asammdf 2.8.0 minimum mdfv4 2179 97
mdfreader 2.7.2 mdfv4 7958 460
mdfreader 2.7.2 compress mdfv4 8170 417
================================================== ========= ========


================================================== ========= ========
Get all channels (36424 calls) Time [ms] RAM [MB]
================================================== ========= ========
asammdf 2.8.0 full mdfv3 772 325
asammdf 2.8.0 low mdfv3 3784 179
asammdf 2.8.0 minimum mdfv3 5076 92
mdfreader 2.7.2 mdfv3 65 428
mdfreader 2.7.2 nodata mdfv3 1231 379
mdfreader 2.7.2 compress mdfv3 487 127
asammdf 2.8.0 full mdfv4 800 389
asammdf 2.8.0 low mdfv4 7025 226
asammdf 2.8.0 minimum mdfv4 9518 100
mdfreader 2.7.2 mdfv4 71 442
mdfreader 2.7.2 nodata mdfv4 1575 404
mdfreader 2.7.2 compress mdfv4 508 145
================================================== ========= ========


================================================== ========= ========
Convert file Time [ms] RAM [MB]
================================================== ========= ========
asammdf 2.8.0 full v3 to v4 3461 751
asammdf 2.8.0 low v3 to v4 4092 331
asammdf 2.8.0 minimum v3 to v4 4852 163
asammdf 2.8.0 full v4 to v3 3732 753
asammdf 2.8.0 low v4 to v3 4348 313
asammdf 2.8.0 minimum v4 to v3 7136 134
================================================== ========= ========


================================================== ========= ========
Merge files Time [ms] RAM [MB]
================================================== ========= ========
asammdf 2.8.0 full v3 8152 1312
asammdf 2.8.0 low v3 9839 456
asammdf 2.8.0 minimum v3 11694 228
mdfreader 2.7.2 v3 10352 2927
mdfreader 2.7.2 compress v3 15314 2940
asammdf 2.8.0 full v4 11938 1434
asammdf 2.8.0 low v4 13154 549
asammdf 2.8.0 minimum v4 17188 229
mdfreader 2.7.2 v4 16536 2941
mdfreader 2.7.2 compress v4 21261 2951
================================================== ========= ========
94 changes: 94 additions & 0 deletions benchmarks/results/x86_asammdf_2.8.0_mdfreader_2.7.2.txt
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,94 @@
Benchmark environment

* 3.6.1 (v3.6.1:69c0db5, Mar 21 2017, 17:54:52) [MSC v.1900 32 bit (Intel)]
* Windows-10-10.0.14393-SP0
* Intel64 Family 6 Model 94 Stepping 3, GenuineIntel
* 16GB installed RAM

Notations used in the results

* full = asammdf MDF object created with memory=full (everything loaded into RAM)
* low = asammdf MDF object created with memory=low (raw channel data not loaded into RAM, but metadata loaded to RAM)
* minimum = asammdf MDF object created with memory=full (lowest possible RAM usage)
* compress = mdfreader mdf object created with compression=blosc
* noDataLoading = mdfreader mdf object read with noDataLoading=True

Files used for benchmark:

* 183 groups
* 36424 channels



================================================== ========= ========
Open file Time [ms] RAM [MB]
================================================== ========= ========
asammdf 2.8.0 full mdfv3 918 264
asammdf 2.8.0 low mdfv3 898 110
asammdf 2.8.0 minimum mdfv3 577 56
mdfreader 2.7.2 mdfv3 2462 395
mdfreader 2.7.2 compress mdfv3 4174 97
mdfreader 2.7.2 noDataLoading mdfv3 911 105
asammdf 2.8.0 full mdfv4 2644 302
asammdf 2.8.0 low mdfv4 2269 137
asammdf 2.8.0 minimum mdfv4 1883 62
mdfreader 2.7.2 mdfv4 5869 403
mdfreader 2.7.2 compress mdfv4 7367 101
mdfreader 2.7.2 noDataLoading mdfv4 3897 110
================================================== ========= ========


================================================== ========= ========
Save file Time [ms] RAM [MB]
================================================== ========= ========
asammdf 2.8.0 full mdfv3 452 267
asammdf 2.8.0 low mdfv3 495 118
asammdf 2.8.0 minimum mdfv3 1206 62
mdfreader 2.7.2 mdfv3 9258 415
asammdf 2.8.0 full mdfv4 642 307
asammdf 2.8.0 low mdfv4 693 146
asammdf 2.8.0 minimum mdfv4 2642 71
mdfreader 2.7.2 mdfv4 8548 422
================================================== ========= ========


================================================== ========= ========
Get all channels (36424 calls) Time [ms] RAM [MB]
================================================== ========= ========
asammdf 2.8.0 full mdfv3 889 268
asammdf 2.8.0 low mdfv3 12707 120
asammdf 2.8.0 minimum mdfv3 13644 66
mdfreader 2.7.2 mdfv3 80 395
mdfreader 2.7.2 nodata mdfv3 1413 310
mdfreader 2.7.2 compress mdfv3 529 97
asammdf 2.8.0 full mdfv4 968 307
asammdf 2.8.0 low mdfv4 14475 144
asammdf 2.8.0 minimum mdfv4 17057 69
mdfreader 2.7.2 mdfv4 72 403
mdfreader 2.7.2 nodata mdfv4 1806 325
mdfreader 2.7.2 compress mdfv4 562 107
================================================== ========= ========


================================================== ========= ========
Convert file Time [ms] RAM [MB]
================================================== ========= ========
asammdf 2.8.0 full v3 to v4 4048 642
asammdf 2.8.0 low v3 to v4 4551 219
asammdf 2.8.0 minimum v3 to v4 5847 121
asammdf 2.8.0 full v4 to v3 4394 639
asammdf 2.8.0 low v4 to v3 5239 198
asammdf 2.8.0 minimum v4 to v3 8392 98
================================================== ========= ========


================================================== ========= ========
Merge files Time [ms] RAM [MB]
================================================== ========= ========
asammdf 2.8.0 full v3 10061 1168
asammdf 2.8.0 low v3 11245 323
asammdf 2.8.0 minimum v3 13618 186
asammdf 2.8.0 full v4 14144 1226
asammdf 2.8.0 low v4 15410 355
asammdf 2.8.0 minimum v4 21417 170
================================================== ========= ========
Loading

0 comments on commit ea0df7c

Please sign in to comment.