merge to master for release 0.9 (#429)

* include LICENSE in source distribution (#365) * [arxml]: rx/tx typo fix * annotate and fix codestyle dbc (#369) * Always use Signal.is_little_endian as bool #326 (#371) * Add coding, remove #!python where inappropriate (#370) * annotation cleanup ticket #323 (#372) * Update cli doc (inspired by #361) (#373) * fix: xls is using the wrong number for arbitration id (#377) * add outputformat for Scapy (#378) * fix special char handling in scapy exporter * Dump factor, offset, min, max as float. Load as numeric (#380) * added --jsonNativeTypes as cli option * [arxml]: merge container PDU and Secured PDU support (#384) * [arxml]: support for containers * [arxml]: create signalgroups for containers * [arxml]: support secured pdus (#383) * fix for wrong default ArbitrationId handling (#386) * fix for #381 wrong default ArbitrationId handling * optional ignoring failures durig character encoding #375 (#388) * fix for #238 (j1939 Frame setters might raise TypeError) (#389) * move j1939 (pgn, prio, source) handling direct to arbitration-id class (fixes #238) * [arxml]: #385 // Arxml Container support and ARXML refactoring (#390) * [json] fix multiplex reading issue (#136) * [arxml]: refactoring, DLC for canFD support canmatrix.Frame.fit_dlc now fits to next correct dlc value, DBC export reniced for candb++ (#385) * [arxml]: refactoring and better type handling #368 * [dbc]: fix #242. J1939 and FD Frames in dbc * [xlsx]: fixes #240 (signal init value in excel template) * [xlsx] feature for for #288 (#393) separate lines for values * switch from optparse to click (#236) (#394) * add Basic cli tests (#395) * Annotation cleanup #323 (#396) * Remove cm prefix (#236) (#399) * add muxing support to scapy output; merge PR #398 also (#400) * [try to] Unify imports (#402) unify imports (#236) * [wireshark] dump/export wireshark lua can subdissector (#404) * add cannelloni dissector for wireshark * cycle_time as generic signal and frame attribute (#405) * implement cycle_time as generic signal and frame attribute / remove Gen[Msg/Sig]cycle from attributes #146 * use effective frame cycle time for export use gcd of singnal-cycletimes for calculating effective cycle time * [dbc] Iss407: Signal Comments with space bevore semicolon broken import (#409) * [dbc] fix for issue #407 Signal Comments with space bevore semicolon broken import * make initial value to a native attribute of Signal class (#408) * [dbf] exended ids corrected * [dbc] optional allow multiple singals with same name in frame #411 (#412) * Fix setup.py for console_scripts (#417) * [dbc] Iss413: fix comment reading with whitespaces in front of ; (#416) * fix for iss #414, keeps defines in dbc: this keeps defines from dbc in matrix object, even if native attribute exists. * convert: selective rx/tx ecu extraction issue #421 (#422) * [arxml] can_fd info (potentional fix for #410) (#418) * add baudrate attributes * Auto deploy (#420) * add auto deploy for tags on master
ebroecker · Nov 8, 2019 · 889c295 · 889c295
1 parent 2bcf1af
commit 889c295
Show file tree

Hide file tree

Showing 64 changed files with 17,071 additions and 1,183 deletions.
diff --git a/.travis.yml b/.travis.yml
@@ -35,3 +35,11 @@ install:
 script:
   - tox
   - tox -e codecov
+
+deploy:
+  provider: pypi
+  user: ebroecker
+  skip_existing: true
+  on:
+    tags: true
+    branch: master 
diff --git a/MANIFEST.in b/MANIFEST.in
@@ -1,2 +1,3 @@
 include versioneer.py
 include canmatrix/_version.py
+include LICENSE
diff --git a/docs/cli.rst b/docs/cli.rst
@@ -5,6 +5,7 @@ Command Line Interface
 1. **canconvert**:
 
     converts CAN database formats between all supported formats.
+
     Example: "*canconvert* someArSystemdescription.arxml file.dbc"
 
 2. **cancompare**:
@@ -17,29 +18,39 @@ Command Line Interface
 General
 _______
 
+If you properly install canmatrix using *pip,* the setuptools create
+two ecxcutables for you: `canconvert` and `cancompare`. You can find these executables
+near to your `python(.exe)`
+
+If you aren't able to find the scripts, you can still run it as
+
 ::
 
-    $ canconvert.py -h
-    $ canconvert.py --help
+$ python -m canmatrix.cli.compare [args]
+$ python -m canmatrix.cli.convert [args]
 
-show help message/usage and exits
+To show help message/usage call ::
 
-::
+    $ canconvert -h
+    $ canconvert --help
 
-    $ canconvert.py -v
+
+Another arguments are ::
+
+    $ canconvert -v
 
 Output verbosity
 
 
 ::
 
-    $ canconvert.py -s
+    $ canconvert -s
 
 don't print status messages to stdout. (only errors)
 
 ::
 
-    $ canconvert.py -f FORCE_OUTPUT
+    $ canconvert -f FORCE_OUTPUT
 
 enforce output format, ignoring output file extension (e.g., -f csv)
 
@@ -51,19 +62,19 @@ __________________________
 
 ::
 
-    $ canconvert.py source.dbc target.xlsx
+    $ canconvert source.dbc target.xlsx
 
 **convert dbc file to dbf:**
 
 ::
 
-    $ canconvert.py source.dbc target.dbf
+    $ canconvert source.dbc target.dbf
 
 **convert arxml file to dbc:**
 
 ::
 
-    $ canconvert.py source.arxml target.dbc
+    $ canconvert source.arxml target.dbc
 
 Note: in case of ``.arxml`` there can be multiple can databases in.
 Thus the target ``target.dbc`` may in this case be called ``BUS-NAME-IN-ARXML_target.dbc``.
@@ -75,15 +86,15 @@ You can even convert to the same format:
 
 ::
 
-    $ canconvert.py source.dbc target.dbc
+    $ canconvert source.dbc target.dbc
 
 Multiple charset support:
 
 **convert dbc file to dbc with different charset:**
 
 ::
 
-    $ canconvert.py --dbcImportEncoding=iso-8859-1 --dbcImportCommentEncoding=cp-1252 --dbcExportEncoding=utf-8 --dbcExportCommentEncoding=utf-8 source.dbc target.dbc
+    $ canconvert --dbcImportEncoding=iso-8859-1 --dbcImportCommentEncoding=cp-1252 --dbcExportEncoding=utf-8 --dbcExportCommentEncoding=utf-8 source.dbc target.dbc
 
 This converts ``source.dbc`` where units are coded in ``iso-8859-1`` and comments are coded in ``cp-1252`` in a ``target.dbc`` where everything is coded in ``utf-8``.
 Similar charset conversions are possible or even mandatory for following formats: dbc, dbf and sym.
@@ -96,31 +107,31 @@ _______________________
 
 ::
 
-    $ canconvert.py --deleteZeroSignals source.dbc target.dbc
+    $ canconvert --deleteZeroSignals source.dbc target.dbc
 
 will delete signals with 0 bit length from matrix
 
 **delete unwanted attributes from signals:**
 
 ::
 
-    $ convert.py --deleteSignalAttributes GenMsgCycleTime,MyAttrib source.dbc target.dbc
+    $ canconvert --deleteSignalAttributes GenMsgCycleTime,MyAttrib source.dbc target.dbc
 
 will delete the attributes ``GenMsgCycleTime`` and ``MyAttrib`` from all signals in ``source.dbc`` and save the result in ``target.dbc``
 
 **delete unwanted attributes from frames:**
 
 ::
 
-    $ convert.py --deleteFrameAttributes GenMsgCycleTime,MyAttrib source.dbc target.dbc
+    $ canconvert --deleteFrameAttributes GenMsgCycleTime,MyAttrib source.dbc target.dbc
 
 will delete the attributes ``GenMsgCycleTime`` and ``MyAttrib`` from all frames in ``source.dbc`` and save the result in ``target.dbc``
 
 **recalculate DLC:**
 
 ::
 
-    $ canconvert.py --recalcDLC=max source.dbc target.dbc
+    $ canconvert --recalcDLC=max source.dbc target.dbc
 
 this will recalculate DLC for each frame in ``source.dbc``.
 In ``target.dlc``  the same DLC like in ``source.dbc`` will be stored, except the calculated DLC is bigger.
@@ -130,7 +141,7 @@ Than the calculated DLC will be stored.
 
 ::
 
-    $ canconvert.py --recalcDLC=force source.dbc target.dbc
+    $ canconvert --recalcDLC=force source.dbc target.dbc
 
 this will recalculate DLC for each frame in ``source.dbc``.
 In ``target.dlc`` the calculated DLC will be stored independently from ``source.dbc``.
@@ -139,23 +150,23 @@ In ``target.dlc`` the calculated DLC will be stored independently from ``source.
 
 ::
 
-    $ canconvert.py --deleteObsoleteDefines source.dbc target.dbc
+    $ canconvert --deleteObsoleteDefines source.dbc target.dbc
 
 this will remove all defines which no attribute exist for in ``source.dbc`` and store the result in ``target.dlc``.
 
 **delete ECU:**
 
 ::
 
-    $ canconvert.py --deleteECU=myEcu,myEcu2 source.dbc target.dbc
+    $ canconvert --deleteECU=myEcu,myEcu2 source.dbc target.dbc
 
 this will remove ECUs ``myEcu`` and ``myEcu2`` in ``source.dbc`` and store the result in ``target.dlc``.
 
 **rename ECU:**
 
 ::
 
-    $ canconvert.py --renameECU=myEcu:myNewEcu,myEcu2:myNewEcu2 source.dbc target.dbc
+    $ canconvert --renameECU=myEcu:myNewEcu,myEcu2:myNewEcu2 source.dbc target.dbc
 
 this will load ``source.dbc`` and rename ECU ``myEcu`` in ``myNewEcu``  and ``myEcu2`` in ``myNewEcu2``.
 The result is stored in ``target.dlc``.
@@ -164,15 +175,15 @@ The result is stored in ``target.dlc``.
 
 ::
 
-    $ canconvert.py --deleteFrame=myFrame,myFrame2 source.dbc target.dbc
+    $ canconvert --deleteFrame=myFrame,myFrame2 source.dbc target.dbc
 
 this will remove frames ``myFrame`` and ``myFrame2`` in ``source.dbc`` and store the result in ``target.dlc``.
 
 **rename Frame:**
 
 ::
 
-    $ canconvert.py --renameFrame=myFrame:myNewFrame,myFrame2:myNewFrame2 source.dbc target.dbc
+    $ canconvert --renameFrame=myFrame:myNewFrame,myFrame2:myNewFrame2 source.dbc target.dbc
 
 this will load ``source.dbc`` and rename frames ``myFrame`` in ``myNewFrame``  and ``myFrame2`` in ``myNewFrame2``.
 The result is stored in ``target.dlc``.
@@ -182,15 +193,15 @@ The result is stored in ``target.dlc``.
 
 ::
 
-    $ canconvert.py --deleteSignal=mySignal,mySignal2 source.dbc target.dbc
+    $ canconvert --deleteSignal=mySignal,mySignal2 source.dbc target.dbc
 
 this will remove signales ``mySignal`` and ``mySignal2`` in ``source.dbc`` and store the result in ``target.dlc``.
 
 **rename Signal:**
 
 ::
 
-    $ canconvert.py --renameSignal=mySignal:myNewSignal,mySignal2:myNewSignal2 source.dbc target.dbc
+    $ canconvert --renameSignal=mySignal:myNewSignal,mySignal2:myNewSignal2 source.dbc target.dbc
 
 this will load ``source.dbc`` and rename signals ``mySignal`` in ``myNewSignal``  and ``mySignal2`` in ``myNewSignal2``.
 The result is stored in ``target.dlc``.
@@ -199,14 +210,14 @@ The result is stored in ``target.dlc``.
 
 ::
 
-    $ canconvert.py --setFrameFd=myFrame,myFrame2 source.dbc target.dbc
+    $ canconvert --setFrameFd=myFrame,myFrame2 source.dbc target.dbc
 
 this will set frame-type of  ``myFrame`` and ``myFrame2`` in ``source.dbc`` to CANFD and store the result in ``target.dlc``
                         list) Syntax: --setFrameFd=myFrame1,mySecondFrame
 
 ::
 
-    $ canconvert.py --unsetFrameFd=myFrame,myFrame2 source.dbc target.dbc
+    $ canconvert --unsetFrameFd=myFrame,myFrame2 source.dbc target.dbc
 
 this will set frame-type of  ``myFrame`` and ``myFrame2`` in ``source.dbc`` to normal (not FD) and store the result in ``target.dlc``
                         list) Syntax: --unsetFrameFd=myFrame1,mySecondFrame
@@ -219,7 +230,7 @@ __________________
 
 ::
 
-    $ canconvert.py --ecus=REAR_ECU source.dbc target.dbc
+    $ canconvert --ecus=REAR_ECU source.dbc target.dbc
 
 This generates a ``target.dbc`` with all Informations out of ``source.dbc`` which are needed for ``REAR_ECU``.
 All frames which are received or sent by ``REAR_ECU`` are extracted. Also all attributes of the frames and the ECU.
@@ -229,45 +240,54 @@ This is some *lite* ECU-Extract.
 
 ::
 
-    $ canconvert.py --ecus=FRONT_ECU,REAR_ECU source.dbc target.dbc
+    $ canconvert --ecus=FRONT_ECU,REAR_ECU source.dbc target.dbc
+
+**extract matrix with frames which FRONT_ECU receives and with frames REAR_ECUS transmits:**
+
+::
+
+    $ canconvert --ecus=FRONT_ECU:rx,REAR_ECU:tx source.dbc target.dbc
+
+
+
 
 **extract frame[s] out of matrix:**
 
 ::
 
-    $ canconvert.py --frames=REAR_FRAME,FRONT_FRAME source.dbc target.dbc
+    $ canconvert --frames=REAR_FRAME,FRONT_FRAME source.dbc target.dbc
 
 Extracts the frames ``REAR_FRAME`` and ``FRONT_FRAME`` with the needed ECUs and attributes.
 
 **merge multiple databases:**
 
 ::
 
-    $ canconvert.py --merge=second.dbc source.dbc target.dbc
+    $ canconvert --merge=second.dbc source.dbc target.dbc
 
 Merges ``source.dbc`` and ``second.dbc`` in ``target.dbc``.
 
 **merge ECU from other  database:**
 
 ::
 
-    $ canconvert.py --merge=second.dbc:ecu=REAR_ECU source.dbc target.dbc
+    $ canconvert --merge=second.dbc:ecu=REAR_ECU source.dbc target.dbc
 
 Merges REAR_ECU out of ``second.dbc`` with ``source.dbc`` and store result in ``target.dbc``.
 
 **merge FRAME from other database:**
 
 ::
 
-    $ canconvert.py --merge=second.dbc:frame=REAR_FRAME source.dbc target.dbc
+    $ canconvert --merge=second.dbc:frame=REAR_FRAME source.dbc target.dbc
 
 Merges REAR_FRAME out of ``second.dbc`` with ``source.dbc`` and store result in ``target.dbc``.
 
 **combinations and multiple extraction possible:**
 
 ::
 
-    $ canconvert.py --merge=second.dbc:ecu=REAR_ECU:ecu=FRONT_ECU:frame=FRAME1:FRAME=FRAME2 source.dbc target.dbc
+    $ canconvert --merge=second.dbc:ecu=REAR_ECU:ecu=FRONT_ECU:frame=FRAME1:FRAME=FRAME2 source.dbc target.dbc
 
 Merges REAR_ECU and FRONT_ECU and FRAME1 and FRAME2 out of ``second.dbc`` with ``source.dbc`` and store result in ``target.dbc``.
 
@@ -403,6 +423,8 @@ ____________________
 
 
 * yaml
+* scapy
+* lua
 * json:
 
  --jsonExportCanard

diff --git a/docs/formats.rst b/docs/formats.rst
@@ -15,16 +15,17 @@ There are several importers (read) and exporters (write) for this object.
     * .sym [peak](http://www.peak-system.com) pcan can description
 
 * supported file formats for export:
-
- * .dbc
- * .dbf
- * .kcd
- * .xls(x)
- * .json [Canard](https://github.com/ericevenchick/CANard) (open source!)
- * .arxml (very basic implementation)
- * .yaml (dump of the python object)
- * .sym
- * .xml fibex
+    * .dbc
+    * .dbf
+    * .kcd
+    * .xls(x)
+    * .json [Canard](https://github.com/ericevenchick/CANard) (open source!)
+    * .arxml (very basic implementation)
+    * .yaml (dump of the python object)
+    * .sym
+    * .xml fibex
+    * .py [scapy] (https://scapy.net/) scapy can message decoder
+    * .lua [wireshark] wireshark can subdissector
 
 
 Export
@@ -73,3 +74,52 @@ ______
 |xls(x)|+   |+    |+            |+        |+        |         |        |+        |p      |p      |          |p           |             |
 +------+----+-----+-------------+---------+---------+---------+--------+---------+-------+-------+----------+------------+-------------+
 
+
+Scapy Export
+____________
+
+Create can frame decoder for some can definition
+
+Example:
+
+    $ canconvert source.dbc target.py
+
+in scapy use like this:
+
+.. code-block:: python
+
+    load_contrib("target")
+    sock = CANSocket("can0", basecls=DBC)
+    pkt = sock.recv()
+
+
+Wireshark Dissector
+___________________
+
+    $ canconvert source.dbc target.py
+
+Example:
+
+    $ canconvert source.sdbc target.lua
+
+in wireshark use like this:
+
+    $ wireshark -X lua_script:target.lua
+
+    Linux socketcan can directly traced with wireshark. In wireshark select some can frame.
+    Right-click and select *decode as* and select *SIGNALDECODE* as subdissector
+
+    *dissect_canneloni.lua* in the canmatrix example folder shows an example for decoding can
+    frames which are packet in UDP frames by cannelone (https://github.com/mguentner/cannelloni, https://github.com/PhilippFux/cannelloni)
+
+    you could use it like:
+
+    .. code-block:: bash
+
+        $ chdir canmatrix/examples
+        $ canconvert some.dbc can_database.lua
+        $ wireshark wireshark -X lua_script:dissect_cannelloni.lua
+
+    note: default cannelloni is mapped to UDP port 3333, you have to change it
+
+    note: *can_database* is hard coded in *dissect_cannelloni.lua*, you can customize it there