feat(crypto): implement message cryptography e2e (#122)

* feat(message): rework message format compatibility

- ensure read compatibility with older message formats, for example, from stegano pro v2.1.1.9
- ensure old message formats are auto upgraded to the current formats

* feat(message): rework message format compatibility

- force message format parameter

* chore(deps): upgrade some dependencies

* feat!: #7 error handling in stegano-core

- make the message struct creation fallible
- apply the necessary fallible code all the layers up
- add an old demo image format for tests

* fix: build

* fix: some build pipeline updates

* fix: the whole iterator topic so that the .net bug is eventually replicated.

That means the maximum width and height is x-1

* chore(deps): move from `zip` to `zip_next`, refine features used by dependencies, get rid of unused deps

* feat(demo-images): add some more older version and newer version generated demo images, all from the windows version

* chore(deps): cargo update + criterion update

* fix(tests): fix tests and the iterator impls

* fix: lints

* feat(jpeg-decoder): add a fork of the jpeg-decoder crate

- move stegano-core and stegano-cli into a crates folder
- add the jpeg-decoder to crates as well

* fix(bench:audio): fix wrong paths on the audio benchmarks

* chore(deps): bump several dependencies

* fix(bench:image): fix wrong paths on the image benchmarks

* fix(lints): make clippy happy

* fix(ci): update and overhault deps on the ci pipeline

* fix(ci): update and overhaul deps on the ci pipeline

* fix(ci): variable typo

* fix(ci): variable typo

* fix(ci): fix dynamic toolchain on matrix build

* feat(seasmoke): add first draft of stegano-seasomke

- that provides 2 simple en/decryption functions
- couple of tests
- exposes a ffi and allows for dlyb builds

* fix(ci): the rust toolchain

* fix(lints): make clippy happy

* outcomment new code that will come next

* fix(ci): missing cargo-binstall

* feat(ci:coverage): use region coverage with codecov instead of line coverage

* feat(ci:coverage): add a codecov config so that we get PR comments

* chore: migrate files from str/string args to Path

* add test for unsupported message

* chore: make sure filename extraction is happening at most inner function

* chore: add byte buffer tests

* feat(msg:cipher): implement message encryption and decryption

- refactor payload related factory and traits further
- introduce a `payload::legacy` module for keeping the legacy only factories around
- break up the `FabA` factory, that does way to much
- better separation for the feature based codecs and the legacy codecs, also for their factories
- ensure the password is handed in from the cli to the cipher factory

TODO:

- rethink the way how payload factory and codec could be wired together maybe with Associated types like GATs
- rethink the complexity on the play of Message, Factory and Codec, it just feels too overcomplicated

* add coverage-helper to see if test code can be excluded from the coverage report

* remove the attribute

* feat: migrate away from unmaintained `zip_next` to `zip@v2`

* feat: migrate from cargo-audit to cargo-deny

* chore: upgrade to clap latest version + add env_logger and log to get more error insights

* fix: #121 better error message for too small carrier images

* feat: migrate to derive macros of clap

* feat(api): refactor cli commands and core api

- separate cli subcommands by files
- introduce a `api` mod in stegano_core
- let stegano_core commands dogfood the new `api`
- [ ] implement unveil_raw
- [ ] introduce `conceil` and `reveil` as new subcommands and api that insists on passwords used, deprecate the other subcommands with a note that using no passwords is too insecure

* docs: add the encryption documentation

* docs: update the crate module docs, use now the new api in the samples

* docs: add method docs to the hide api

* docs: add method docs to the unveil api

* add few more tests for hide api

* add few more tests for hide api

* feat: introduce the password type

* cleanup: drop `core::command` api, in favor of the new `stegano_core::api` module

also:
- refine the `RawMessage` struct, to support also the `PayloadCodecFactory`

* add `impl AsRef<[u8]>` for `RawMessage`

* docs: update the module docs

* docs: more update on consitency and related stuff

* feat: add interactive password asking if none was provided

* only use one password interaction funtion

* get rid of speculate, pull few deps up to the workspace manifest

* remove very old unused code

* reduce the exposed api surface, cleanup old code, reimplement benchmarks as micro benchmarks with nightly features

* fix fmt issues

* fix linting

* fix coverage creation

* minor rearrange of use

* fix lints

.cargo/config.toml

@@ -0,0 +1,7 @@
+
+
+[alias]
+lint = "clippy --all-targets"
+benchmarks = "bench --features benchmarks --locked"
+ntest = "nextest run --locked"
+coverage = "llvm-cov --workspace --codecov --output-path codecov.json"

.github/workflows/build.yml

@@ -1,12 +1,12 @@
 name: Build
 on:
   push:
-    branches: [ '*' ]
+    branches: ["*"]
     paths-ignore:
       - "**/docs/**"
       - "**.md"
   pull_request:
-    branches: [ main ]
+    branches: [main]
     paths-ignore:
       - "**/docs/**"
       - "**.md"
@@ -16,132 +16,97 @@ on:
         required: true
 
 jobs:
-  check:
-    name: check
-    strategy:
-      fail-fast: false
-      matrix:
-        version: [ 'ubuntu-latest']
-    runs-on: ${{ matrix.version }}
-    steps:
-      - uses: actions/checkout@v2
-      - name: setup | rust
-        uses: actions-rs/toolchain@v1
-        with:
-          toolchain: stable
-          default: true
-          profile: minimal
-          components: clippy, rustfmt
-      - uses: Swatinem/rust-cache@v1
-      - run: cargo check
-
   lint:
     name: lint
-    needs: check
     strategy:
       fail-fast: false
       matrix:
-        version: [ 'ubuntu-latest']
+        version: ["ubuntu-latest"]
         cargo-cmd:
           - fmt --all -- --check
-          - clippy --all-targets --all-features -- -D warnings
+          - lint
     runs-on: ${{ matrix.version }}
     steps:
-      - uses: actions/checkout@v2
+      - uses: actions/checkout@v4
       - name: setup | rust
-        uses: actions-rs/toolchain@v1
-        with:
-          toolchain: stable
-          default: true
-          profile: minimal
-          components: clippy, rustfmt
-      - uses: Swatinem/rust-cache@v1
+        uses: dtolnay/rust-toolchain@stable
       - run: cargo ${{ matrix['cargo-cmd'] }}
 
   tests:
     name: test
-    needs: check
     strategy:
       fail-fast: false
       matrix:
-        version: [ 'macos-latest', 'ubuntu-latest', 'windows-latest']
-        rust: [ nightly, stable ]
+        version: ["macos-latest", "ubuntu-latest", "windows-latest"]
+        rust: [nightly, stable]
     runs-on: ${{ matrix.version }}
     continue-on-error: ${{ matrix.rust == 'nightly' }}
     steps:
-      - uses: actions/checkout@v2
+      - uses: actions/checkout@v4
       - name: setup | rust
-        uses: actions-rs/toolchain@v1
+        uses: dtolnay/rust-toolchain@stable
         with:
           toolchain: ${{ matrix.rust }}
-          default: true
-          profile: minimal
-      - uses: Swatinem/rust-cache@v1
       - name: cargo test
-        run: cargo test --all --locked -- -Z unstable-options
+        run: cargo test --all --locked
 
   coverage:
     name: coverage report
-    needs: check
     strategy:
       fail-fast: false
       matrix:
-        version: ['ubuntu-latest']
+        version: ["ubuntu-latest"]
     runs-on: ${{ matrix.version }}
     steps:
-      - uses: actions/checkout@v2
+      - uses: actions/checkout@v4
       - name: setup | rust
-        uses: actions-rs/toolchain@v1
-        with:
-          toolchain: stable
-          default: true
-          profile: minimal
-      - uses: Swatinem/rust-cache@v1
-      - name: cargo tarpaulin
-        uses: actions-rs/[email protected]
-        with:
-          version: '0.15.0'
-          args: '-- --test-threads 1'
+        uses: dtolnay/rust-toolchain@stable
+      - uses: cargo-bins/cargo-binstall@main
+      - run: cargo binstall -y cargo-llvm-cov
+      - run: cargo coverage
       - name: Upload to codecov.io
-        uses: codecov/codecov-action@v2
+        uses: codecov/codecov-action@v4
         with:
-          token: ${{ secrets.CODECOV_TOKEN }}
+          file: codecov.json
+          fail_ci_if_error: false
+        env:
+          CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}
       - name: Archive code coverage results
-        uses: actions/upload-artifact@v2
+        uses: actions/upload-artifact@v4
         with:
           name: code-coverage-report
-          path: cobertura.xml
+          path: codecov.json
 
-  pkg-deb:
-    name: binaray package .deb
-    needs: check
+  benchmark:
+    name: benchmark
     runs-on: ubuntu-latest
+    continue-on-error: true
     steps:
-      - uses: actions/checkout@v2
-      - name: cargo deb
-        uses: sassman/rust-deb-builder@v1
-        with:
-          package: stegano-cli
-      - name: Archive deb artifact
-        uses: actions/upload-artifact@v2
-        with:
-          name: stegano-cli-amd64-static.deb
-          path: target/x86_64-unknown-linux-musl/debian/stegano-cli*.deb
+      - uses: actions/checkout@v4
+      - name: setup | rust
+        uses: dtolnay/rust-toolchain@nightly
+      - name: run benchmarks
+        run: cargo +nightly benchmarks
+
+  # pkg-deb:
+  #   name: binaray package .deb
+  #   needs: check
+  #   runs-on: ubuntu-latest
+  #   steps:
+  #     - uses: actions/checkout@v4
+  #     - name: cargo deb
+  #       uses: sassman/rust-deb-builder@v1
+  #       with:
+  #         package: stegano-cli
+  #     - name: Archive deb artifact
+  #       uses: actions/upload-artifact@v2
+  #       with:
+  #         name: stegano-cli-amd64-static.deb
+  #         path: target/x86_64-unknown-linux-musl/debian/stegano-cli*.deb
 
   audit:
-    name: security audit
-    needs: check
+    name: security audit and license check
     runs-on: ubuntu-latest
     steps:
-      - uses: actions/checkout@v2
-      - name: setup | rust
-        uses: actions-rs/toolchain@v1
-        with:
-          toolchain: stable
-          default: true
-          profile: minimal
-      - uses: Swatinem/rust-cache@v1
-      - name: audit
-        uses: actions-rs/audit-check@v1
-        with:
-          token: ${{ secrets.GITHUB_TOKEN }}
+      - uses: actions/checkout@v4
+      - uses: EmbarkStudios/cargo-deny-action@v1

.github/workflows/release-binary-assets.yml

@@ -31,14 +31,14 @@ jobs:
             cross: false
             binName: stegano.exe
     steps:
-      - uses: actions/checkout@v2
+      - uses: actions/checkout@v3
       - name: Setup Rust
         uses: actions-rs/toolchain@v1
         with:
           toolchain: stable
           target: ${{ matrix.target }}
           override: true
-      - uses: Swatinem/rust-cache@v1
+      - uses: Swatinem/rust-cache@v2
       - name: Build
         uses: actions-rs/cargo@v1
         with:
@@ -71,4 +71,4 @@ jobs:
           artifactContentType: application/octet-stream
           omitBodyDuringUpdate: true
           omitNameDuringUpdate: true
-          omitPrereleaseDuringUpdate: true
+          omitPrereleaseDuringUpdate: true

.github/workflows/release.yml

@@ -1,10 +1,10 @@
 name: Release
 on:
   push:
-    tags: 
-      - 'v[0-9]+.[0-9]+.[0-9]+'
-      - 'v[0-9]+.[0-9]+.[0-9]-alpha.[0-9]+'
-      - 'v[0-9]+.[0-9]+.[0-9]-beta.[0-9]+'
+    tags:
+      - "v[0-9]+.[0-9]+.[0-9]+"
+      - "v[0-9]+.[0-9]+.[0-9]-alpha.[0-9]+"
+      - "v[0-9]+.[0-9]+.[0-9]-beta.[0-9]+"
     paths-ignore:
       - "**/docs/**"
       - "**.md"
@@ -18,19 +18,12 @@ jobs:
   publish:
     name: post / cargo publish
     needs: doing-a-build
-    runs-on: macos-latest
+    runs-on: ubuntu-latest
     steps:
-      - uses: actions/checkout@v2
+      - uses: actions/checkout@v4
       - name: setup | rust
-        uses: actions-rs/toolchain@v1
-        with:
-          toolchain: stable
-          default: true
-          profile: minimal
-      - uses: Swatinem/rust-cache@v1
-      - env:
-          CARGO_REGISTRY_TOKEN: ${{ secrets.CARGO_REGISTRY_TOKEN }}
-        run: cargo publish --manifest-path stegano-core/Cargo.toml
+        uses: dtolnay/rust-toolchain@stable
+      - run: cargo publish --manifest-path stegano-core/Cargo.toml
       # without this sleep there was an error that the just published version is not available for stegano-cli as dependency
       - name: waiting for crates.io to get ready with stegano-core latest version
         run: sleep 60
@@ -43,14 +36,9 @@ jobs:
     needs: publish
     runs-on: ubuntu-latest
     steps:
-      - uses: actions/checkout@v2
+      - uses: actions/checkout@v4
       - name: setup | rust
-        uses: actions-rs/toolchain@v1
-        with:
-          toolchain: stable
-          default: true
-          profile: minimal
-      - uses: Swatinem/rust-cache@v1
+        uses: dtolnay/rust-toolchain@stable
       - name: Get version from tag
         id: tag_name
         run: echo ::set-output name=current_version::${GITHUB_REF#refs/tags/v}
@@ -75,4 +63,4 @@ jobs:
           release_name: Release ${{ steps.changelog_reader.outputs.version }}
           body: ${{ steps.changelog_reader.outputs.changes }}
           prerelease: ${{ steps.changelog_reader.outputs.status == 'prereleased' }}
-          draft: ${{ steps.changelog_reader.outputs.status == 'unreleased' }}
+          draft: ${{ steps.changelog_reader.outputs.status == 'unreleased' }}