NIOS2 architecture, Unblob eval, restart scan, semgrep, ...

[m-1-k-3]

• What kind of change does this PR introduce? (Bug fix, feature, docs update, ...)

All kind of changes

• What is the current behavior? (You can also link to an open issue here)

Missing NIOS2 architecture support
Missing restart functionality of stopped scans
Unblob extractor should be furter evaluated (see https://unblob.org/)
No uninstallation routine
Installer crashes on full installation tries on Ubuntu
CVE values should be black/white listed

• What is the new behavior (if this is a feature change)? If possible add a screenshot.

NIOS2 architecture support now integrated in S13 and S115 modules (S14 - r2 module is currently missing)
For restarting a scan just restart it and EMBA will instruct you. This feature is currently in an very early experimental state!
Unblob is now integrated via P61 module for further evaluation -> the results are currenlty not further used from EMBA
CSV logs are stored in a separate directory: /LOG_DIR/csv_logs
Installer should not crash anymore on full installation mode on Ubuntu -> The full installation is not supported on Ubuntu
Installer does not download the exploitdb multiple times anymore
print_output not using echo anymore -> switched to printf with safe_echo functionality
Additionally there is a new escape_echo function for untrusted output piped to another command
Closes #303
Closes #302
Addresses and closes #193 from EMBA side (further addressing needed later in EMBArk reporter)
For white/blacklisting of CVE values there are two new configs you can adjust if you want CVE values black or whitelisted
Better UEFI firmware handling (not executing all EMBA modules on detected UEFI firmware)
Make EMBA nearly complete semgrep compatible and introduced semgrep into check_project.sh - In the future new code needs to be also semgrep compatible (we are currently not passing the IFS check and the status bar is also not fixed)
The installer is now also installing semgrep for later semgrep modules as mentioned in this issue #230

• Does this PR introduce a breaking change? (What changes might users need to make in their application due to this PR?)

CSV logs are moved from main log directory to csv_logs sub directory
From now on new pull requests need to pass a semgrep check with bash rules enabled

• Other information:

New docker image needed for unblob module. Everything else should work fine with the available image. The new docker image will be updated soon.

Screenshot of unblob module:

Semgrep check via check_project.sh:

Scan restart:

After more testing we are going to release version 1.1.2

[BenediktMKuehne]

Looks great.
Should be easy to integrate the black-/whitelist into EMBArk

[m-1-k-3]

Looks great. Should be easy to integrate the black-/whitelist into EMBArk

Warning: We have moved our csv files to a sub directory. I think this needs to be addressed in EMBArk

[p4cx]

I looked over the most part of the new code - it is insanely amazing! Some smaller stuff I found. Please for the future: Can you split your PRs with different branches - I would prefer more PRs but with less code changed in it. It is really confusing to guess for which new feature which code belongs to and what is just a small fix overall.

[m-1-k-3]

Please for the future: Can you split your PRs with different branches - I would prefer more PRs but with less code changed in it. It is really confusing to guess for which new feature which code belongs to and what is just a small fix overall.

I will try it. On the other hand it will get complicated as I need most of the changes (which are then split in multiple PR) for further tests. And as the review process sometimes stucks I have no complete environment as long as the PRs are not merged into the master. Do you have a alternative solution for this?

[BenediktMKuehne]

Did something change regarding the scan-profiles?
(ubuntu-vm)

[BenediktMKuehne]

I will try it. On the other hand it will get complicated as I need most of the changes (which are then split in multiple PR) for further tests. And as the review process sometimes stucks I have no complete environment as long as the PRs are not merged into the master. Do you have a alternative solution for this?

I just switched to Git worktrees for my EMBArk workflow.
Makes switching between branches without committing easier. I don't know though, how much that would help with interdependent branches...

[m-1-k-3]

Did something change regarding the scan-profiles? (ubuntu-vm)

I don't know what is happening on your machine. Works on mine ;)

[m-1-k-3]

Did something change regarding the scan-profiles? (ubuntu-vm)

I don't know what is happening on your machine. Works on mine ;)

Ubuntu related ... can also reproduce it on my Ubuntu

[p4cx]

Without messing around with branches, no, I have no idea how to improve it. Since your development process is very agile and dependent on previous features, you might create a branch, implement the new feature, and after creating the PR, create a new branch from that branch. After we merge the PR, we can merge the master branch with your second branch and so on. But it can get really messy. I'd appreciate if you could try it - if it doesn't work for you we can go back to the current process.

[m-1-k-3]

Did something change regarding the scan-profiles? (ubuntu-vm)

I don't know what is happening on your machine. Works on mine ;)

Ubuntu related ... can also reproduce it on my Ubuntu

It is the notification system which does not support the -p parameter on Ubuntu. I will fix this