Profile Tests

[roddhjav]

This issue aims to present and discuss the various tests applied to the profiles as well as their current stage of deployment.

Current Status

• Build: make

  • All CI jobs validate the profiles syntax and ensure they can be safely loaded into a kernel.
  • Ensure the profile entry point (@{exec_path}) is defined.

• Checks: make check
  Check basic style of profiles:

  • Ensure apparmor.d header & license
  • Ensure 2 spaces indentation
  • Ensure local include for profile and subprofiles
  • Ensure abi 4 is used
  • Ensure modern profile naming
  • Ensure vim:syntax=apparmor

• Integration Tests: make bats
  Integration tests for core cli profiles.

  • Run simple cli commands to ensure no logs are raised.
  • Uses the bats test system.
  • Run in the Gihub Action as well as in test VM images.
  • The tests are bootstrapped from tldr pages and edited to fit the test suite. Run go run ./tests/cmd -b to generate them. They are basic, but they serve as a good starting point.
  • WIP: about 30 profiles are currently tested.
  • See: https://apparmor.pujol.io/development/integration/

Caution

Do not run the integration tests (make bats) outside a system dedicated for this purpose. The tests can be destructive and wipe your data.

Plan

For more complex software suite, more integration tests need to be done. The plan is to run existing integration suite from these very software in an environment with apparmor.d profiles.

• Systemd

  • They use mkosi to generate a VM image to run their own integration tests.
  • See https://www.codethink.co.uk/articles/2024/systemd-integration-testing-part-1/

• Gnome

  • They use openQA to run their integration tests.
  • See https://gitlab.gnome.org/GNOME/openqa-tests/

[nobody43]

I have the following draft:
https://gist.github.com/nobody43/7a21f518dbebdd48a3c2e7af7934a63b
Is it needed? Tested on Ubuntu 24.04.

Q: Why python?
A: libapparmor could be utilized

Q: Why reimplement?
A: libapparmor is too tightly coupled with actually loading the profiles

Q: Is it on par with apparmor_parser?
A: No, even libapparmor does not claim 1:1 compatibility

Q: What's implemented?
A: grep reason profile_tests.py

Q: What's not implemented?
A: Mutually exclusive abstractions, dangerous file access, missing owner, tunables expansion, file rules to abstraction comparison, etc

Q: Is it reliable?
A: Not tested thoroughly yet

[nobody43]

@roddhjav Should I work on my version further, or it's not the approach you want?

[roddhjav]

@nobody43 Please continue working on your linter. This is 100% the way to go.

make check is a fast and dirty check to catch basic issues (it is highly inspired from what apparmor has). Your linter is the proper solution.

My current priority is to work on profile testing (that goes along with the sub packages #464).

Therefore your work on the linter perfectly goes along with it. You are more than welcome to integrate it with this project.

[nobody43]

Understood, I'll come up with something solid after 2 weeks.