Merge branch 'develop' into shadowserver-dynamic-config

Author: elsif2

.gitignore

@@ -11,7 +11,7 @@
 *.profile
 .vscode/
 .profile
-intelmq.egg-info
+*.egg-info
 build
 dist
 *.old

CHANGELOG.md

@@ -23,12 +23,14 @@
 ### Core
 - `intelmq.lib.message`: For invalid message keys, add a hint on the failure to the exception: not allowed by configuration or not matching regular expression (PR#2398 by Sebastian Wagner).
 - `intelmq.lib.exceptions.InvalidKey`: Add optional parameter `additional_text` (PR#2398 by Sebastian Wagner).
+- Change the way we discover bots to allow easy extending based on the entry point name. (PR#2413 by Kamil Mankowski)
 - `intelmq.lib.mixins`: Add a new class, `StompMixin` (defined in a new submodule: `stomp`),
   which provides certain common STOMP-bot-specific operations, factored out from
   `intelmq.bots.collectors.stomp.collector` and `intelmq.bots.outputs.stomp.output`
   (PR#2408 by Jan Kaliszewski).
 
 ### Development
+- Makefile: Add codespell and test commands (PR#2425 by Sebastian Wagner).
 
 ### Data Format
 
@@ -68,11 +70,13 @@
 
 ### Documentation
 - Add a readthedocs configuration file to fix the build fail (PR#2403 by Sebastian Wagner).
+- Add a guide of developing extensions packages (PR#2413 by Kamil Mankowski)
 - Update/fix/improve the stuff related to the STOMP bots and integration with the *n6*'s
   Stream API (PR#2408 by Jan Kaliszewski).
 - Complete documentation overhaul. Change to markdown format. Uses the mkdocs-material (PR#2419 by Filip Pokorný).
 
 ### Packaging
+- Add `pendulum` to suggested packages, as it is required for the sieve bot (PR#2424 by Sebastian Wagner).
 
 ### Tests
 

Makefile

@@ -12,4 +12,10 @@ docs: mkdocs.yml docs/* intelmq/etc/feeds.yaml intelmq/etc/harmonization.conf in
 	mkdocs build
 
 clean:
-	rm -rf docs_build .mypy_cache .coverage .pytest_cache dist
+	rm -rf docs_build .mypy_cache .coverage .pytest_cache dist
+
+codespell:
+	codespell -x .github/workflows/codespell.excludelines
+
+test:
+	pytest --no-cov -v intelmq/tests/ && echo "Success!"

contrib/example-extension-package/mybots/__init__.py

@@ -0,0 +1,24 @@
+"""
+SPDX-FileCopyrightText: 2023 CERT.at GmbH <https://cert.at/>
+SPDX-License-Identifier: AGPL-3.0-or-later
+"""
+
+# Use your package as usual
+from mybots.lib import common
+
+from intelmq.lib.bot import CollectorBot
+
+
+class ExampleAdditionalCollectorBot(CollectorBot):
+    """
+    This is an example bot provided by an extension package
+    """
+
+    def process(self):
+        report = self.new_report()
+        if self.raw:  # noqa: Set as parameter
+            report['raw'] = common.return_value('example')
+        self.send_message(report)
+
+
+BOT = ExampleAdditionalCollectorBot

contrib/example-extension-package/mybots/bots/__init__.py

@@ -0,0 +1,9 @@
+"""
+
+SPDX-FileCopyrightText: 2023 CERT.at GmbH <https://cert.at/>
+SPDX-License-Identifier: AGPL-3.0-or-later
+"""
+
+
+def return_value(value):
+    return value