!!! FEATURE: AssetUsage as CatchUpHook

[dlubitz]

This is an overhaul of the latest "asset usage" implementation which where using a dedicated projection to keep track of asset usages. As this was not handling all necessary events we tried to improve this, but got stuck due to the need of a lot of information of the graph, workspaces, etc (see #5109).

Therfore we decided to try a new approach to use a CatchUpHook instead, which enables us to access all needed information without re-implementing the whole graph logic.

General approach
The package stores the usage of an asset per node and property in a dimension space point. It only stores the usage in the highest/base workspace (usually "live") as long as there are no changes in the same property in a lower/dependent workspace (e.g. user workspace). This allows to get easily an answer to the following questions:

1. Which nodes do I need to edit, to remove all usages for a safe asset removal (Neos Asset Usage/Media Browser)
2. Which cache entries do I need to flush on a change to an asset (this requires an additional traversal over all dependent workspaces).

CatchUpHook
The CatchUpHook is keeping track of all changes to node properties referencing an asset. On replay the CatchUpHook is not handling the events for performance reasons.

Indexing Command
The indexing command fills up the index initially. It can also be used to rebuild the index after projection replays or imports of events.

./flow assetusage:index --contentrepository default

Fixes: #4808 #5084

[bwaidelich]

(how) is that related?

[dlubitz]

The behat tests are not working without this change, as the CatchUpHook expects the doctrine:migrate command to be executed. But it's not when we run the Neos.ContentRepository.BehavioralTests.

This was not an issue before, as these tests where only relying on the projection tables. But the new CatchUpHook now, gets executed also in these test cases.

https://github.com/neos/neos-development-collection/blob/9.0/.composer.json#L34-L41

[mhsdesign]

That is our long standing: you have to do doctrine migrate sometimes problem otherwise it will not work problem that youll fix with the catchup thing as far as i know. see slack: https://neos-project.slack.com/archives/C04PYL8H3/p1727209924914109?thread_ts=1727117857.409749&cid=C04PYL8H3

[mhsdesign]

Upps i was mistaken, this is just really similar to this problem: #5005

In #4878 i also discovered that disabling the catchup hooks will speed up the tests ... and as they are just run along without any assertions in the core tests we should consider doing this some point.

[dlubitz]

Yes, true. But we need to be able to enable it for the test cases where we test the catchup hook 🤷‍♂️

[bwaidelich]

Works like a charm, thanks for the hard work!!
Just a couple of comments, nothing severe!

[mhsdesign]

Post merge + 0.5 i just skimmed through most places thats why no +1 xD