Generalised ECS reactivity with Observers (#10839)

# Objective

- Provide an expressive way to register dynamic behavior in response to
ECS changes that is consistent with existing bevy types and traits as to
provide a smooth user experience.
- Provide a mechanism for immediate changes in response to events during
command application in order to facilitate improved query caching on the
path to relations.

## Solution

- A new fundamental ECS construct, the `Observer`; inspired by flec's
observers but adapted to better fit bevy's access patterns and rust's
type system.

---

## Examples
There are 3 main ways to register observers. The first is a "component
observer" that looks like this:
```rust
world.observe(|trigger: Trigger<OnAdd, Transform>, query: Query<&Transform>| {
    let transform = query.get(trigger.entity()).unwrap();
});
```
The above code will spawn a new entity representing the observer that
will run it's callback whenever the `Transform` component is added to an
entity. This is a system-like function that supports dependency
injection for all the standard bevy types: `Query`, `Res`, `Commands`
etc. It also has a `Trigger` parameter that provides information about
the trigger such as the target entity, and the event being triggered.
Importantly these systems run during command application which is key
for their future use to keep ECS internals up to date. There are similar
events for `OnInsert` and `OnRemove`, and this will be expanded with
things such as `ArchetypeCreated`, `TableEmpty` etc. in follow up PRs.

Another way to register an observer is an "entity observer" that looks
like this:
```rust
world.entity_mut(entity).observe(|trigger: Trigger<Resize>| {
    // ...
});
```
Entity observers run whenever an event of their type is triggered
targeting that specific entity. This type of observer will de-spawn
itself if the entity (or entities) it is observing is ever de-spawned so
as to not leave dangling observers.

Entity observers can also be spawned from deferred contexts such as
other observers, systems, or hooks using commands:
```rust
commands.entity(entity).observe(|trigger: Trigger<Resize>| {
    // ...
});
```

Observers are not limited to in built event types, they can be used with
any type that implements `Event` (which has been extended to implement
Component). This means events can also carry data:

```rust
#[derive(Event)]
struct Resize { x: u32, y: u32 }

commands.entity(entity).observe(|trigger: Trigger<Resize>, query: Query<&mut Size>| {
    let event = trigger.event();
    // ...
});

// Will trigger the observer when commands are applied.
commands.trigger_targets(Resize { x: 10, y: 10 }, entity);
```

You can also trigger events that target more than one entity at a time:

```rust
commands.trigger_targets(Resize { x: 10, y: 10 }, [e1, e2]);
```

Additionally, Observers don't _need_ entity targets:

```rust
app.observe(|trigger: Trigger<Quit>| {
})

commands.trigger(Quit);
```

In these cases, `trigger.entity()` will be a placeholder.

Observers are actually just normal entities with an `ObserverState` and
`Observer` component! The `observe()` functions above are just shorthand
for:

```rust
world.spawn(Observer::new(|trigger: Trigger<Resize>| {});
```

This will spawn the `Observer` system and use an `on_add` hook to add
the `ObserverState` component.

Dynamic components and trigger types are also fully supported allowing
for runtime defined trigger types.

## Possible Follow-ups
1. Deprecate `RemovedComponents`, observers should fulfill all use cases
while being more flexible and performant.
2. Queries as entities: Swap queries to entities and begin using
observers listening to archetype creation triggers to keep their caches
in sync, this allows unification of `ObserverState` and `QueryState` as
well as unlocking several API improvements for `Query` and the
management of `QueryState`.
3. Trigger bubbling: For some UI use cases in particular users are
likely to want some form of bubbling for entity observers, this is
trivial to implement naively but ideally this includes an acceleration
structure to cache hierarchy traversals.
4. All kinds of other in-built trigger types.
5. Optimization; in order to not bloat the complexity of the PR I have
kept the implementation straightforward, there are several areas where
performance can be improved. The focus for this PR is to get the
behavior implemented and not incur a performance cost for users who
don't use observers.

I am leaving each of these to follow up PR's in order to keep each of
them reviewable as this already includes significant changes.

---------

Co-authored-by: Alice Cecile <[email protected]>
Co-authored-by: MiniaczQ <[email protected]>
Co-authored-by: Carter Anderson <[email protected]>

Author: 4 people

Cargo.toml

@@ -2534,6 +2534,17 @@ description = "Systems run in parallel, but their order isn't always determinist
 category = "ECS (Entity Component System)"
 wasm = false
 
+[[example]]
+name = "observers"
+path = "examples/ecs/observers.rs"
+doc-scrape-examples = true
+
+[package.metadata.example.observers]
+name = "Observers"
+description = "Demonstrates observers that react to events (both built-in life-cycle events and custom events)"
+category = "ECS (Entity Component System)"
+wasm = true
+
 [[example]]
 name = "3d_rotation"
 path = "examples/transforms/3d_rotation.rs"

crates/bevy_app/src/app.rs

@@ -8,7 +8,7 @@ use bevy_ecs::{
     intern::Interned,
     prelude::*,
     schedule::{ScheduleBuildSettings, ScheduleLabel},
-    system::SystemId,
+    system::{IntoObserverSystem, SystemId},
 };
 #[cfg(feature = "trace")]
 use bevy_utils::tracing::info_span;
@@ -829,6 +829,15 @@ impl App {
 
         None
     }
+
+    /// Spawns an [`Observer`] entity, which will watch for and respond to the given event.
+    pub fn observe<E: Event, B: Bundle, M>(
+        &mut self,
+        observer: impl IntoObserverSystem<E, B, M>,
+    ) -> &mut Self {
+        self.world_mut().observe(observer);
+        self
+    }
 }
 
 type RunnerFn = Box<dyn FnOnce(App) -> AppExit>;

crates/bevy_ecs/README.md

@@ -307,4 +307,52 @@ fn reader(mut reader: EventReader<MyEvent>) {
 
 A minimal set up using events can be seen in [`events.rs`](examples/events.rs).
 
+### Observers
+
+Observers are systems that listen for a "trigger" of a specific `Event`:
+
+```rust
+use bevy_ecs::prelude::*;
+
+#[derive(Event)]
+struct MyEvent {
+    message: String
+}
+
+let mut world = World::new();
+
+world.observe(|trigger: Trigger<MyEvent>| {
+    println!("{}", trigger.event().message);
+});
+
+world.flush();
+
+world.trigger(MyEvent {
+    message: "hello!".to_string(),
+});
+```
+
+These differ from `EventReader` and `EventWriter` in that they are "reactive". Rather than happening at a specific point in a schedule, they happen _immediately_ whenever a trigger happens. Triggers can trigger other triggers, and they all will be evaluated at the same time!
+
+Events can also be triggered to target specific entities:
+
+```rust
+use bevy_ecs::prelude::*;
+
+#[derive(Event)]
+struct Explode;
+
+let mut world = World::new();
+let entity = world.spawn_empty().id();
+
+world.observe(|trigger: Trigger<Explode>, mut commands: Commands| {
+    println!("Entity {:?} goes BOOM!", trigger.entity());
+    commands.entity(trigger.entity()).despawn();
+});
+
+world.flush();
+
+world.trigger_targets(Explode, entity);
+```
+
 [bevy]: https://bevyengine.org/

crates/bevy_ecs/macros/src/component.rs

@@ -18,6 +18,10 @@ pub fn derive_event(input: TokenStream) -> TokenStream {
     TokenStream::from(quote! {
         impl #impl_generics #bevy_ecs_path::event::Event for #struct_name #type_generics #where_clause {
         }
+
+        impl #impl_generics #bevy_ecs_path::component::Component for #struct_name #type_generics #where_clause {
+            const STORAGE_TYPE: #bevy_ecs_path::component::StorageType = #bevy_ecs_path::component::StorageType::SparseSet;
+        }
     })
 }
 

crates/bevy_ecs/macros/src/lib.rs

@@ -74,6 +74,7 @@ pub fn derive_bundle(input: TokenStream) -> TokenStream {
         .collect::<Vec<_>>();
 
     let mut field_component_ids = Vec::new();
+    let mut field_get_component_ids = Vec::new();
     let mut field_get_components = Vec::new();
     let mut field_from_components = Vec::new();
     for (((i, field_type), field_kind), field) in field_type
@@ -87,6 +88,9 @@ pub fn derive_bundle(input: TokenStream) -> TokenStream {
                 field_component_ids.push(quote! {
                 <#field_type as #ecs_path::bundle::Bundle>::component_ids(components, storages, &mut *ids);
                 });
+                field_get_component_ids.push(quote! {
+                    <#field_type as #ecs_path::bundle::Bundle>::get_component_ids(components, &mut *ids);
+                });
                 match field {
                     Some(field) => {
                         field_get_components.push(quote! {
@@ -133,6 +137,13 @@ pub fn derive_bundle(input: TokenStream) -> TokenStream {
                 #(#field_component_ids)*
             }
 
+            fn get_component_ids(
+                components: &#ecs_path::component::Components,
+                ids: &mut impl FnMut(Option<#ecs_path::component::ComponentId>)
+            ){
+                #(#field_get_component_ids)*
+            }
+
             #[allow(unused_variables, non_snake_case)]
             unsafe fn from_components<__T, __F>(ctx: &mut __T, func: &mut __F) -> Self
             where
@@ -435,6 +446,10 @@ pub fn derive_system_param(input: TokenStream) -> TokenStream {
                     <#fields_alias::<'_, '_, #punctuated_generic_idents> as #path::system::SystemParam>::apply(&mut state.state, system_meta, world);
                 }
 
+                fn queue(state: &mut Self::State, system_meta: &#path::system::SystemMeta, world: #path::world::DeferredWorld) {
+                    <#fields_alias::<'_, '_, #punctuated_generic_idents> as #path::system::SystemParam>::queue(&mut state.state, system_meta, world);
+                }
+
                 unsafe fn get_param<'w, 's>(
                     state: &'s mut Self::State,
                     system_meta: &#path::system::SystemMeta,

crates/bevy_ecs/src/archetype.rs

@@ -23,6 +23,7 @@ use crate::{
     bundle::BundleId,
     component::{ComponentId, Components, StorageType},
     entity::{Entity, EntityLocation},
+    observer::Observers,
     storage::{ImmutableSparseSet, SparseArray, SparseSet, SparseSetIndex, TableId, TableRow},
 };
 use std::{
@@ -119,6 +120,7 @@ pub(crate) struct AddBundle {
     /// For each component iterated in the same order as the source [`Bundle`](crate::bundle::Bundle),
     /// indicate if the component is newly added to the target archetype or if it already existed
     pub bundle_status: Vec<ComponentStatus>,
+    pub added: Vec<ComponentId>,
 }
 
 /// This trait is used to report the status of [`Bundle`](crate::bundle::Bundle) components
@@ -202,12 +204,14 @@ impl Edges {
         bundle_id: BundleId,
         archetype_id: ArchetypeId,
         bundle_status: Vec<ComponentStatus>,
+        added: Vec<ComponentId>,
     ) {
         self.add_bundle.insert(
             bundle_id,
             AddBundle {
                 archetype_id,
                 bundle_status,
+                added,
             },
         );
     }
@@ -314,6 +318,9 @@ bitflags::bitflags! {
         const ON_ADD_HOOK    = (1 << 0);
         const ON_INSERT_HOOK = (1 << 1);
         const ON_REMOVE_HOOK = (1 << 2);
+        const ON_ADD_OBSERVER = (1 << 3);
+        const ON_INSERT_OBSERVER = (1 << 4);
+        const ON_REMOVE_OBSERVER = (1 << 5);
     }
 }
 
@@ -335,6 +342,7 @@ pub struct Archetype {
 impl Archetype {
     pub(crate) fn new(
         components: &Components,
+        observers: &Observers,
         id: ArchetypeId,
         table_id: TableId,
         table_components: impl Iterator<Item = (ComponentId, ArchetypeComponentId)>,
@@ -348,6 +356,7 @@ impl Archetype {
             // SAFETY: We are creating an archetype that includes this component so it must exist
             let info = unsafe { components.get_info_unchecked(component_id) };
             info.update_archetype_flags(&mut flags);
+            observers.update_archetype_flags(component_id, &mut flags);
             archetype_components.insert(
                 component_id,
                 ArchetypeComponentInfo {
@@ -580,21 +589,45 @@ impl Archetype {
 
     /// Returns true if any of the components in this archetype have `on_add` hooks
     #[inline]
-    pub(crate) fn has_on_add(&self) -> bool {
+    pub fn has_add_hook(&self) -> bool {
         self.flags().contains(ArchetypeFlags::ON_ADD_HOOK)
     }
 
     /// Returns true if any of the components in this archetype have `on_insert` hooks
     #[inline]
-    pub(crate) fn has_on_insert(&self) -> bool {
+    pub fn has_insert_hook(&self) -> bool {
         self.flags().contains(ArchetypeFlags::ON_INSERT_HOOK)
     }
 
     /// Returns true if any of the components in this archetype have `on_remove` hooks
     #[inline]
-    pub(crate) fn has_on_remove(&self) -> bool {
+    pub fn has_remove_hook(&self) -> bool {
         self.flags().contains(ArchetypeFlags::ON_REMOVE_HOOK)
     }
+
+    /// Returns true if any of the components in this archetype have at least one [`OnAdd`] observer
+    ///
+    /// [`OnAdd`]: crate::world::OnAdd
+    #[inline]
+    pub fn has_add_observer(&self) -> bool {
+        self.flags().contains(ArchetypeFlags::ON_ADD_OBSERVER)
+    }
+
+    /// Returns true if any of the components in this archetype have at least one [`OnInsert`] observer
+    ///
+    /// [`OnInsert`]: crate::world::OnInsert
+    #[inline]
+    pub fn has_insert_observer(&self) -> bool {
+        self.flags().contains(ArchetypeFlags::ON_INSERT_OBSERVER)
+    }
+
+    /// Returns true if any of the components in this archetype have at least one [`OnRemove`] observer
+    ///
+    /// [`OnRemove`]: crate::world::OnRemove
+    #[inline]
+    pub fn has_remove_observer(&self) -> bool {
+        self.flags().contains(ArchetypeFlags::ON_REMOVE_OBSERVER)
+    }
 }
 
 /// The next [`ArchetypeId`] in an [`Archetypes`] collection.
@@ -681,6 +714,7 @@ impl Archetypes {
         unsafe {
             archetypes.get_id_or_insert(
                 &Components::default(),
+                &Observers::default(),
                 TableId::empty(),
                 Vec::new(),
                 Vec::new(),
@@ -782,6 +816,7 @@ impl Archetypes {
     pub(crate) unsafe fn get_id_or_insert(
         &mut self,
         components: &Components,
+        observers: &Observers,
         table_id: TableId,
         table_components: Vec<ComponentId>,
         sparse_set_components: Vec<ComponentId>,
@@ -808,6 +843,7 @@ impl Archetypes {
                     (sparse_start..*archetype_component_count).map(ArchetypeComponentId);
                 archetypes.push(Archetype::new(
                     components,
+                    observers,
                     id,
                     table_id,
                     table_components.into_iter().zip(table_archetype_components),
@@ -832,6 +868,20 @@ impl Archetypes {
             archetype.clear_entities();
         }
     }
+
+    pub(crate) fn update_flags(
+        &mut self,
+        component_id: ComponentId,
+        flags: ArchetypeFlags,
+        set: bool,
+    ) {
+        // TODO: Refactor component index to speed this up.
+        for archetype in &mut self.archetypes {
+            if archetype.contains(component_id) {
+                archetype.flags.set(flags, set);
+            }
+        }
+    }
 }
 
 impl Index<RangeFrom<ArchetypeGeneration>> for Archetypes {