Proposal: New ECS implementation + bundles and accessors

[caelunshun]

In implementing custom components for plugins, I realized that the ECS implementation needs to allocate custom plugin components within the plugin's linear memory. We thus have two options:

• continue using a further modified fork of hecs; or
• implement our own ECS.

I am inclined to proceed with the latter because a custom ECS designed to work with the plugin system will be more flexible. Also, I have some ideas to solve #397 that we can integrate into the new ECS.

New ECS design

This is my plan for the new ECS:

• Like EnTT, it will be based on sparse sets, whereas hecs uses archetypes. Sparse sets are easier and safer to implement than archetypes, and they have better performance characteristics when inserting/deleting components at a high frequency. (Our component-based event system does a lot of insertions and deletions, which convinces me that sparse sets are the best choice for Feather.)
• It will support more flexibility with the borrow checker. In particular, components will have a stable location in memory, even if new entities and components are created. As a result, it should be possible to add/remove components and spawn entities inside a query loop, which will be convenient for some use cases in Quill.
• The Component trait will not be implemented for all T; each component type needs to implement the trait instead. While this is less convenient in some areas, it prevents querying for types that are not components, like bundles. (We'll provide a derive macro for Component to avoid too much boilerplate.)
• It will have built-in support for bundles and bundle accessors as an alternative implementation to HList and Plucker for components #388 and High-Level Entity Methods (WIP) #397. My vision is this: each entity type has its own Bundle struct, which is the set of components it spawns with. The ECS crate provides a derive macro that generates an accessor struct for the bundle that can be queried. The bundle accessor is a wrapper for EntityId that provides convenience methods to access components, and it can also include methods that access multiple components, like item_in_main_hand(). In code, this looks like:

#[derive(Bundle)]
#[bundle(generate_accessor = "Player")] // generates the Player struct
pub struct PlayerBundle {
    /// another bundle containing default components for all living entities, like `Position` and `Health`
    /// (this emulates inheritance via composition)
    #[bundle(inherit)]
    base: LivingEntityBundle,
    inventory: Inventory,
    player_marker: PlayerMarker,
}

impl Player {
    pub fn item_in_main_hand(&self) -> &ItemStack { /* ... */ }
}

// To spawn a player:
game.spawn_entity(PlayerBundle {
    base: LivingEntityBundle {
        pos,
        ..Default::default()
    },
    ..Default::default()
});

// The accessor for the bundle can be queried using a new `query_as` function
// to distinguish accessors from real components.
for (entity_id, player): (EntityId, Player) in game.query_as::<Player, ()>() {
    // the accessor provides methods to conveniently get components
    println!("{:?}", player.position());
    // it can also provide convenience functions that access multiple components
    // (HeldItem and Inventory in this case)
    println!("Player is holding {:?}", player.item_in_main_hand());
}

// Accessors also work with abstract bundles like LivingEntity
for (entity_id, entity): (EntityId, LivingEntity) in game.query_as::<LivingEntity, ()>() {
    println!("{:?}", entity.position());
}

I opened this issue to collect feedback in the proposal, in the hope that we can resolve any issues with it before I implement something we don't want. So feedback is welcome from all.

[Schuwi]

Hmm but if I understand the bundle approach correctly that would mean that any bundle query would query every single component that constitutes an entity type, which would defeat the whole purpose of ECS of only accessing the data you need right?

[caelunshun]

The bundles would be optimized by special-casing them in the ECS so that query_as can efficiently yield only which entities satisfy the bundle. The data would be accessed lazily when the getter/setter methods are called, e.g. position() lazily accesses the entity's position.

We could also remove the getter/setter methods for components, but we'd need to leave functions like item_in_main_hand because they need to access multiple components.

[ambeeeeee]

Should we also mention the event graph plans here?

[kirawi]

It will support more flexibility with the borrow checker. In particular, components will have a stable location in memory, even if new entities and components are created. As a result, it should be possible to add/remove components and spawn entities inside a query loop, which will be convenient for some use cases in Quill.

Exactly what I'd love to have.

[Defman]

@caelunshun do you have todo list, if some of us want to help out?

Also should the addition of components and entities not be queued up and processed after the query anyway. Ie the ECS don't insert/remove components or spawn new entities until all queries are dropped (kinda like lock)?

[Schuwi]

How about not having bundles for each entity type but instead bundles for capabilities? E.g. a SendMessage bundle to send messages to every entity that can receive them (bad example except if we also wanna have the console as an ECS entity) or an ItemInHand bundle for all entites that can have an item in hand.