Opportunistically use dense iteration for archetypal iteration

[re0312]

Objective

• currently, bevy employs sparse iteration if any of the target components in the query are stored in a sparse set. it may lead to increased cache misses in some cases, potentially impacting performance.
• partial fixes Opprotunistically use dense iteration when an archetype and its table are 1:1 #12381

Solution

• use dense iteration when an archetype and its table have the same entity count.
• to avoid introducing complicate unsafe noise, this pr only implement for for_each style iteration.
• added a benchmark to test performance for hybrid iteration.

Performance

nearly 2x win in specific scenarios, and no performance degradation in other test cases.

[re0312]

this pr also unblocks some compiler auto SIMD ,consider following code

pub fn new() -> Self {
        let mut world = World::new();

        for _ in 0..10000 {
            world.spawn((TableData(0.0), SparseData(0.0))).id();
        }

        let query = world.query_filtered::<&mut TableData, With<SparseData>>();
        Self(world, query)
    }

    #[inline(never)]
    pub fn run(&mut self) {
        self.1
            .iter_mut(&mut self.0)
            .for_each(|mut v1| v1.0 += 1.)
}

[cBournhonesque]

I think it would be worthwhile to add a comment explaining that this an optimization for hybrid queries where the archetype is actually dense

[alice-i-cecile]

Does this fully fix #2144? I suppose not, since it only optimizes for_each.

[re0312]

Does this fully fix #2144? I suppose not, since it only optimizes for_each.

yeah , Optimizing for-style iteration is much more complicated because it could significantly degrade performance due to introducing branch in the hot path.

[alice-i-cecile]

Sounds good. Let's leave that to follow-up then.

[cBournhonesque]

v.shuffle(&mut deterministic_rand());

I think this should have a comment to explain why you're doing it.
If I understand correctly you have archetypes
A: [TableData, SparseData]
B: [TableData]

Entities are added in this order:
e0: A
e1: B
e2: A
e3: B
...

So that the order of entities is
Table: [e0, e1, e2, ...]
A: [e0, e2, ..]
B: [e1, e3, ..]

Then if you just despawned all the entities of B in order, you would remove e1, then e3, etc.
Each despawn call would do a swap_remove in both the archetype and the table.
So the order of entities in the table would be somewhat jumbled compared with the order in A, which is still [e0, e2, ...]

By adding the shuffling, you want to guarantee that their orders are very different; which maximimes the benefits of your PR (which keeps cache-locality by iterating through the table order, instead of the archetype order

[cBournhonesque]

Could you please explain why this code is faster?

        for _ in 0..10000 {
            world.spawn((TableData(0.0), SparseData(0.0))).id();
        }

        let query = world.query_filtered::<&mut TableData, With<SparseData>>();
        Self(world, query)

In this case the entities should be iterated in the same order before/after your change, no?

[re0312]

In this case the entities should be iterated in the same order before/after your change, no?

it would theoretically have the same cache locality. however, dense iteration could imply to the compiler that we are iterating over continuous memory which could enable automatic SIMD optimizations , This could potentially make the operation nearly 4 times faster.

[ItsDoot]

Not an ECS expert, but the code quality overall looks fine.

[cart]

Makes sense to me! Nice work.

[alice-i-cecile]

Thank you to everyone involved with the authoring or reviewing of this PR! This work is relatively important and needs release notes! Head over to bevyengine/bevy-website#1667 if you'd like to help out.