Skip to content

Commit

Permalink
Replace snapshots with hash-based cross-platform determinism test (#555)
Browse files Browse the repository at this point in the history
# Objective

Currently, Avian has a cross-platform determinism test that produces [insta](https://insta.rs/) snapshots, which are compared between platforms in CI with GitHub Actions.

However, these snapshots are quite large, and always pollute diffs in PRs (such as this one) whenever changes affecting simulation behavior are made. Insta must also be installed to generate these snapshots, making it annoying for new contributors to deal with. Overall, it's a hassle.

The test itself is also very simplistic and only tests collisions.

Instead of using snapshots, we should just run the simulation for a while, and compute a single transform hash based on the position and rotation of all bodies. This is much simpler and more convenient.

## Solution

Add a 2D test for cross-platform determinism, simulating pairs of objects constrained via revolute joints falling to the ground. After 500 steps, a transform hash is computed and compared against the expected value. If they don't match, the test will fail with a message indicating that the expected hash should be updated if the change in behavior was intended.

```
test tests::determinism_2d::cross_platform_determinism_2d ... FAILED

failures:

---- tests::determinism_2d::cross_platform_determinism_2d stdout ----
thread 'tests::determinism_2d::cross_platform_determinism_2d' panicked at crates\avian2d\../../src\tests\determinism_2d.rs:63:5:

Expected hash 0xa1c6b39, found hash 0xa38b5132 instead.
If changes in behavior were expected, update the hash in src/tests/determinism_2d.rs on line 61.
```

This test is based on Box2D's [`FallingHinges`](https://github.com/erincatto/box2d/blob/90c2781f64775085035655661d5fe6542bf0fbd5/samples/sample_determinism.cpp) test and sample.

There is also a new `determinism_2d` example, which is a visual representation of the test.

https://github.com/user-attachments/assets/2d212fe7-14b3-4dff-b1d0-96c531da6918

While making this, I stumbled upon some bugs as well:

- The max correction for revolute joints is way too small, often causing explosive behavior when it is exceeded.
- When the `PhysicsSchedule` runs, `Time<Substeps>` is only set once the substepping loop starts. Earlier in the schedule, the previous value is used. However, some systems like `update_contact_softness` might need the substep time before that. Using the previous value can lead to seemingly indeterministic results e.g. when resetting scenes.

I have fixed both of these issues.
  • Loading branch information
Jondolf authored Nov 11, 2024
1 parent 5deada5 commit 2563e71
Show file tree
Hide file tree
Showing 11 changed files with 425 additions and 1,576 deletions.
7 changes: 6 additions & 1 deletion crates/avian2d/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -76,9 +76,10 @@ bitflags = "2.5.0"
examples_common_2d = { path = "../examples_common_2d" }
benches_common_2d = { path = "../benches_common_2d" }
bevy_math = { version = "0.15.0-rc", features = ["approx"] }
glam = { version = "0.29", features = ["bytemuck"] }
approx = "0.5"
bytemuck = "1.19"
criterion = { version = "0.5", features = ["html_reports"] }
insta = "1.0"
bevy_mod_debugdump = { git = "https://github.com/andriyDev/bevy_mod_debugdump", branch = "bevy-0.15" }

[[example]]
Expand All @@ -105,6 +106,10 @@ required-features = ["2d", "default-collider"]
name = "custom_collider"
required-features = ["2d"]

[[example]]
name = "determinism_2d"
required-features = ["2d", "default-collider", "enhanced-determinism"]

[[example]]
name = "fixed_joint_2d"
required-features = ["2d", "default-collider"]
Expand Down
253 changes: 253 additions & 0 deletions crates/avian2d/examples/determinism_2d.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,253 @@
//! An example providing a visual demonstration of the 2D cross-platform determinism test
//! in `src/tests/determinism_2d`.
//!
//! This scene is designed to produce a chaotic result engaging:
//!
//! - the contact solver
//! - speculative collision
//! - joints and joint limits
//!
//! Once the simulation has run for a while, a transform hash is computed.
//! The determinism test compares this to the expected value for every PR on multiple platforms using GitHub Actions.
//! Every time simulation behavior changes, the expected hash must be updated.
//!
//! This test is based on the `FallingHinges` test in the Box2D physics engine:
//! <https://github.com/erincatto/box2d/blob/90c2781f64775085035655661d5fe6542bf0fbd5/samples/sample_determinism.cpp>

use avian2d::{
math::{AdjustPrecision, Scalar, Vector, PI},
prelude::*,
};
use bevy::{
color::palettes::tailwind::CYAN_400, input::common_conditions::input_just_pressed, prelude::*,
render::camera::ScalingMode,
};
use bytemuck::{Pod, Zeroable};

// How many steps to record the hash for.
const STEP_COUNT: usize = 500;

const ROWS: u32 = 30;
const COLUMNS: u32 = 4;

fn main() {
App::new()
.add_plugins((
DefaultPlugins,
PhysicsPlugins::default().with_length_unit(0.5),
PhysicsDebugPlugin::default(),
))
.init_resource::<Step>()
.add_systems(Startup, (setup_scene, setup_ui))
.add_systems(PostProcessCollisions, ignore_joint_collisions)
.add_systems(FixedUpdate, update_hash)
.add_systems(
PreUpdate,
// Reset the scene when the R key is pressed.
(clear_scene, setup_scene)
.chain()
.run_if(input_just_pressed(KeyCode::KeyR)),
)
.run();
}

#[derive(Resource, Default, Deref, DerefMut)]
struct Step(usize);

fn setup_scene(
mut commands: Commands,
mut materials: ResMut<Assets<ColorMaterial>>,
mut meshes: ResMut<Assets<Mesh>>,
) {
commands.spawn((
Camera2d,
Projection::from(OrthographicProjection {
scaling_mode: ScalingMode::FixedHorizontal {
viewport_width: 40.0,
},
..OrthographicProjection::default_2d()
}),
Transform::from_xyz(0.0, 7.5, 0.0),
));

let ground_shape = Rectangle::new(40.0, 2.0);
commands.spawn((
Name::new("Ground"),
RigidBody::Static,
Collider::from(ground_shape),
Mesh2d(meshes.add(ground_shape)),
MeshMaterial2d(materials.add(Color::WHITE)),
Transform::from_xyz(0.0, -1.0, 0.0),
));

let half_size = 0.25;
let square_shape = Rectangle::new(2.0 * half_size, 2.0 * half_size);
let square_collider = Collider::from(square_shape);
let square_mesh = meshes.add(square_shape);
let square_material = materials.add(Color::from(CYAN_400));

let offset = 0.4 * half_size;
let delta_x = 10.0 * half_size;
let x_root = -0.5 * delta_x * (COLUMNS as f32 - 1.0);

for col in 0..COLUMNS {
let x = x_root + col as f32 * delta_x;

let mut prev_entity = None;

for row in 0..ROWS {
let entity = commands
.spawn((
Name::new("Square ({col}, {row})"),
RigidBody::Dynamic,
square_collider.clone(),
Mesh2d(square_mesh.clone()),
MeshMaterial2d(square_material.clone()),
Transform::from_xyz(
x + offset * row as f32,
half_size + 2.0 * half_size * row as f32,
0.0,
)
.with_rotation(Quat::from_rotation_z(0.1 * row as f32 - 1.0)),
))
.id();

if row & 1 == 0 {
prev_entity = Some(entity);
} else {
commands.spawn((
Name::new(format!(
"Revolute Joint ({}, {})",
prev_entity.unwrap(),
entity
)),
RevoluteJoint::new(prev_entity.unwrap(), entity)
.with_angle_limits(-0.1 * PI, 0.2 * PI)
.with_compliance(0.0001)
.with_local_anchor_1(Vec2::splat(half_size).adjust_precision())
.with_local_anchor_2(Vec2::new(offset, -half_size).adjust_precision()),
));
prev_entity = None;
}
}
}
}

#[derive(Component)]
struct StepText;

#[derive(Component)]
struct HashText;

fn setup_ui(mut commands: Commands) {
let font = TextFont {
font_size: 20.0,
..default()
};

commands
.spawn((
Text::new("Step: "),
font.clone(),
Node {
position_type: PositionType::Absolute,
top: Val::Px(5.0),
left: Val::Px(5.0),
..default()
},
))
.with_child((TextSpan::new("0"), font.clone(), StepText));

commands
.spawn((
Text::new("Hash: "),
font.clone(),
Node {
position_type: PositionType::Absolute,
top: Val::Px(30.0),
left: Val::Px(5.0),
..default()
},
))
.with_child((TextSpan::default(), font.clone(), HashText));

commands.spawn((
Text::new("Press R to reset scene"),
font.clone(),
Node {
position_type: PositionType::Absolute,
top: Val::Px(5.0),
right: Val::Px(5.0),
..default()
},
));
}

// TODO: This should be an optimized built-in feature for joints.
fn ignore_joint_collisions(joints: Query<&RevoluteJoint>, mut collisions: ResMut<Collisions>) {
for joint in &joints {
collisions.remove_collision_pair(joint.entity1, joint.entity2);
}
}

fn clear_scene(
mut commands: Commands,
query: Query<
Entity,
Or<(
With<RigidBody>,
With<Collider>,
With<RevoluteJoint>,
With<Camera>,
)>,
>,
mut step: ResMut<Step>,
) {
step.0 = 0;
for entity in &query {
commands.entity(entity).despawn_recursive();
}
}

#[derive(Pod, Zeroable, Clone, Copy)]
#[repr(C)]
struct Isometry {
translation: Vector,
rotation: Scalar,
}

fn update_hash(
transforms: Query<(&Position, &Rotation), With<RigidBody>>,
mut step_text: Single<&mut TextSpan, With<StepText>>,
mut hash_text: Single<&mut TextSpan, (With<HashText>, Without<StepText>)>,
mut step: ResMut<Step>,
) {
step_text.0 = step.to_string();
step.0 += 1;

if step.0 > STEP_COUNT {
return;
}

let mut hash = 5381;
for (position, rotation) in &transforms {
let isometry = Isometry {
translation: position.0,
rotation: rotation.as_radians(),
};
hash = djb2_hash(hash, bytemuck::bytes_of(&isometry));
}

if step.0 == STEP_COUNT {
hash_text.0 = format!("0x{:x} (step {})", hash, step.0);
} else {
hash_text.0 = format!("0x{:x}", hash);
}
}

fn djb2_hash(mut hash: u32, data: &[u8]) -> u32 {
for &byte in data {
hash = (hash << 5).wrapping_add(hash + byte as u32);
}
hash
}

This file was deleted.

1 change: 0 additions & 1 deletion crates/avian3d/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -83,7 +83,6 @@ benches_common_3d = { path = "../benches_common_3d" }
bevy_math = { version = "0.15.0-rc", features = ["approx"] }
approx = "0.5"
criterion = { version = "0.5", features = ["html_reports"] }
insta = "1.0"
bevy_mod_debugdump = { git = "https://github.com/andriyDev/bevy_mod_debugdump", branch = "bevy-0.15" }

[[example]]
Expand Down

This file was deleted.

Loading

0 comments on commit 2563e71

Please sign in to comment.