Include helper function to test chart equality?

[joelostblom]

What is your suggestion?

Would it be useful to have the equivalent of pandas assert_frame_equal for Altair charts? I was drafting a function for this to use when grading student submissions and thought maybe it would be beneficial to have more widely available.

Example usage:

chart1 = alt.Chart("data.csv").mark_point().encode(x="A:Q", y="B:Q")
chart2 = alt.Chart("data.csv").mark_point().encode(x="A:Q", y="C:Q")

assert_chart_equal(chart1, chart2)

AssertionError: Value mismatch at 'encoding.y.field': B != C

and

assert_chart_equal(chart1, chart2.mark_point(color="grey"))

AssertionError: Key mismatch: 'mark.color' was unexpected.

Function draft

This could be extended to e.g. optionally exclude keys from comparisons (such as the data/datasets which should probably be excluded by default).

import altair as alt


def assert_chart_equal(expected, actual):
    expected_dict = expected.to_dict()
    actual_dict = actual.to_dict()
    assert_dict_equal(expected_dict, actual_dict)


def assert_dict_equal(expected_dict, actual_dict, path=""):
    # Check all keys in dict1
    for key in expected_dict:
        if key not in actual_dict:
            raise AssertionError(
                f"Key mismatch: '{path + key}' was expected, but not found."
            )
        else:
            # If both values are dictionaries, recurse into them
            if isinstance(expected_dict[key], dict) and isinstance(
                actual_dict[key], dict
            ):
                assert_dict_equal(
                    expected_dict[key], actual_dict[key], path + key + "."
                )
            # Compare the values
            elif expected_dict[key] != actual_dict[key]:
                raise AssertionError(
                    f"Value mismatch at '{path + key}': {expected_dict[key]} != {actual_dict[key]}"
                )

    # Check for any extra keys in dict2
    for key in actual_dict:
        if key not in expected_dict:
            raise AssertionError(f"Key mismatch: '{path + key}' was unexpected.")

Have you considered any alternative solutions?

If we don't see a reason to include this in the main library, I can put it in altair ally, although it wouldn't be as visible or easy to access there.

There are already some packages that assert equality of dictionaries and json specs (e.g. deepdiff), but they don't have that easy to understand error messages and seem like unnecessary dependencies to take on.

[binste]

This would be very handy! One issue will be the global counter for parameter names, see #3416 where this came up for Streamlit. See here how they solved it on their side.

[mattijn]

Interesting suggestion! Where will these proposed functions be located? Within eg a new module like altair.testing?

[firasm]

I'm teaching a new undergrad course in data visualization (with Altair as the library) and this would be really useful for me to test/compare charts and plots with an autograder.

Ultimately I want something like this, from VizGrader