UI hangs rendering Global Asset Lineage of only 200 assets

[cohenj20]

Dagster version

dagster, version 1.6.11

What's the issue?

I'm running into a strange dagster UI problem.

I'm using dagster to visualize a dependency graph for objects in my company's DW. I essentially am using an asset factory pattern that receives a csv of different objects and their parents and iteratively generates assets with the proper dependencies. The dagster deployment loads without issues, and navigating the assets within the deployment I can see that the asset factory correctly grouped assets and linked them to their parents/children. When I go to load the global lineage graph, however, graph hangs on "Rendering a large number of assets, please wait…" and never renders.

What did you expect to happen?

I expect that the lineage graph renders quickly, as according to this blogpost dagster should be capable of rapidly rendering far more than 200 assets now.

How to reproduce?

Use the below asset factory code (asset.py) and the __init__.py definition code below in a generic dagster project, placing the attached test_hierarchy.csv in the directory and startup the UI. Go to global lineage graph to see whether the assets render.

test_hierarchy.csv

# assets.py

from importlib.metadata import metadata
import json
import os
from tokenize import group
from unicodedata import name

from dagster import asset, Output, MetadataValue, AssetExecutionContext # import the `dagster` library
import pandas as pd
from pandas import DataFrame
from sqlalchemy import create_engine

view_df = pd.read_csv('../test_hierarchy.csv')

single_parent_df = view_df[view_df['parent_count'] == 1]

multiple_parent_df = view_df[view_df['parent_count'] > 1]

assets = []

for i, view in single_parent_df.iterrows():
    if single_parent_df['parentname'][i] == 'None':
        @asset(name=single_parent_df['objectname'][i], group_name=single_parent_df['objectschema'][i]) # add the asset decorator to tell Dagster this is an asset
        def compute(context: AssetExecutionContext):

            context.add_output_metadata({
                "data_preview" : MetadataValue.md(view_df.head().to_markdown())
            })

        assets.append(compute)
    else:
        @asset(name=single_parent_df['objectname'][i], deps=[single_parent_df['parentname'][i]], group_name=single_parent_df['objectschema'][i]) # add the asset decorator to tell Dagster this is an asset
        def compute(context: AssetExecutionContext):

            context.add_output_metadata({
                "data_preview" : MetadataValue.md(view_df.head().to_markdown())
            })
        
        assets.append(compute)


for object in list(multiple_parent_df['objectname'].unique()):

    deps = list(view_df[view_df['objectname'] == object]['parentname'])

    object_schema = list(multiple_parent_df[multiple_parent_df['objectname'] == object]['objectschema'])[0]

    @asset(name=object, deps=deps, group_name=object_schema) # add the asset decorator to tell Dagster this is an asset
    def compute(context: AssetExecutionContext):
        # table_preview = pd.read_sql(f"SELECT TOP 10 * FROM {view_df['objectschema'][i]}.{view_df['objectname'][i]}", con)

        pass

    assets.append(compute)

# __init__.py

from dagster import Definitions, load_assets_from_modules

from .assets import assets

all_assets = load_assets_from_modules([assets])

defs = Definitions(
    assets=(assets),
)

Deployment type

Local

Deployment details

Running dagster locally. Using a python venv.

Additional information

No response

Message from the maintainers

Impacted by this issue? Give it a 👍! We factor engagement into prioritization.

[bengotow]

Hey folks, I pulled down the sample above (thank you for the clear repro!) and loaded it up, but I'm not able to reproduce the UI hanging on the latest master branch. I tried opening/closing the asset groups, etc. We did update Dagre in the last two weeks, so it's possible that impacted this behavior. I'm going to revert that dagre upgrade and see if this repro'd with the old version.

[cohenj20]

Hey @bengotow

We did update Dagre in the last two weeks, so it's possible that impacted this behavior. I'm going to revert that dagre upgrade and see if this repro'd with the old version.

Was there any change in the UI behavior after reverting?

[cohenj20]

I uninstalled and re-installed dagster, dagster-webserver, and dagster-pandas and the problem was resolved.

[cohenj20]

Unfortunately, in the test I ran before closing the issue I used the wrong dependency table (not the one I provided for reproduction when I opened the issue). Usig the proper data set the problem seems to have persisted through the version upgrades.

[salazarm]

Hi @cohenj20, We're aware of an issue with dagre (the layout library we use) and are trying to figure out how to mitigate it. For now the best I can offer is a work-around. If you update the URl to be /asset-groups?expanded=fake (The key here is the expanded=fake) this will reset all of the groups to closed. The issue only happens when certain combinations of groups are open so try not to open all of the groups at once.

[cohenj20]

I did what you suggested, however the UI still hangs. I try to chnge the URL to http://127.0.0.1:3000/asset-groups?expanded=fake but it changes directly to http://127.0.0.1:3000/asset-groups?expanded=fake&open-nodes%5B%5D=dagster_project&open-nodes%5B%5D=dagster_project%3Aetl&open-nodes%5B%5D=dagster_project%3Adbo&open-nodes%5B%5D=dagster_project%3Areport&open-nodes%5B%5D=dagster_project%3Ainterface&open-nodes%5B%5D=dagster_project%3Alive.

What could be the reason that in @bengotow 's environment the problem didn't present itself?

[salazarm]

@cohenj20 Try /asset-groups?expanded=fake&open-nodes%5B%5D=fake

[cohenj20]

@salazarm No dice :/

I also tried creating a fresh dagster project, but it still hangs.

[cohenj20]

@salazarm I may have discovered something...

I was messing around with which combinations of asset groups cause the issue, and I noticed that if the report asset group is selected, even if it's the only asset group selected, the UI hangs. As long as report is unselected, all assets display instantly on the graph.

Update: This makes sense, on second thought, as the sum of assets in report is more than all of the others combined.

[salazarm]

@cohenj20 Do you see any errors in the developer console? If so can you share them here

[cohenj20]

@salazarm I do not see anything.

[salazarm]

Would you be open to a zoom call so I can take a look? Keen on getting to the bottom of this.

[salazarm]

I was messing around with which combinations of asset groups cause the issue, and I noticed that if the report asset group is selected, even if it's the only asset group selected, the UI hangs. As long as report is unselected, all assets display instantly on the graph.

I will pull on this thread

[cohenj20]

@salazarm If we can do it in an hour, sure. I need to step away for a bit.

[salazarm]

@cohenj20 Yes that would work for me. Feel free to pop into https://meet.google.com/zje-xrev-vig?hs=122&authuser=0, I'll be waiting in there.

[salazarm]

@cohenj20 I believe the issue you're facing is that the screen just looks blank? Correct? If so, I think there isn't actually a bug, it's just the default zooming took you to a spot where there where no assets. Try clicking on an asset in the sidebar to see if it zooms you to it.

[cohenj20]

For visibility, on the zoom call @salazarm and I confirmed there was the following error in the developer console:

Uncaught DOMException: Failed to execute 'importScripts' on 'WorkerGlobalScope': The script at 'http://127.0.0.1:3000/_next/static/chunks/framework-d6095a5336d3e425.js' failed to load.
    at __webpack_require__.f.i (http://127.0.0.1:3000/_next/static/chunks/358.85428c524e2522c9.js:1:2431)
    at http://127.0.0.1:3000/_next/static/chunks/358.85428c524e2522c9.js:1:1317
    at Array.reduce (<anonymous>)
    at __webpack_require__.e (http://127.0.0.1:3000/_next/static/chunks/358.85428c524e2522c9.js:1:1265)
    at Array.map (<anonymous>)
    at __webpack_require__.x (http://127.0.0.1:3000/_next/static/chunks/358.85428c524e2522c9.js:1:2853)
    at http://127.0.0.1:3000/_next/static/chunks/358.85428c524e2522c9.js:1:2935
    at http://127.0.0.1:3000/_next/static/chunks/358.85428c524e2522c9.js:1:2939

[salazarm]

This will be fixed by #19046

[salazarm]

#21129 *