Skip to content

feat(sdk) add dashboard & chart entity #13669

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 9 commits into from
Jun 6, 2025
Merged

Conversation

yoonhyejin
Copy link
Collaborator

@yoonhyejin yoonhyejin commented Jun 2, 2025

Adds Dashboard & Charts entity to SDK.

Supported actions

  • add input datasets to chart ( == upstream dataset)
  • add input datasets to dashboard ( == upstream dataset)
  • add chart to dashboard ( "contents" of a dashboard)
  • add dashboards to dashboard ( == upstream dashboard)

@github-actions github-actions bot added the ingestion PR or Issue related to the ingestion of metadata label Jun 2, 2025
Copy link

codecov bot commented Jun 2, 2025

Codecov Report

Attention: Patch coverage is 84.53608% with 60 lines in your changes missing coverage. Please review.

✅ All tests successful. No failed tests found.

Files with missing lines Patch % Lines
metadata-ingestion/src/datahub/sdk/dashboard.py 78.44% 47 Missing ⚠️
metadata-ingestion/src/datahub/sdk/chart.py 91.66% 13 Missing ⚠️

📢 Thoughts on this report? Let us know!

Copy link

alwaysmeticulous bot commented Jun 2, 2025

🔴 Meticulous spotted visual differences in 4 of 1347 screens tested: view and approve differences detected.

Meticulous evaluated ~10 hours of user flows against your PR.

Last updated for commit 2a309f9. This comment will update as new commits are pushed.

Copy link

codecov bot commented Jun 2, 2025

Bundle Report

Changes will decrease total bundle size by 3.76kB (-0.02%) ⬇️. This is within the configured threshold ✅

Detailed changes
Bundle name Size Change
datahub-react-web-esm 19.65MB -3.76kB (-0.02%) ⬇️

Affected Assets, Files, and Routes:

view changes for bundle: datahub-react-web-esm

Assets Changed:

Asset Name Size Change Total Size Change (%)
assets/index-*.js -3.76kB 16.02MB -0.02%

@yoonhyejin yoonhyejin marked this pull request as ready for review June 2, 2025 11:12
@yoonhyejin yoonhyejin requested a review from Copilot June 2, 2025 11:12
Copy link

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR adds new Dashboard and Chart entities to the SDK along with their respective tests, golden files, and example usage. Key changes include:

  • New tests and golden files for Dashboard and Chart entities
  • Updates to the entity client, SDK internals, and examples for handling Dashboard and Chart operations
  • Adjusted type aliases and entity registry to support new entities

Reviewed Changes

Copilot reviewed 16 out of 16 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
tests/unit/sdk_v2/test_dashboard.py Adds basic and complex tests for Dashboard entity functionality.
tests/unit/sdk_v2/test_chart.py Adds tests for Chart entity functionality including input dataset operations.
dashboard_golden and chart_golden JSON files Provide expected golden outputs for Dashboard and Chart entities.
src/datahub/sdk/entity_client.py Adds overloads for DashboardUrn and ChartUrn to support the new entities.
src/datahub/sdk/chart.py Implements the Chart entity, including methods for setting properties and managing input datasets.
src/datahub/sdk/_shared.py Updates type aliases to include ChartUrn and DashboardUrn.
src/datahub/sdk/_all_entities.py & init.py Registers and exports the new Dashboard and Chart classes.
examples/library/create_dashboard_complex.py Example demonstrating complex Dashboard creation with relationships.
examples/library/create_dashboard.py Basic Dashboard creation example.
examples/library/create_chart_complex.py Complex Chart example showing input dataset operations.
examples/library/create_chart.py Basic Chart creation example.
Comments suppressed due to low confidence (1)

metadata-ingestion/examples/library/create_dashboard_complex.py:30

  • The Dashboard constructor is passed an extra 'dashboards' field which is not supported by the current Dashboard API. Consider removing this parameter or using a supported method to set related dashboards.
dashboards=[dashboard1.urn],

@datahub-cyborg datahub-cyborg bot added the needs-review Label for PRs that need review from a maintainer. label Jun 2, 2025
@datahub-cyborg datahub-cyborg bot added pending-submitter-response Issue/request has been reviewed but requires a response from the submitter and removed needs-review Label for PRs that need review from a maintainer. labels Jun 2, 2025
Copy link
Contributor

@sgomezvillamor sgomezvillamor left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall it looks great!
I left some minor suggestion and a question about patching

@datahub-cyborg datahub-cyborg bot added needs-review Label for PRs that need review from a maintainer. and removed pending-submitter-response Issue/request has been reviewed but requires a response from the submitter labels Jun 3, 2025
client.entities.upsert(dataset)

# add a new dataset to the chart
chart.add_input_dataset(input_datasets[2])
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

input_datasets[2] >> chart

Do you think an operator like that would improve clarity? 🤔

Just an idea that came to mind. Could be backlog material if it seems useful.

@datahub-cyborg datahub-cyborg bot removed the needs-review Label for PRs that need review from a maintainer. label Jun 4, 2025
@datahub-cyborg datahub-cyborg bot added the pending-submitter-response Issue/request has been reviewed but requires a response from the submitter label Jun 4, 2025
Copy link
Contributor

@sgomezvillamor sgomezvillamor left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just did a more detailed review and left a few comments. I might’ve been a bit picky in some cases, just trying to be helpful 😅

One thought that came to mind while reviewing: for the lineage methods, we currently have separate set|add|remove methods for each entity. eg add_input_datasets, add_charts, etc.

Have we considered a more generic approach like a unique add|set method that takes a union of supported types and internally figures out where to add them? I understand this might not work as well for set operations, but I thought it might be worth sharing in case it wasn’t explored during the design discussions.

Copy link
Collaborator

@hsheth2 hsheth2 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Had a couple things to add on top of sergio's thorough review

@datahub-cyborg datahub-cyborg bot added needs-review Label for PRs that need review from a maintainer. and removed pending-submitter-response Issue/request has been reviewed but requires a response from the submitter labels Jun 5, 2025
Copy link
Contributor

@sgomezvillamor sgomezvillamor left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🏆

@datahub-cyborg datahub-cyborg bot added pending-submitter-merge and removed needs-review Label for PRs that need review from a maintainer. labels Jun 6, 2025
@yoonhyejin yoonhyejin merged commit e82cc66 into master Jun 6, 2025
62 checks passed
@yoonhyejin yoonhyejin deleted the sdk/dashboard-entity branch June 6, 2025 07:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ingestion PR or Issue related to the ingestion of metadata pending-submitter-merge
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants