Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: update ml system UI #12334

Merged
merged 57 commits into from
Jan 29, 2025
Merged
Changes from 1 commit
Commits
Show all changes
57 commits
Select commit Hold shift + click to select a range
7008a23
[wip] update ui changes
yoonhyejin Jan 14, 2025
6148597
fix typo
yoonhyejin Jan 14, 2025
bbcc2d4
add mlmodellineageinfo
yoonhyejin Jan 14, 2025
963b060
Merge branch 'master' into feat/update-mlflow-ui
yoonhyejin Jan 14, 2025
8dd3c25
update mapper for lineageinfo and add uni tests
yoonhyejin Jan 15, 2025
2046691
[wip] add script for creating mlmodel
yoonhyejin Jan 15, 2025
a5f9a3f
spotlessapply
yoonhyejin Jan 15, 2025
054bc1f
feat(graphql/versioning): Add versioning support to graphql; mutation…
asikowitz Jan 15, 2025
06bc8c1
add back fragments.graphql
yoonhyejin Jan 15, 2025
5a6923c
fix tests
RyanHolstien Jan 16, 2025
69da53a
Merge remote-tracking branch 'andrew-fork/versioning-graphql' into fe…
yoonhyejin Jan 16, 2025
314b869
fix fragments.graphql
yoonhyejin Jan 16, 2025
43e3d07
[wip] fix ui bugs
yoonhyejin Jan 16, 2025
e365b06
fix rendering issues
yoonhyejin Jan 17, 2025
88db564
add run event in script
yoonhyejin Jan 17, 2025
b62ef19
update script and add islinegae:True back
yoonhyejin Jan 17, 2025
b5066c6
Merge branch 'master' into feat/update-mlflow-ui
yoonhyejin Jan 17, 2025
d4dc426
add mlModel to DPIinput relationships entityTypes
yoonhyejin Jan 20, 2025
31dd659
Merge branch 'master' into feat/update-mlflow-ui
yoonhyejin Jan 20, 2025
3c1c3b7
fix container dpi preview
yoonhyejin Jan 21, 2025
cdc23a8
spotlessapply
yoonhyejin Jan 21, 2025
8a92827
Merge branch 'master' into feat/update-mlflow-ui
yoonhyejin Jan 21, 2025
0209b7e
fix prefect test
yoonhyejin Jan 21, 2025
2adba1d
fix linting
yoonhyejin Jan 21, 2025
fe1e9cd
fix linting
yoonhyejin Jan 21, 2025
757b914
fix yarn lint
yoonhyejin Jan 21, 2025
c4999be
fix ruff check
yoonhyejin Jan 21, 2025
7a3bdf5
[wip] fix linting
yoonhyejin Jan 21, 2025
f7336bc
fix linting
yoonhyejin Jan 21, 2025
a4c600c
Merge branch 'master' into feat/update-mlflow-ui
yoonhyejin Jan 22, 2025
da7aee9
update golden files for dbt & fivetran
yoonhyejin Jan 22, 2025
534e3b4
Merge branch 'master' into feat/update-mlflow-ui
yoonhyejin Jan 22, 2025
01f17c3
update golden files for airflow
yoonhyejin Jan 22, 2025
24a9070
update golden files for dags
yoonhyejin Jan 22, 2025
28359e3
fix airflow tests to include dataplatforminstance
yoonhyejin Jan 22, 2025
43fe5e2
fix linitng
yoonhyejin Jan 22, 2025
0501a2b
hacky fix for search filters test
shirshanka Jan 22, 2025
398450c
fix searchfilters cypress test
shirshanka Jan 22, 2025
8cdc27d
some golden file updates
shirshanka Jan 23, 2025
c52a462
Add argparser for token (#6)
ryota-cloud Jan 23, 2025
b198047
update goldens
shirshanka Jan 23, 2025
e48a7a6
fix preview entity
yoonhyejin Jan 23, 2025
426d80c
Merge branch 'master' into feat/update-mlflow-ui
yoonhyejin Jan 23, 2025
0d99464
Merge branch 'master' into feat/update-mlflow-ui
yoonhyejin Jan 24, 2025
2f4aaea
remove commented out code
yoonhyejin Jan 24, 2025
9f7f298
fix empty ml model message
yoonhyejin Jan 24, 2025
946854c
remove console logs
yoonhyejin Jan 24, 2025
fa713df
pr feedback pair programming
asikowitz Jan 28, 2025
a1d48c2
revert metadata-ingestion/ & metadata-ingestion-modules/ files to head
yoonhyejin Jan 28, 2025
8a42b7e
revert some of the changes manually
yoonhyejin Jan 28, 2025
a33ef73
revert some of the changes manually
yoonhyejin Jan 28, 2025
d4af718
update stylings
yoonhyejin Jan 28, 2025
07adba4
Merge branch 'master' into feat/update-mlflow-ui
yoonhyejin Jan 28, 2025
e43ed52
Merge branch 'master' into feat/update-mlflow-ui
yoonhyejin Jan 28, 2025
a1c0125
revert search filters
asikowitz Jan 28, 2025
dfd66bf
fix properties.name mapping
yoonhyejin Jan 28, 2025
2b9a418
fix yarnlint
yoonhyejin Jan 29, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
fix properties.name mapping
yoonhyejin committed Jan 28, 2025
commit dfd66bfad5a7fa5c8d79b9a2f95baaa2779c7c60
Original file line number Diff line number Diff line change
@@ -147,9 +147,7 @@ private void mapDataProcessProperties(
dataProcessInstanceProperties.getCustomProperties(), entityUrn));
}
if (dataProcessInstanceProperties.hasCreated()) {
com.linkedin.datahub.graphql.generated.AuditStamp created =
AuditStampMapper.map(context, dataProcessInstanceProperties.getCreated());
properties.setCreated(created);
dpi.setCreated(AuditStampMapper.map(context, dataProcessInstanceProperties.getCreated()));
}
dpi.setProperties(properties);
}
Original file line number Diff line number Diff line change
@@ -40,6 +40,12 @@ public MLModelGroupProperties apply(
result.setCreated(
TimeStampToAuditStampMapper.map(context, mlModelGroupProperties.getCreated()));
}
if (mlModelGroupProperties.getName() != null) {
result.setName(mlModelGroupProperties.getName());
} else {
// backfill name from URN for backwards compatibility
result.setName(entityUrn.getEntityKey().get(1)); // indexed access is safe here
}

if (mlModelGroupProperties.hasLastModified()) {
result.setLastModified(
Original file line number Diff line number Diff line change
@@ -19,6 +19,9 @@ public void testMapMLModelGroupProperties() throws URISyntaxException {
// Set description
input.setDescription("a ml trust model group");

// Set Name
input.setName("ML trust model group");

// Create URN
Urn groupUrn =
Urn.createFromString(
@@ -31,6 +34,7 @@ public void testMapMLModelGroupProperties() throws URISyntaxException {
// Verify mapped properties
assertNotNull(result);
assertEquals(result.getDescription(), "a ml trust model group");
assertEquals(result.getName(), "ML trust model group");

// Verify lineage info is null as in the mock data
assertNotNull(result.getMlModelLineageInfo());
Original file line number Diff line number Diff line change
@@ -21,7 +21,7 @@ export const Preview = ({
return (
<DefaultPreviewCard
url={entityRegistry.getEntityUrl(EntityType.Mlmodel, model.urn)}
name={model.name || ''}
name={model.properties?.['propertiesName'] || model.name || ''}
urn={model.urn}
description={model.description || ''}
platformInstanceId={model.dataPlatformInstance?.instanceId}
Original file line number Diff line number Diff line change
@@ -139,7 +139,7 @@ export class MLModelGroupEntity implements Entity<MlModelGroup> {
};

displayName = (data: MlModelGroup) => {
return data.name || data.urn;
return data.properties?.name || data.name || data.urn;
};

getGenericEntityProperties = (mlModelGroup: MlModelGroup) => {
Original file line number Diff line number Diff line change
@@ -19,7 +19,7 @@ export const Preview = ({
return (
<DefaultPreviewCard
url={entityRegistry.getEntityUrl(EntityType.MlmodelGroup, group.urn)}
name={group?.name || ''}
name={group?.properties?.['propertiesName'] || group?.name || ''}
urn={group.urn}
platformInstanceId={group.dataPlatformInstance?.instanceId}
description={group?.description || ''}
Original file line number Diff line number Diff line change
@@ -67,134 +67,134 @@
export default function MLGroupModels() {
const baseEntity = useBaseEntity<GetMlModelGroupQuery>();
const entityRegistry = useEntityRegistry();
const modelGroup = baseEntity?.mlModelGroup;

const models =
baseEntity?.mlModelGroup?.incoming?.relationships
?.map((relationship) => relationship.entity)
.filter(notEmpty) || [];

const columns = [
{
title: 'Name',
dataIndex: 'name',
key: 'name',
width: 300,

Check warning on line 82 in datahub-web-react/src/app/entity/mlModelGroup/profile/ModelGroupModels.tsx

Codecov / codecov/patch

datahub-web-react/src/app/entity/mlModelGroup/profile/ModelGroupModels.tsx#L70-L82

Added lines #L70 - L82 were not covered by tests
render: (name, record) => (
render: (_: any, record) => (
<NameContainer>

Check warning on line 84 in datahub-web-react/src/app/entity/mlModelGroup/profile/ModelGroupModels.tsx

Codecov / codecov/patch

datahub-web-react/src/app/entity/mlModelGroup/profile/ModelGroupModels.tsx#L84

Added line #L84 was not covered by tests
<NameLink href={entityRegistry.getEntityUrl(EntityType.Mlmodel, record.urn)}>{name}</NameLink>
<NameLink href={entityRegistry.getEntityUrl(EntityType.Mlmodel, record.urn)}>{record?.properties?.propertiesName || record?.name }</NameLink>
</NameContainer>
),
},
{
title: 'Version',
key: 'version',
width: 70,
render: (_: any, record: any) => (
<VersionContainer>{record.versionProperties?.version?.versionTag || '-'}</VersionContainer>
),
},
{
title: 'Created At',
key: 'createdAt',
width: 150,
render: (_: any, record: any) => (
<Typography.Text>
{record.properties?.createdTS?.time
? moment(record.properties.createdTS.time).format('YYYY-MM-DD HH:mm:ss')
: '-'}
</Typography.Text>
),
},
{
title: 'Aliases',
key: 'aliases',
width: 200,
render: (_: any, record: any) => {
const aliases = record.versionProperties?.aliases || [];

return (
<TagContainer>
{aliases.map((alias) => (
<Pill
key={alias.versionTag}
label={alias.versionTag}
colorScheme="blue"
clickable={false}
/>
))}
</TagContainer>
);
},
},
{
title: 'Tags',
key: 'tags',
width: 200,
render: (_: any, record: any) => {
const tags = record.properties?.tags || [];

return (
<TagContainer>
{tags.map((tag) => (
<Pill key={tag} label={tag} clickable={false} />
))}
</TagContainer>
);
},
},
{
title: 'Description',
dataIndex: 'description',
key: 'description',
width: 300,
render: (_: any, record: any) => {
const editableDesc = record.editableProperties?.description;
const originalDesc = record.description;

return <Typography.Text>{editableDesc || originalDesc || '-'}</Typography.Text>;
},
},
];

Check warning on line 158 in datahub-web-react/src/app/entity/mlModelGroup/profile/ModelGroupModels.tsx

Codecov / codecov/patch

datahub-web-react/src/app/entity/mlModelGroup/profile/ModelGroupModels.tsx#L86-L158

Added lines #L86 - L158 were not covered by tests

return (
<ModelsContainer>
<Typography.Title level={3}>Model Group Details</Typography.Title>
<InfoItemContainer justifyContent="left">
<InfoItem title="Created At">
<InfoItemContent>
{modelGroup?.properties?.created?.time
? moment(modelGroup.properties.created.time).format('YYYY-MM-DD HH:mm:ss')
: '-'}
</InfoItemContent>
</InfoItem>
<InfoItem title="Last Modified At">
<InfoItemContent>
{modelGroup?.properties?.lastModified?.time
? moment(modelGroup.properties.lastModified.time).format('YYYY-MM-DD HH:mm:ss')
: '-'}
</InfoItemContent>
</InfoItem>
{modelGroup?.properties?.created?.actor && (
<InfoItem title="Created By">
<InfoItemContent>{modelGroup.properties.created?.actor}</InfoItemContent>
</InfoItem>
)}
</InfoItemContainer>
<Typography.Title level={3}>Models</Typography.Title>
<StyledTable
columns={columns}
dataSource={models}
pagination={false}
rowKey="urn"
expandable={{
defaultExpandAllRows: true,
expandRowByClick: true,
}}
locale={{
emptyText: <EmptyTab tab="mlModel" />,
}}
/>
</ModelsContainer>

Check warning on line 198 in datahub-web-react/src/app/entity/mlModelGroup/profile/ModelGroupModels.tsx

Codecov / codecov/patch

datahub-web-react/src/app/entity/mlModelGroup/profile/ModelGroupModels.tsx#L161-L198

Added lines #L161 - L198 were not covered by tests
);
}
2 changes: 1 addition & 1 deletion datahub-web-react/src/app/shared/time/timeUtils.tsx
Original file line number Diff line number Diff line change
@@ -208,39 +208,39 @@
}

export function formatDuration(durationMs: number): string {
const duration = moment.duration(durationMs);
const hours = Math.floor(duration.asHours());
const minutes = duration.minutes();
const seconds = duration.seconds();

if (hours === 0 && minutes === 0) {

Check warning on line 216 in datahub-web-react/src/app/shared/time/timeUtils.tsx

Codecov / codecov/patch

datahub-web-react/src/app/shared/time/timeUtils.tsx#L211-L216

Added lines #L211 - L216 were not covered by tests
return `${seconds}secs`;
return `${seconds} secs`;
}

if (hours === 0) {
return minutes === 1 ? `${minutes} min` : `${minutes} mins`;
}

const minuteStr = minutes > 0 ? ` ${minutes} mins` : '';
return hours === 1 ? `${hours} hr${minuteStr}` : `${hours} hrs${minuteStr}`;
}

Check warning on line 226 in datahub-web-react/src/app/shared/time/timeUtils.tsx

Codecov / codecov/patch

datahub-web-react/src/app/shared/time/timeUtils.tsx#L218-L226

Added lines #L218 - L226 were not covered by tests

export function formatDetailedDuration(durationMs: number): string {
const duration = moment.duration(durationMs);
const hours = Math.floor(duration.asHours());
const minutes = duration.minutes();
const seconds = duration.seconds();

const parts: string[] = [];

if (hours > 0) {
parts.push(hours === 1 ? `${hours} hr` : `${hours} hrs`);
}
if (minutes > 0) {
parts.push(minutes === 1 ? `${minutes} min` : `${minutes} mins`);
}
if (seconds > 0) {
parts.push(`${seconds} secs`);
}
return parts.join(' ');
}

Check warning on line 246 in datahub-web-react/src/app/shared/time/timeUtils.tsx

Codecov / codecov/patch

datahub-web-react/src/app/shared/time/timeUtils.tsx#L229-L246

Added lines #L229 - L246 were not covered by tests
2 changes: 2 additions & 0 deletions datahub-web-react/src/graphql/lineage.graphql
Original file line number Diff line number Diff line change
@@ -272,6 +272,7 @@ fragment lineageNodeProperties on EntityWithRelationships {
removed
}
properties {
propertiesName: name
createdTS: created {
time
actor
@@ -322,6 +323,7 @@ fragment lineageNodeProperties on EntityWithRelationships {
comment
}
properties {
propertiesName: name
createdTS: created {
time
actor
6 changes: 6 additions & 0 deletions datahub-web-react/src/graphql/search.graphql
Original file line number Diff line number Diff line change
@@ -886,6 +886,9 @@ fragment searchResultsWithoutSchemaField on Entity {
...structuredPropertiesFields
}
}
properties {
propertiesName: name
}
}
... on MLModelGroup {
name
@@ -908,6 +911,9 @@ fragment searchResultsWithoutSchemaField on Entity {
...structuredPropertiesFields
}
}
properties {
propertiesName: name
}
}
... on Tag {
name
Loading