Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[13] refact!: table removal #270

Merged
merged 90 commits into from
Aug 30, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
90 commits
Select commit Hold shift + click to select a range
ba8bf0e
components/actions refactors
v-rocheleau Jul 4, 2023
4975cf2
Merge branch 'master' into features/table-removal
v-rocheleau Jul 5, 2023
6e9b316
remove tables from redux and react
v-rocheleau Jul 5, 2023
17d24c6
data types manager view
v-rocheleau Jul 6, 2023
6cd1a38
chore: begin work on new ingestion process
davidlougheed Jul 10, 2023
8d0bc52
delete dt from ds
v-rocheleau Jul 10, 2023
cc82448
get dataset data-type summary
v-rocheleau Jul 17, 2023
8960bf9
working data types component with deletion modal
v-rocheleau Jul 18, 2023
53565d1
data-type summary modals
v-rocheleau Jul 19, 2023
39274b3
lint
v-rocheleau Jul 19, 2023
e48d56f
review refactors
v-rocheleau Jul 19, 2023
2171ebf
redux store by dataset id
v-rocheleau Jul 20, 2023
a520eb2
lint
v-rocheleau Jul 20, 2023
3f98eb5
Merge branch 'features/table-removal-wes' into features/table-removal
v-rocheleau Jul 20, 2023
83bf21f
workflow dataset selection
v-rocheleau Jul 27, 2023
27f1c1c
Merge branch 'master' into features/table-removal
v-rocheleau Aug 1, 2023
65f58d1
submit worflows to wes
v-rocheleau Aug 1, 2023
960fd5f
ingestion url workflow param
v-rocheleau Aug 3, 2023
aca3586
workflow selection refactors
v-rocheleau Aug 9, 2023
656680c
lint fix
v-rocheleau Aug 9, 2023
f786d6f
ingest from dropbox
v-rocheleau Aug 10, 2023
9eb8dbf
fix data-type summaries
v-rocheleau Aug 10, 2023
f98dd45
lint fix
v-rocheleau Aug 10, 2023
6c199c8
lint: address review comment
davidlougheed Aug 17, 2023
30c9c08
refact: rename delete to clear data for data type and add a useCallba…
davidlougheed Aug 17, 2023
6356880
style: don't use monospace for data type label
davidlougheed Aug 17, 2023
44e580e
chore: disable clear data button if 0-count
davidlougheed Aug 17, 2023
be93967
refact: memorize a modal onCancel handler
davidlougheed Aug 17, 2023
66db573
refact: memoization in DatasetDataTypes
davidlougheed Aug 17, 2023
3f3b5b1
lint
davidlougheed Aug 17, 2023
9a65378
chore: rm service url from tags
davidlougheed Aug 21, 2023
0bdf545
fix(style): overflow issue in dataset data types display
davidlougheed Aug 21, 2023
38ab0cd
chore: remove dead tabletreeselect component
davidlougheed Aug 21, 2023
e3c7ddd
refact: remove mentions of tables
davidlougheed Aug 21, 2023
6239c79
refact: display project/dataset/data type info in runrequest instead …
davidlougheed Aug 21, 2023
67a226d
refact: data type counting in dataset overview
davidlougheed Aug 21, 2023
cb0ef56
lint
davidlougheed Aug 21, 2023
393b421
refact: convert RunLastContent to use dataset IDs
davidlougheed Aug 21, 2023
c24de81
chore: re-enable last ingestion display
davidlougheed Aug 21, 2023
e9ccb42
fix: a useSelector in LastIngestionTable
davidlougheed Aug 21, 2023
487c248
style: tweak last ingest table / run list spacing + disable pagination
davidlougheed Aug 21, 2023
4ffd586
Merge remote-tracking branch 'origin/master' into features/table-removal
davidlougheed Aug 21, 2023
f6bdbd9
style: wording for data type dropdown
davidlougheed Aug 21, 2023
fa05634
fix: memoize run setup inputs table + don't show hidden inputs
davidlougheed Aug 21, 2023
03b8908
fix: invalid date showing in workflow run detail
davidlougheed Aug 21, 2023
739a6f6
fix disable clear data type
v-rocheleau Aug 22, 2023
b2e639a
variants data-types redux store
v-rocheleau Aug 23, 2023
2b91620
lint
v-rocheleau Aug 23, 2023
b5b6c48
variants dataset summary data in redux store
v-rocheleau Aug 23, 2023
ac90f86
data-type select with labels
v-rocheleau Aug 23, 2023
cb7a942
service url selection from data type when clearing dataset data
v-rocheleau Aug 24, 2023
ae8037d
Implement function to retrieve datasets by ID
noctillion Aug 24, 2023
9c96d42
Dispatch dataType information by dataset retrival
noctillion Aug 24, 2023
9da056d
Pass activeDataset as prop
noctillion Aug 24, 2023
29053f2
Implement filter datatypes by dataset
noctillion Aug 24, 2023
5800c70
Implement activate dataTypes tabs by dataset
noctillion Aug 24, 2023
454621d
lint
noctillion Aug 24, 2023
b29cadd
refactor out private path prefix from aggregation search endpoint
v-rocheleau Aug 24, 2023
a5cf9b0
lint
v-rocheleau Aug 24, 2023
5ef23db
Merge branch 'features/table-removal' into features/table-removal-dat…
noctillion Aug 25, 2023
6787c58
chore: use self-declared data service status for services
davidlougheed Aug 25, 2023
2929e75
Optimize dataset fetching to run concurrently
noctillion Aug 26, 2023
b380d6b
Reformat line
noctillion Aug 26, 2023
3432f3b
chore: move datasetsByID calculation to redux store & reorganize meta…
davidlougheed Aug 28, 2023
8acef26
lint
davidlougheed Aug 28, 2023
2e67a36
remove faulty disable condition
v-rocheleau Aug 28, 2023
e953069
fix datasets reducer
v-rocheleau Aug 29, 2023
b7d5cc6
datasets data-types actions refactor
v-rocheleau Aug 29, 2023
407eba9
lint
v-rocheleau Aug 29, 2023
80c856c
simplify fetchUserDependentData
v-rocheleau Aug 29, 2023
cb48d88
lint
v-rocheleau Aug 29, 2023
3bc70bd
Merge pull request #283 from bento-platform/features/table-removal-da…
v-rocheleau Aug 29, 2023
6c0bfe6
refact: memoization in DatasetSelectionModal
davidlougheed Aug 29, 2023
e0fe672
refact: use state datasetsByID in ManagerIngestionContent
davidlougheed Aug 29, 2023
c0843f0
refact: component for showing project/dataset title in ManagerIngesti…
davidlougheed Aug 29, 2023
eacd94b
lint: rm commented-out code
davidlougheed Aug 29, 2023
cd3b92a
refact: use redux datasetsByID in ExplorerDatasetSearch
davidlougheed Aug 29, 2023
fab56e4
lint: rm useless fragments in DataTypeSummaryModal
davidlougheed Aug 29, 2023
1b26068
lint: DataTypeExplorationModal
davidlougheed Aug 29, 2023
0f6539d
lint: DataTypeSelect
davidlougheed Aug 29, 2023
9b0b54a
Merge remote-tracking branch 'origin/master' into features/table-removal
davidlougheed Aug 30, 2023
44dc496
chore: tweak rendering + memoize biosample/experiment/individual tabl…
davidlougheed Aug 30, 2023
25c0b1b
lint
davidlougheed Aug 30, 2023
68d785e
refact: move search results table to folder
davidlougheed Aug 30, 2023
9f6bee7
refact: memoization + lint for ExplorerSearchResultsTable
davidlougheed Aug 30, 2023
89beb20
fix: rm alternateIds from biosample ID render
davidlougheed Aug 30, 2023
4eb8926
fix: wrong props in explorersearchresultstable
davidlougheed Aug 30, 2023
30b3f70
feat: biosample links with hashes from individuals + biosamples page
davidlougheed Aug 30, 2023
d2fc2fe
feat: individual id link component used in biosamples table too
davidlougheed Aug 30, 2023
af90344
Merge pull request #284 from bento-platform/feat/explorer/table-impro…
davidlougheed Aug 30, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 15 additions & 0 deletions src/components/ConfirmationModal.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
import { Modal } from "antd";

const { confirm } = Modal;

const genericConfirm = ({title, content, onOk, onCancel, ...rest}) => {
confirm({
title: title,
content: content,
onOk: onOk,
onCancel,
...rest,
});
};

export default genericConfirm;
1 change: 0 additions & 1 deletion src/components/ServiceContent.js
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,6 @@ const ServiceContent = () => {
/>
</Spin>
</Col>
{/* TODO: Tables */}
</Row>
<Typography.Title level={3}>Services</Typography.Title>
<ServiceList />
Expand Down
23 changes: 13 additions & 10 deletions src/components/ServiceList.js
Original file line number Diff line number Diff line change
Expand Up @@ -215,16 +215,19 @@ const ServiceList = () => {
const [requestModalService, setRequestModalService] = useState(null);

const dataSource = useSelector((state) =>
Object.entries(state.bentoServices.itemsByKind).map(([kind, service]) => ({
...service,
key: kind,
serviceInfo: state.services.itemsByKind[kind] ?? null,
status: {
status: kind in state.services.itemsByKind,
dataService: service.data_service,
},
loading: state.services.isFetching,
})),
Object.entries(state.bentoServices.itemsByKind).map(([kind, service]) => {
const serviceInfo = state.services.itemsByKind[kind] ?? null;
return {
...service,
key: kind,
serviceInfo,
status: {
status: kind in state.services.itemsByKind,
dataService: serviceInfo?.bento?.dataService,
},
loading: state.services.isFetching,
};
}),
);

const isAuthenticated = useSelector(
Expand Down
44 changes: 26 additions & 18 deletions src/components/datasets/Dataset.js
Original file line number Diff line number Diff line change
Expand Up @@ -12,19 +12,24 @@ import {
deleteDatasetLinkedFieldSetIfPossible,
} from "../../modules/metadata/actions";

import {
fetchDatasetDataTypesSummaryIfPossible,
fetchDatasetSummaryIfPossible,
} from "../../modules/datasets/actions";

import {INITIAL_DATA_USE_VALUE} from "../../duo";
import {simpleDeepCopy, nop} from "../../utils/misc";
import LinkedFieldSetTable from "./linked_field_set/LinkedFieldSetTable";
import LinkedFieldSetModal from "./linked_field_set/LinkedFieldSetModal";
import DatasetOverview from "./DatasetOverview";
import DatasetTables from "./DatasetTables";
import {FORM_MODE_ADD, FORM_MODE_EDIT} from "../../constants";
import {datasetPropTypesShape, projectPropTypesShape} from "../../propTypes";
import DatasetDataTypes from "./DatasetDataTypes";


const DATASET_CARD_TABS = [
{key: "overview", tab: "Overview"},
{key: "tables", tab: "Data Tables"},
{key: "data_types", tab: "Data Types"},
{key: "linked_field_sets", tab: "Linked Field Sets"},
{key: "data_use", tab: "Consent Codes and Data Use"},
];
Expand Down Expand Up @@ -71,7 +76,6 @@ class Dataset extends Component {
contact_info: value.contact_info || "",
data_use: simpleDeepCopy(value.data_use || INITIAL_DATA_USE_VALUE),
linked_field_sets: value.linked_field_sets || [],
tables: value.tables || [],

fieldSetAdditionModalVisible: false,

Expand All @@ -82,13 +86,20 @@ class Dataset extends Component {
},

selectedTab: "overview",
selectedTable: null,
};

this.handleFieldSetDeletion = this.handleFieldSetDeletion.bind(this);
}


componentDidMount() {
if (this.state.identifier) {
this.props.fetchDatasetSummary(this.state.identifier);
this.props.fetchDatasetDataTypesSummary(this.state.identifier);
}
}


handleFieldSetDeletion(fieldSet, index) {
const deleteModal = Modal.confirm({
title: `Are you sure you want to delete the "${fieldSet.name}" linked field set?`,
Expand Down Expand Up @@ -123,13 +134,11 @@ class Dataset extends Component {
const tabContents = {
overview: <DatasetOverview dataset={this.state}
project={this.props.project}
isPrivate={isPrivate}
isFetchingTables={this.props.isFetchingTables} />,
tables: <DatasetTables dataset={this.state}
project={this.props.project}
isPrivate={isPrivate}
isFetchingTables={this.props.isFetchingTables}
onTableIngest={this.props.onTableIngest || nop} />,
isPrivate={isPrivate} />,
data_types: <DatasetDataTypes dataset={this.state}
project={this.props.project}
isPrivate={isPrivate}
onDatasetIngest={this.props.onDatasetIngest}/>,
linked_field_sets: (
<>
<Typography.Title level={4}>
Expand Down Expand Up @@ -281,24 +290,21 @@ Dataset.propTypes = {
mode: PropTypes.oneOf(["public", "private"]),

project: projectPropTypesShape,
strayTables: PropTypes.arrayOf(PropTypes.object),

value: datasetPropTypesShape,

isFetchingTables: PropTypes.bool,

onEdit: PropTypes.func,
onTableIngest: PropTypes.func,
onDatasetIngest: PropTypes.func,

addLinkedFieldSet: PropTypes.func,
deleteProjectDataset: PropTypes.func,
deleteLinkedFieldSet: PropTypes.func,

fetchDatasetSummary: PropTypes.func,
fetchDatasetDataTypesSummary: PropTypes.func,
};

const mapStateToProps = state => ({
isFetchingTables: state.services.isFetchingAll
|| state.projectTables.isFetching
|| state.projects.isFetchingWithTables, // TODO: Hiccup
isSavingDataset: state.projects.isSavingDataset,
isDeletingDataset: state.projects.isDeletingDataset,
});
Expand All @@ -309,6 +315,8 @@ const mapDispatchToProps = (dispatch, ownProps) => ({
deleteProjectDataset: dataset => dispatch(deleteProjectDatasetIfPossible(ownProps.project, dataset)),
deleteLinkedFieldSet: (dataset, linkedFieldSet, linkedFieldSetIndex) =>
dispatch(deleteDatasetLinkedFieldSetIfPossible(dataset, linkedFieldSet, linkedFieldSetIndex)),
fetchDatasetSummary: (datasetId) => dispatch(fetchDatasetSummaryIfPossible(datasetId)),
fetchDatasetDataTypesSummary: (datasetId) => dispatch(fetchDatasetDataTypesSummaryIfPossible(datasetId)),
});

export default connect(mapStateToProps, mapDispatchToProps)(Dataset);
132 changes: 132 additions & 0 deletions src/components/datasets/DatasetDataTypes.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,132 @@
import React, {useCallback, useMemo, useState} from "react";
import { useSelector, useDispatch } from "react-redux";
import { Button, Col, Row, Table, Typography } from "antd";

import PropTypes from "prop-types";
import { datasetPropTypesShape, projectPropTypesShape } from "../../propTypes";
import { clearDatasetDataType } from "../../modules/metadata/actions";
import { fetchDatasetDataTypesSummaryIfPossible } from "../../modules/datasets/actions";
import genericConfirm from "../ConfirmationModal";
import DataTypeSummaryModal from "./datatype/DataTypeSummaryModal";
import { nop } from "../../utils/misc";

const NA_TEXT = <span style={{ color: "#999", fontStyle: "italic" }}>N/A</span>;

const DatasetDataTypes = React.memo(
({isPrivate, project, dataset, onDatasetIngest}) => {
const dispatch = useDispatch();
const datasetDataTypes = useSelector((state) => Object.values(
state.datasetDataTypes.itemsById[dataset.identifier]?.itemsById));
const datasetSummaries = useSelector((state) => state.datasetSummaries.itemsById[dataset.identifier]);
const isFetchingDataset = useSelector(
(state) => state.datasetDataTypes.itemsById[dataset.identifier]?.isFetching);

const [datatypeSummaryVisible, setDatatypeSummaryVisible] = useState(false);
const [selectedDataType, setSelectedDataType] = useState(null);

const selectedSummary = (selectedDataType !== null && datasetSummaries)
? datasetSummaries[selectedDataType.id]
: {};

const handleClearDataType = useCallback((dataType) => {
genericConfirm({
title: `Are you sure you want to delete the "${dataType.label || dataType.id}" data type?`,
content: "Deleting this means all instances of this data type contained in the dataset " +
"will be deleted permanently, and will no longer be available for exploration.",
onOk: async () => {
await dispatch(clearDatasetDataType(dataset.identifier, dataType.id));
await dispatch(fetchDatasetDataTypesSummaryIfPossible(dataset.identifier));
},
});
}, [dispatch, dataset]);

const showDatatypeSummary = useCallback((dataType) => {
setSelectedDataType(dataType);
setDatatypeSummaryVisible(true);
}, []);

const dataTypesColumns = useMemo(() => [
{
title: "Name",
key: "label",
render: (dt) =>
isPrivate ? (
<a onClick={() => showDatatypeSummary(dt)}>
{dt.label ?? NA_TEXT}
</a>
) : dt.label ?? NA_TEXT,
defaultSortOrder: "ascend",
sorter: (a, b) => a.label.localeCompare(b.label),
},
{
title: "Count",
dataIndex: "count",
render: (c) => (c ?? NA_TEXT),
},
...(isPrivate ? [
{
title: "Actions",
key: "actions",
width: 230,
render: (dt) => (
<Row gutter={10}>
<Col span={12}>
<Button
davidlougheed marked this conversation as resolved.
Show resolved Hide resolved
icon="import"
style={{ width: "100%" }}
onClick={() => (onDatasetIngest || nop)(project, dataset, dt)}
>
Ingest
</Button>
</Col>
<Col span={12}>
<Button
type="danger"
icon="delete"
disabled={ !dt.count || dt.count && dt.count === 0}
onClick={() => handleClearDataType(dt)}
style={{ width: "100%" }}
>
Clear
</Button>
</Col>
</Row>
),
},
] : null),
], [isPrivate, project, dataset, onDatasetIngest]);

const onDataTypeSummaryModalCancel = useCallback(() => setDatatypeSummaryVisible(false), []);

return (
<>
<DataTypeSummaryModal
dataType={selectedDataType}
summary={selectedSummary}
visible={datatypeSummaryVisible}
onCancel={onDataTypeSummaryModalCancel}
/>

<Typography.Title level={4}>
Data Types
</Typography.Title>

<Table
bordered
dataSource={datasetDataTypes}
rowKey="id"
columns={dataTypesColumns}
loading={isFetchingDataset}
/>
</>
);
});

DatasetDataTypes.propTypes = {
isPrivate: PropTypes.bool,
project: projectPropTypesShape,
dataset: datasetPropTypesShape,
onDatasetIngest: PropTypes.func,
};

export default DatasetDataTypes;
8 changes: 4 additions & 4 deletions src/components/datasets/DatasetFormModal.js
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ import DatasetForm from "./DatasetForm";
import {
addProjectDataset,
saveProjectDataset,
fetchProjectsWithDatasetsAndTables,
fetchProjectsWithDatasets,
} from "../../modules/metadata/actions";

import {nop} from "../../utils/misc";
Expand Down Expand Up @@ -53,7 +53,7 @@ class DatasetFormModal extends Component {
}

async handleSuccess(values) {
await this.props.fetchProjectsWithDatasetsAndTables(); // TODO: If needed / only this project...
await this.props.fetchProjectsWithDatasets(); // TODO: If needed / only this project...
await (this.props.onOk || nop)({...(this.props.initialValue || {}), values});
if ((this.props.mode || FORM_MODE_ADD) === FORM_MODE_ADD) this.form.resetFields();
}
Expand Down Expand Up @@ -107,7 +107,7 @@ DatasetFormModal.propTypes = {

addProjectDataset: PropTypes.func,
saveProjectDataset: PropTypes.func,
fetchProjectsWithDatasetsAndTables: PropTypes.func,
fetchProjectsWithDatasets: PropTypes.func,
};

const mapStateToProps = state => ({
Expand All @@ -120,5 +120,5 @@ const mapStateToProps = state => ({
export default connect(mapStateToProps, {
addProjectDataset,
saveProjectDataset,
fetchProjectsWithDatasetsAndTables,
fetchProjectsWithDatasets,
})(DatasetFormModal);
Loading