Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add new feature to change taxon tree type in tree viewer #5036

Merged
merged 101 commits into from
Aug 1, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
101 commits
Select commit Hold shift + click to select a range
321cb86
Add new feature to change taxon tree type in tree viewer
CarolineDenis Jun 24, 2024
0c7ada5
Add todos
CarolineDenis Jun 24, 2024
adbb505
Typo fix
CarolineDenis Jun 24, 2024
8fed4b3
Add comment
CarolineDenis Jun 24, 2024
c57a811
Merge branch 'production' into issue-4829
acwhite211 Jun 24, 2024
a520069
add special taxon trees api
acwhite211 Jun 25, 2024
76a6b1a
add permisions to api
acwhite211 Jun 25, 2024
67b29e0
New taxon def promise, return arrays of tree def
CarolineDenis Jun 25, 2024
43f94f1
Filter the different tree def
CarolineDenis Jun 25, 2024
987ffb8
Change tree type in the tree viewer
CarolineDenis Jun 26, 2024
fc36e4c
test: cache type of tree per level
CarolineDenis Jun 26, 2024
78b9bce
Add endpoint for all tree information
melton-jason Jun 27, 2024
f4dbd76
Cleanup and refactor frontend code
melton-jason Jun 27, 2024
dcc229a
Add comment to resourceApi.ts
melton-jason Jun 27, 2024
c64b8b6
Merge branch 'production' into issue-4829
melton-jason Jun 27, 2024
5015078
Display all ranks available for taxons
CarolineDenis Jun 27, 2024
6f23223
Add fix me
CarolineDenis Jun 27, 2024
410229d
Define correct tree def when adding children to node
CarolineDenis Jun 28, 2024
ee8919a
Display all ranks in query mapping for taxon trees
CarolineDenis Jun 28, 2024
a198594
Test: group ranks by texon tree
CarolineDenis Jun 28, 2024
f03d86a
test: group ranks by tree type
CarolineDenis Jun 28, 2024
7f7b99c
Fetch resource for label
CarolineDenis Jul 1, 2024
d4b9a0a
Move group labels for taxon in seperate function
CarolineDenis Jul 1, 2024
5b0d3f8
Renaming variable
CarolineDenis Jul 1, 2024
5f65b18
Renaming
CarolineDenis Jul 1, 2024
f89b8a9
Create tree type label and group in LineComponent
CarolineDenis Jul 1, 2024
1a5dd21
Create tree result variable
CarolineDenis Jul 1, 2024
7e9c71c
Keep previous group name
CarolineDenis Jul 1, 2024
2bdfb93
Small fix
CarolineDenis Jul 1, 2024
eab0e19
Change state type
CarolineDenis Jul 1, 2024
1357a08
Change fetchResult return
CarolineDenis Jul 1, 2024
835b5e9
Remove taxonGroup component
CarolineDenis Jul 1, 2024
175f50a
Create fetchTree results promise
CarolineDenis Jul 1, 2024
3d05a34
Add treeDefName to getMappingLineDate
CarolineDenis Jul 2, 2024
695090e
Add missing code
CarolineDenis Jul 2, 2024
c1863b9
Remove import
CarolineDenis Jul 2, 2024
6c5d86d
Remove unused code
CarolineDenis Jul 2, 2024
db0281c
Fix types
CarolineDenis Jul 3, 2024
37686ef
Remove comment
CarolineDenis Jul 3, 2024
c1dcfae
Use TreeDef id in cache over name and define getTreeDefinitions function
melton-jason Jul 5, 2024
4ec8ada
Fix scoping for fetching treeDef info and resolve TS errors
melton-jason Jul 5, 2024
7af53c5
Resolve failing tests
melton-jason Jul 6, 2024
de9c082
Lint code with ESLint and Prettier
melton-jason Jul 6, 2024
6358721
Merge branch 'production' into issue-4829
melton-jason Jul 6, 2024
f8544d9
Typo
CarolineDenis Jul 8, 2024
372ac4e
Remove taxon tree sub-headers in query mapping
CarolineDenis Jul 8, 2024
0b6dc98
Add useMemo
CarolineDenis Jul 9, 2024
82fb29d
Use treeDefinitionId over filter
melton-jason Jul 10, 2024
dad3036
Lint code with ESLint and Prettier
melton-jason Jul 10, 2024
0dd176b
Separate useMemo
CarolineDenis Jul 10, 2024
abb9934
Specify an explicit 'all' parameter for getting tree info
melton-jason Jul 10, 2024
9ace902
Use the table label over definition name in TreeSelectDialog
melton-jason Jul 10, 2024
f635aa9
Remove unused file import
melton-jason Jul 10, 2024
369de4a
Remove grouping ranks for tree tables
CarolineDenis Jul 10, 2024
4f6d251
Simplify rank grouping in navigator
CarolineDenis Jul 10, 2024
e93bd23
Update tests
CarolineDenis Jul 10, 2024
91bec1c
Group same rank together in the QB
CarolineDenis Jul 10, 2024
c9862ba
Chnage types
CarolineDenis Jul 15, 2024
78e471f
Use all TreeDefinitionItems to determine genus rankId for TaxonTiles
melton-jason Jul 16, 2024
4d0baf4
Recalculate treeData in TreeSelectDialog when treeRanksPromise completes
melton-jason Jul 16, 2024
1992ac8
Remove unused permission
melton-jason Jul 16, 2024
9d6ac47
Add proper type to table_permissions_checker
melton-jason Jul 16, 2024
380c627
Merge branch 'production' into issue-4829
acwhite211 Jul 22, 2024
d7947a5
query_construct get_taxon_treedef
acwhite211 Jul 23, 2024
6d27d50
taxon filter_by_collection
acwhite211 Jul 23, 2024
9166ac2
more robust get_taxon_treedef
acwhite211 Jul 23, 2024
02ae3eb
filter taxon trees by collection in query execution
acwhite211 Jul 24, 2024
198cde2
barvis update for multiple taxon trees
acwhite211 Jul 24, 2024
067f0d8
fix taxon in filter_by_collection
acwhite211 Jul 24, 2024
f21d2a5
fix collection.collectionname
acwhite211 Jul 24, 2024
6cc2edc
redefine get_treedef
acwhite211 Jul 24, 2024
013853b
TaxonTreeDefItem execution filter
acwhite211 Jul 24, 2024
d171a9c
mypy error fix
acwhite211 Jul 24, 2024
488cdf2
fix workbench unit test setup
acwhite211 Jul 25, 2024
30f86df
fix taxon tiles with multiple trees
acwhite211 Jul 26, 2024
40b1a32
fix partial merge, get_taxon_treedef_ids_by_collection
acwhite211 Jul 29, 2024
1ca6146
Merge remote-tracking branch 'origin/issue-4829-something' into issue…
acwhite211 Jul 29, 2024
b4cca06
set_discipline_for_taxon_treedefs in migration
acwhite211 Jul 29, 2024
04f562a
Handle null taxontreedef.discipline
acwhite211 Jul 29, 2024
de5eec7
fix taxon scoping in specify_trees api
acwhite211 Jul 29, 2024
7ea832a
change filter logic
acwhite211 Jul 29, 2024
b62b735
Simplify filters
realVinayak Jul 29, 2024
c230a33
(bug): resolve misc attribute errors
realVinayak Jul 29, 2024
f4e1961
(refactor): Remove unused parameter
realVinayak Jul 29, 2024
760023a
(bug): Resolve dup treedefs
realVinayak Jul 29, 2024
9dc4a81
Merge remote-tracking branch 'origin/production' into issue-4829
realVinayak Jul 30, 2024
3c9ae3f
Merge branch 'production' into issue-4829
realVinayak Jul 30, 2024
047602c
push workbench scoping to next PR
acwhite211 Jul 30, 2024
b981d9a
fix mypy error
acwhite211 Jul 30, 2024
128cd06
push migration exception handling to another PR
acwhite211 Jul 30, 2024
c60d2ed
Restrict results in Search Tree to currently viewed Tree
melton-jason Jul 30, 2024
d7c163a
(bug): Push down tree-scope to node
realVinayak Jul 30, 2024
004af18
(refactor): Remove redundant callbacks
realVinayak Jul 30, 2024
1d215ba
(bug): Resolve dup treedefitems
realVinayak Jul 30, 2024
04c48ea
(bug): Resolve treedef at discipline being skipped
realVinayak Jul 31, 2024
e62546f
(bug): Resolve attribute error
realVinayak Jul 31, 2024
4201323
Simplify tree def scoping
melton-jason Jul 31, 2024
0f9058f
Extract scoping treedefs into function and restore flat
melton-jason Jul 31, 2024
c23ec25
Unify tree scoping code on backend
melton-jason Jul 31, 2024
97ce8b1
Directly use tree search filters over get_treedefs function
melton-jason Jul 31, 2024
473814e
Merge remote-tracking branch 'origin/production' into issue-4829
CarolineDenis Aug 1, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
43 changes: 27 additions & 16 deletions specifyweb/barvis/views.py
Original file line number Diff line number Diff line change
@@ -1,29 +1,40 @@
from django.http import HttpResponse

from sqlalchemy.sql.expression import func, distinct
from django.db.models import Count, Q

from specifyweb.middleware.general import require_GET
from specifyweb.specify.views import login_maybe_required
from specifyweb.specify.filter_by_col import filter_by_collection
from specifyweb.specify.api import toJson

from specifyweb.stored_queries.models import Determination, Taxon
from specifyweb.specify.models import Taxon

from django.db import connection


@require_GET
@login_maybe_required
def taxon_bar(request):
"Returns the data for creating a taxon tiles visualization."
cursor = connection.cursor()
cursor.execute("""
SELECT t.TaxonID,
t.RankID,
t.ParentID,
t.Name,
(SELECT COUNT(*) FROM determination d WHERE t.TaxonID = d.TaxonID AND d.IsCurrent = 1)
FROM taxon t
WHERE t.TaxonTreeDefID = %s
""", [request.specify_collection.discipline.taxontreedef_id])
# "Returns the data for creating a taxon tiles visualization."
# cursor = connection.cursor()
# cursor.execute("""
# SELECT t.TaxonID,
# t.RankID,
# t.ParentID,
# t.Name,
# (SELECT COUNT(*) FROM determination d WHERE t.TaxonID = d.TaxonID AND d.IsCurrent = 1)
# FROM taxon t
# WHERE t.TaxonTreeDefID = %s
# """, [request.specify_collection.discipline.taxontreedef_id])

# Implementing the previous SQL query in Django ORM:
taxons = (
Taxon.objects.annotate(
current_determination_count=Count(
'determinations', filter=Q(determinations__iscurrent=True))
)
.values_list("id", "rankid", "parent_id", "name", "current_determination_count")
)
filtered_taxons = filter_by_collection(taxons, request.specify_collection)
result = toJson(list(filtered_taxons))

# SELECT d.TaxonID, COUNT(DISTINCT d.CollectionObjectID), t.ParentID
# FROM determination d
Expand All @@ -33,7 +44,7 @@ def taxon_bar(request):
# GROUP BY d.TaxonId
# ORDER BY d.TaxonId
# """, [request.specify_collection.id])
result = toJson(cursor.fetchall())
# result = toJson(cursor.fetchall())
# session = Session()
# query = session.query(
# Determination.TaxonID,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -448,6 +448,12 @@ export const ResourceBase = Backbone.Model.extend({
this.trigger('change', this);
return undefined;
}
/*
* Needed for taxonTreeDef on discipline because field.isVirtual equals false
*/
case 'one-to-one': {
return value;
}
}
if (!field.isVirtual)
softFail('Unhandled setting of relationship field', {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -5,16 +5,14 @@ import { getPref } from '../InitialContext/remotePrefs';
import { fetchPossibleRanks } from '../PickLists/TreeLevelPickList';
import { formatUrl } from '../Router/queryString';
import type { BusinessRuleResult } from './businessRules';
import type {
AnyTree,
FilterTablesByEndsWith,
TableFields,
} from './helperTypes';
import type { AnyTree, TableFields } from './helperTypes';
import type { SpecifyResource } from './legacyTypes';
import { idFromUrl } from './resource';
import type { Tables } from './types';

// eslint-disable-next-line unicorn/prevent-abbreviations
export type TreeDefItem<TREE extends AnyTree> =
FilterTablesByEndsWith<`${TREE['tableName']}TreeDefItem`>;
Tables[`${TREE['tableName']}TreeDefItem`];

export const initializeTreeRecord = (
resource: SpecifyResource<AnyTree>
Expand All @@ -36,7 +34,11 @@ export const treeBusinessRules = async (
const possibleRanks =
parentDefItem === undefined
? undefined
: await fetchPossibleRanks(resource, parentDefItem.get('rankId'));
: await fetchPossibleRanks(
resource,
parentDefItem.get('rankId'),
idFromUrl(parentDefItem.get('treeDef'))!
);

const doExpandSynonymActionsPref = getPref(
`sp7.allow_adding_child_to_synonymized_parent.${resource.specifyTable.name}`
Expand Down
1 change: 1 addition & 0 deletions specifyweb/frontend/js_src/lib/components/Forms/Save.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -204,6 +204,7 @@ export function SaveButton<SCHEMA extends AnySchema = AnySchema>({

const ButtonComponent = saveBlocked ? Button.Danger : Button.Save;
const SubmitComponent = saveBlocked ? Submit.Danger : Submit.Save;

// Don't allow cloning the resource if it changed
const isChanged = saveRequired || externalSaveRequired;

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ import { ajax } from '../../utils/ajax';
import type { RA } from '../../utils/types';
import { tables } from '../DataModel/tables';
import {
getTreeDefinitionItems,
strictGetTreeDefinitionItems,
treeRanksPromise,
} from '../InitialContext/treeRanks';
import {
Expand Down Expand Up @@ -98,7 +98,8 @@ function useGenusRankId(): number | false | undefined {
async () =>
treeRanksPromise.then(
() =>
getTreeDefinitionItems('Taxon', false)!.find(
// REFACTOR: Narrow the TreeDefinition used to find the rankId of the Genus Rank
strictGetTreeDefinitionItems('Taxon', false, 'all').find(
(item) => (item.name || item.title)?.toLowerCase() === 'genus'
)?.rankId ?? false
),
Expand Down
147 changes: 89 additions & 58 deletions specifyweb/frontend/js_src/lib/components/InitialContext/treeRanks.ts
Original file line number Diff line number Diff line change
Expand Up @@ -3,32 +3,35 @@
* given discipline.
*/

import { ajax } from '../../utils/ajax';
import { getCache } from '../../utils/cache';
import { f } from '../../utils/functools';
import type { RA } from '../../utils/types';
import { defined } from '../../utils/types';
import {
caseInsensitiveHash,
sortFunction,
unCapitalize,
} from '../../utils/utils';
import { caseInsensitiveHash } from '../../utils/utils';
import type {
AnySchema,
AnyTree,
FilterTablesByEndsWith,
SerializedRecord,
SerializedResource,
} from '../DataModel/helperTypes';
import type { SpecifyResource } from '../DataModel/legacyTypes';
import { fetchContext as fetchDomain, schema } from '../DataModel/schema';
import { getDomainResource } from '../DataModel/scoping';
import { serializeResource } from '../DataModel/serializers';
import { genericTables } from '../DataModel/tables';
import type { Tables } from '../DataModel/types';

let treeDefinitions: {
readonly [TREE_NAME in AnyTree['tableName']]: {
readonly definition: SpecifyResource<Tables[`${TREE_NAME}TreeDef`]>;
readonly ranks: RA<SerializedResource<Tables[`${TREE_NAME}TreeDefItem`]>>;
};
} = undefined!;
export type TreeInformation = {
readonly [TREE_NAME in AnyTree['tableName']]: RA<{
readonly definition: SerializedResource<FilterTablesByEndsWith<'TreeDef'>>;
readonly ranks: RA<
SerializedResource<FilterTablesByEndsWith<'TreeDefItem'>>
>;
}>;
};

let treeDefinitions: TreeInformation = undefined!;

/*
* FEATURE: allow reordering trees
Expand All @@ -37,7 +40,6 @@ let treeDefinitions: {
const commonTrees = ['Geography', 'Storage', 'Taxon'] as const;
const treesForPaleo = ['GeologicTimePeriod', 'LithoStrat'] as const;
export const allTrees = [...commonTrees, ...treesForPaleo] as const;
const paleoDiscs = new Set(['paleobotany', 'invertpaleo', 'vertpaleo']);
/*
* Until discipline information is loaded, assume all trees are appropriate in
* this discipline
Expand All @@ -60,46 +62,36 @@ export const treeRanksPromise = Promise.all([
import('../Permissions').then(async ({ fetchContext }) => fetchContext),
import('../DataModel/tables').then(async ({ fetchContext }) => fetchContext),
fetchDomain,
])
.then(async ([{ hasTreeAccess, hasTablePermission }]) =>
hasTablePermission('Discipline', 'read')
? getDomainResource('discipline')
?.fetch()
.then((discipline) => {
if (!f.has(paleoDiscs, discipline?.get('type')))
disciplineTrees = commonTrees;
})
.then(async () =>
Promise.all(
disciplineTrees
.filter((treeName) => hasTreeAccess(treeName, 'read'))
.map(async (treeName) =>
getDomainResource(getTreeScope(treeName) as 'discipline')
?.rgetPromise(
`${unCapitalize(treeName) as 'geography'}TreeDef`
)
.then((treeDefinition) => {
const ranks = {
definition: treeDefinition,
ranks: Array.from(
treeDefinition.getDependentResource('treeDefItems')
?.models ?? [],
(definitionItem) => serializeResource(definitionItem)
).sort(sortFunction(({ rankId }) => rankId)),
};
]).then(async () =>
ajax<{
readonly [TREE_NAME in AnyTree['tableName']]: RA<{
readonly definition: SerializedRecord<Tables[`${TREE_NAME}TreeDef`]>;
readonly ranks: RA<SerializedRecord<Tables[`${TREE_NAME}TreeDefItem`]>>;
}>;
}>('/api/specify_trees/', {
CarolineDenis marked this conversation as resolved.
Show resolved Hide resolved
headers: { Accept: 'application/json' },
}).then(({ data }) => {
const treeNames = new Set(
Object.keys(data).map((key) => key.toLowerCase())
);
disciplineTrees = allTrees.filter((treeName) =>
treeNames.has(treeName.toLowerCase())
);

return [treeName, ranks] as const;
})
)
)
)
: []
)
.then((ranks) => {
// @ts-expect-error
treeDefinitions = Object.fromEntries(ranks.filter(Boolean));
treeDefinitions = Object.fromEntries(
Object.entries(data).map(([treeName, information]) => [
treeName,
information.map(({ definition, ranks }) => ({
definition: serializeResource(
definition as SerializedRecord<FilterTablesByEndsWith<'TreeDef'>>
),
ranks: ranks.map((rank) => serializeResource(rank)),
})),
])
);
return treeDefinitions;
});
})
);

function getTreeScope(
treeName: AnyTree['tableName']
Expand All @@ -114,22 +106,61 @@ function getTreeScope(
);
}

export function getTreeDefinitions<TREE_NAME extends AnyTree['tableName']>(
tableName: TREE_NAME,
treeDefinitionId: number | 'all' | undefined = getCache(
'tree',
`definition${tableName}`
)
): TreeInformation[TREE_NAME] {
const specificTreeDefinitions = caseInsensitiveHash(
defined(treeDefinitions),
tableName
);

return typeof treeDefinitionId === 'number'
? specificTreeDefinitions.filter(
({ definition }) => definition.id === treeDefinitionId
)
: specificTreeDefinitions;
}

export function getTreeDefinitionItems<TREE_NAME extends AnyTree['tableName']>(
tableName: TREE_NAME,
includeRoot: boolean
): typeof treeDefinitions[TREE_NAME]['ranks'] | undefined {
const definition = caseInsensitiveHash(treeDefinitions, tableName);
return definition?.ranks.slice(includeRoot ? 0 : 1);
includeRoot: boolean,
treeDefinitionId: number | 'all' | undefined = getCache(
'tree',
`definition${tableName}`
)
): TreeInformation[TREE_NAME][number]['ranks'] | undefined {
const specificTreeDefinitions =
treeDefinitions === undefined
? undefined
: caseInsensitiveHash(treeDefinitions, tableName);

return specificTreeDefinitions === undefined
? undefined
: typeof treeDefinitionId === 'number'
? specificTreeDefinitions
.find(({ definition }) => definition.id === treeDefinitionId)
?.ranks.slice(includeRoot ? 0 : 1)
: specificTreeDefinitions.flatMap(({ ranks }) =>
ranks.slice(includeRoot ? 0 : 1)
);
}

export const strictGetTreeDefinitionItems = <
TREE_NAME extends AnyTree['tableName']
>(
tableName: TREE_NAME,
includeRoot: boolean
): typeof treeDefinitions[TREE_NAME]['ranks'] =>
includeRoot: boolean,
treeDefinitionId: number | 'all' | undefined = getCache(
'tree',
`definition${tableName}`
)
): TreeInformation[TREE_NAME][number]['ranks'] =>
defined(
getTreeDefinitionItems(tableName, includeRoot),
getTreeDefinitionItems(tableName, includeRoot, treeDefinitionId),
`Unable to get tree ranks for a ${tableName} table`
);

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -136,7 +136,8 @@ async function recursiveResourceResolve(
return [];
const tableRanks = strictGetTreeDefinitionItems(
treeTableName as 'Geography',
false
false,
'all'
);
const currentRank = defined(
tableRanks.find(({ rankId }) => rankId === resource.get('rankId')),
Expand Down
Loading
Loading