ITIS (id 2144): test report #8
Comments
|
file ITIS_GSDs+Updates_forCoL_2020-03-03_yr_log
|
|
file ITIS_GSDs+Updates_forCoL_2020-03-03_yr_log ITIS_ac19
|
|
@gdower set up a timer for automatic import. Standard day: |
|
we have a built in timer we should use. After stable ids are completed we should try it with all datasets on dev for a while and activate it on prod if no issues pop up. We did use it in the early days already |
|
ITIS of 2020-12-21 is synced on 2021-01-08 |
|
|
@gdower fixed broken sectors 2021-02-01 ITIS of 2020-12-21 is re-synced on 2021-02-01 |
|
ITIS of 2021-01-26 was imported (no broken sectors) and synced on 2021-02-03. |
|
@yroskov , some newer ITIS GSDs that don't appear to overlap with other sources: SMALL BUT IMPORTANT FAMILY: Additional newer ITIS GSDs that appear to be "GAPS" in COL: Animalia : Bilateria : Protostomia : Ecdysozoa : Arthropoda : Hexapoda : Insecta : Pterygota : Neoptera : Paraneoptera : Hemiptera : Heteroptera : Cimicomorpha : Cimicoidea : Polyctenidae Animalia : Bilateria : Protostomia : Ecdysozoa : Arthropoda : Hexapoda : Insecta : Pterygota : Neoptera : Paraneoptera : Hemiptera : Heteroptera : Pentatomomorpha : Pentatomoidea : Acanthosomatidae |
|
@DaveNicolson thank you! Families Varroidae, Cimicidae, Polyctenidae & Acanthosomatidae have been assembled in CoL and synced from ITIS of 2021-01-26. |
|
@yroskov the February ITIS load was just put online. There are two new GSDs that appear to be gaps in COL now. They are: The latter was split out of the former. I see COL is not using any ranks between Decapoda and superfamilies in it. I've noted the GSD superfamilies for each of the two infraorder GSDs so you can handle them as you wish. |
|
The import of the ITIS 2021-02-26 release finished: https://data.catalogueoflife.org/dataset/2144/imports |
|
@DaveNicolson thank you! Infraorders Astacidea and Glypheidea have been assembled in CoL as direct children of Decapoda and synced from ITIS of 2021-02-26. (Hmm, it's a mess in the Tree: now all superfamilies from WoRMS Brachyura gone under infraorder Not Assigned). |
|
ITIS of 2021-02-26 (id 2144) is synced on 2021-03-02. |
|
March 30 load for ITIS is complete, and the full exports are available. A new smaller GSD that is a gap now in COL is Order Lophogastrida (ITIS TSN 89808), which is in the ITIS hierarchy here: |
|
Thank you Dave! We already processed a new version. Without Lophogastrida yet. I'll add it as a new sector right now in the clearinghouse for a next release. |
|
Order Lophogastrida established as a sector and synced 2021-04-02 (after launch of preview release). |
|
ITIS of 2021-04-27 (id 2144) is imported in CoL & synced on 2021-04-30. |
|
Two (or three, if you prefer) new (for ITIS) GSDs that may or may not be wanted by COL (none of them appears in COL now, I believe): Animalia : Bilateria : Protostomia : Ecdysozoa : Arthropoda : Crustacea : Malacostraca : Eumalacostraca : Eucarida : Decapoda : Pleocyemata : Caridea : Alpheoidea : Alpheidae Animalia : Bilateria : Protostomia : Ecdysozoa : Arthropoda : Crustacea : Malacostraca : Eumalacostraca : Eucarida : Decapoda : Pleocyemata : Achelata & Polychelida [former "Palinura", now split] |
|
@DaveNicolson thank you! Family Alpheidae is assembled now as a direct child of superfamily Alpheoidea. Infraorders Achelata & Polychelida are assembled as children of suborder Pleocyemata. All 3 new sectors synced and will appear in CoL of May. |
|
New version of ITIS is available, dated 26 May 2021. We have some modest new GSDs that appear to be gaps in COL: Animalia : Bilateria : Protostomia : Ecdysozoa : Arthropoda : Chelicerata : Euchelicerata : Arachnida : Acariformes : Trombidiformes : Prostigmata : Anystina : Erythraeoidea : Smarididae (family) (we are working on the other family of the superfamily, not ready yet though) Animalia : Bilateria : Protostomia : Ecdysozoa : Arthropoda : Chelicerata : Euchelicerata : Arachnida : Acariformes : Trombidiformes : Prostigmata : Anystina : Calyptostomatoidea (superfamily) Animalia : Bilateria : Protostomia : Ecdysozoa : Arthropoda : Crustacea : Malacostraca : Eumalacostraca : Eucarida : Decapoda : Pleocyemata : Stenopodidea (infraorder) |
|
Thanks @DaveNicolson! I imported the new version, although it won't be assembled into the Catalogue until Yuri returns next week: https://data.catalogueoflife.org/dataset/2144/imports |
|
3 new sectors established as:
|
|
ITIS of 2021-05-26 (id 2144, all sectors) is synced on 2021-06-07. |
|
The June ITIS load is completed, and you're welcome to import it for your use. As already discussed (in a Spring 2021 Taxonomic Group meeting), COL should consider whether to adopt the new ITIS GSD for the mosquito family, Culicidae, under Order Diptera in COL, in place of Systema Dipterorum's data (if Thomas Pape still agrees, it is really his call, the numbers below are not so different after all). If you want it, it is in ITIS here: COL now shows these stats for the family, per Systema Dipterorum: ITIS now shows these stats for the family: The update was based primarily on the 2021 "Mosquitoes of the World" volumes (Wilkerson et al.), but also used the data from Harbach's "Mosquito Taxonomic Inventory" website. We tried to include the alternative combinations in use (in synonymy, where appropriate), especially for known disease vectors, and the full data set includes over 5400 scientific names (all ranks, all usages), which should help link names with various conflicting taxonomic sources from the last ~45 years. |
|
ITIS of 2021-06-29 imported in the checklist bank 2021-07-01.
I have have set up a patch, repeating metadata from the portal: https://www.catalogueoflife.org/data/dataset/2144 |
|
The new diff UI even works for the larger ITIS changes: https://data.catalogueoflife.org/dataset/2144/diff?attempts=36..37 |
|
I have deleted the 5 ITIS merge sectors which should have removed all linked data in the project. @yroskov could you do a brief check if you spot sth unusual? I also deployed a new backend and UI with a new |
|
@thomasstjerne also changed the UI to ask a user whether he wants a full or partial delete of a sector when you hit the delete. So you can pick if you want to retain higher taxa or delete it all |
|
it is working as expected!
|
|
@DaveNicolson, should we expect updates from ITIS at the end of May? (2024 Annual Checklist will be released in June) |
|
@yroskov Unfortunately, the next ITIS load will be late June (and it should include an updated classification of the Coccinellidae to subgenus, which should allow you to take the GSDs for genus Rhyzobius and tribe Epilachnini, at last). After that we should return to monthly loads. |
|
Thanks! Looking forward the end of June for the July release :) |
|
ITIS is wrapping up a load that was finalized and dated 26 June 2024 (I emailed Geoff a link to get it since it's not up yet on the ITIS site). In addition to updates to existing GSDs COL sources from ITIS (particularly in Mammalia), we also finally updated the "ladybird beetle" (Coccinellidae) classification from family to subgenus. This should allow COL to take up the GSDs for two parts of the family that we previously completed (last fall):
The hierarchy within Arthropoda was updated, and there are some changes that could affect at least one GSD... Based on Dr. Zhang's recommendations following the Taxonomy Group meetings, (A) Arachnida is recognized again as a full class, and Class Euchelicerata was dropped (it remains useful for Arachnida+Xuphosura vs. Pycnogonida, but forced the prior 'unpopular' rank shift that we're undoing), (B) [Class Merostomata: Order Xiphosura] replaces [Subclass Xiphosura: Order Xiphosurida], (C) Superorder Opilioacariformes ITIS GSD should become Order Opilioacarida ITIS GSD since Dr. Zhang indicates acarine workers generally consider it to be basal WITHIN Superorder Parasitiformes... Those changes are in-line with recommendations discussed in the Taxonomy Group meeting (and afterward directly with Dr. Zhang via email). In summary of the Arachnida-related changes, here's what ITIS will now show (once the load is completed and the DB connections stabilized): Subphylum Chelicerata We also made some hierarchy changes within Crustacea, and it will hopefully make a lot more sense now (even though everyone knows much of it isn't "final"). Of course you'll have your own needs for handling COL's hierarchy, but I hope this is of some use. |
|
@mdoering, very strange happened to ITIS sectors in CLB project 3. All ITIS taxa appears as CoL sectors in the right window, despite they are not real sectors in the left window (and should not be sectors in the global CoL). Seems, it caused by Extended Catalogue: https://www.checklistbank.org/catalogue/3/assembly?assemblyTaxonKey=c53306f8-8660-4754-940f-b1f0cefc62ea&datasetKey=2144 As result, I am not able to establish tribe Epilachnini and genus Rhyzobius as new GSD sectors, because they are already shown as part of ITIS sectors in CoL: |
|
@mdoering, how I can sync only global sectors from ITIS, if I have this situation:
|
|
There are 2 options. 
@thomasstjerne The sync and rematch all selected buttons are pretty hidden and I can access them only when I hover over the Dataset column. The second option is just to hit sync all - merge sectors will be blocked as long as the projects |
|
ITIS of 2024-06-26; imported 2024-07-01
Metrics
Synced 2024-07-08 |
|
ITIS of 2024-07-23; imported 2024-08-05
Metrics
Synced (all "attached", excluding "merged"?) 2024-08-06 |
|
The August ITIS export is available for download & processing (load completed on 16th, but re-exported on 20th to resolve an issue). Aside from some updated GSDs and additions of groups COL gets from elsewhere, there are some small new ITIS GSDs that are empty in COL... Cryptococcidae, found in ITIS here (but in current COL classification it would go under infraorder Coccidomorpha): Coelostomidiidae, found ITIS here (but in current COL classification it would go under infraorder Coccidomorpha): Forgot another small one: |
|
@DaveNicolson, as I can see, children taxa of Cryptococcidae, Coelostomidiidae & Callipappidae are already present in the CoL under different families as a part of global ScaleNet checklist. Update of entire ScaleNet is in the hands of @gdower. I hope, we'll be able to update all scales in 2024. Keep fingers crossed. At this moment, we have an awful mess in the classification of scales due to mixture of 2004 ScaleNet data and empty insertions from unnamed source: https://www.catalogueoflife.org/?taxonKey=C22KK (@mdoering, are these artifacts of Extended Catalogue?) |
|
OK, great to hear ScaleNet's to be updated in COL!! We're following their current data, and Daniel has exchanged emails with editors who have made edits as a result. I'm really glad to hear it's being update in COL!! I'll omit such updates based on ScaleNet from COL notices in the future. |
|
ITIS of 2024-08-16; imported 2024-09-03
Metrics
TASKS
Synced 2024-09-11 |
|
A new version of ITIS, dated 20 Sep 2024, is available in the usual download page. I separately sent Geoff the new list of extinct TSNs from ITIS. This version includes a number of updates in groups COL takes from other sources (mainly in Coccoidea/Coccomorpha), but no new/updated GSDs for COL. |
|
ITIS of 2024-09-20; imported 2024-10-01
Metrics
TASKS
Resolved 2024-10-03:
Synced 2024-10-03 Button "Sync all sectors from dataset 2144" has been used. The list showed 5 "merged" sectors despite the option "Include merged sources" not being activated:
|
|
There is a new version of ITIS available for download and use, although the updates added this month are all in groups that COL doesn't take from ITIS. Separately, I emailed Geoff the updated list of TSNs for valid/accepted taxa that are extinct. |
|
|
ITIS of 2024-10-22; imported 2024-11-04
Metrics
TASKS
Resolved 2024-11-06:
Synced 2024-11-06 (see the comment in October) |
|
The November 2024 ITIS load is complete, and in addition to updating some bird groups, we have also added a GSD covering the bacterial Class Cyanophyceae (cynaobacteria). ITIS acquired the list via contract with workers at UFlorida, and integrated the existing ITIS data into a clean list. The parent of the group in ITIS is a new named published recently, Phylum Cyanobacteriota, and this phylum will remain incomplete until we can fold in the other tiny class with its new name Class Vampirovibrionophyceae (which isn't photosynthetic!)... so far it looks like there is just a single species (with genus, family and order), and we'll have to move that from where we placed it back a dozen years ago, but none of those additional modifications can happen at least until the December load. Class Cyanophyceae now contains 4828 valid/accepted species, and authorship and unacceptability reasons follow the 'botanical' Code, where it has traditionally been treated. I'm not sure how COL will want to handle this, since it is WITHIN the older GSD for Kingdom Bacteria, but hopefully it won't cause problems! I'll email Yury and Geoff the new list of extinct TSNs momentarily. |
|
ITIS of 2024-11-19; imported 2024-12-02
Metrics
TASKS
Resolved 2024-12-02:
Synced 2024-12-02 |
|
New ITIS version is available from the usual place. Updates are all in groups COL gets from ITIS (including birds, Amblypygi, a tiny class (Vampirovibrionophyceae, predatory "cyanobacteria" not covered by the cyanobacteria list ITIS loaded last month) that completes the group containing cyanobacteria, and some tweaks in Mammalia). I sent the new list of TSNs to be marked as "extinct" to Geoff and Yuri. |
https://www.checklistbank.org/dataset/2144/
Source of global sectors:
file ITIS_GSDs+Updates_forCoL_2020-03-03.xlsx
From: Nicolson, David
Sent: Tuesday, March 3, 2020 23:01
To: Roskov, Yury
Cc: Orrell, Thomas
Subject: Initial list of ITIS GSDs for addition (or consideration) to CoL
Yuri,
OK, here is my first pass trying to detect ITIS GSDs that should (or could) be added or updated in CoL. It includes GSDs we added or updated in ITIS since the last time CoL was updated for ITIS (mid-2017), as well as a few cases where ITIS loaded a GSD that was not noted to you previously. I left out groups where CoL already has a solid/active source, assuming the source seemed to actually be providing a reasonably complete GSD (vs. an "aspirational" GSD that is not very close to complete).
They are sorted according to their placement in ITIS now, via a hierarchy column. Those with yellow question marks may or may not be used in CoL; a few already have a source for CoL, but I suggest at least considering switching to ITIS due to various issues.
I have included a few things that we will shortly have loaded into ITIS, and a few that we are actively working on now (for inclusion in ITIS later this year, likely before the ITIS CoLdp export is ready).
If we realize we missed anything I will let you know.
Thanks, Dave
The text was updated successfully, but these errors were encountered: