De-duplicate ontology descriptions (OBOFoundry#1966)

Closes OBOFoundry#1965

Author: cthoyt

ontology/aism.md

@@ -33,5 +33,3 @@ activity_status: active
 repository: https://github.com/insect-morphology/aism
 preferredPrefix: AISM
 ---
-
-The AISM contains terms used in insect biodiversity research for describing structures of the exoskeleton and the skeletomuscular system. It aims to serve as the basic backbone of generalized terms to be expanded with order-specific terminology.

ontology/apo.md

@@ -30,5 +30,3 @@ publications:
   - id: https://www.ncbi.nlm.nih.gov/pubmed/20157474
     title: "New mutant phenotype data curation system in the Saccharomyces Genome Database"
 ---
-
-A structured controlled vocabulary for the phenotypes of Ascomycete fungi

ontology/cheminf.md

@@ -27,5 +27,3 @@ activity_status: active
 repository: https://github.com/semanticchemistry/semanticchemistry
 preferredPrefix: CHEMINF
 ---
-
-Includes terms for the descriptors commonly used in cheminformatics software applications and the algorithms which generate them.

ontology/cto.md

@@ -20,5 +20,3 @@ repository: https://github.com/ClinicalTrialOntology/CTO
 preferredPrefix: CTO
 domain: health
 ---
-
-The core Ontology of Clinical Trials (CTO) will serve as a structured resource integrating basic terms and concepts in the context of clinical trials. Thereby covering clinicaltrails.gov. CoreCTO will serve as a basic ontology to generate extended versions for specific applications such as annotation of variables in study documents from clinical trials.

ontology/fbbi.md

@@ -25,5 +25,3 @@ build:
 activity_status: active
 repository: https://github.com/CRBS/Biological_Imaging_Methods_Ontology
 ---
-
-A structured controlled vocabulary of sample preparation, visualization and imaging methods used in biomedical research.

ontology/hao.md

@@ -31,5 +31,3 @@ publications:
   - id: https://www.ncbi.nlm.nih.gov/pubmed/21209921
     title: "A gross anatomy ontology for hymenoptera"
 ---
-
-A structured controlled vocabulary of the anatomy of the Hymenoptera (bees, wasps, and ants)

ontology/mamo.md

@@ -23,5 +23,3 @@ activity_status: active
 preferredPrefix: MAMO
 domain: simulation
 ---
-
-The Mathematical Modelling Ontology (MAMO) is a classification of the types of mathematical models used mostly in the life sciences, their variables, relationships and other relevant features.

ontology/mpath.md

@@ -27,5 +27,3 @@ activity_status: active
 repository: https://github.com/PaulNSchofield/mpath
 preferredPrefix: MPATH
 ---
-
-A structured controlled vocabulary of mutant and transgenic mouse pathology phenotypes

ontology/oarcs.md

@@ -20,5 +20,3 @@ products:
 activity_status: active
 preferredPrefix: OARCS
 ---
-
-OArCS is an ontology describing the Arthropod ciruclatory system.

ontology/ohmi.md

@@ -21,5 +21,3 @@ repository: https://github.com/ohmi-ontology/ohmi
 preferredPrefix: OHMI
 domain: organisms
 ---
-
-The Ontology of Host-Microbiome Interactions aims to ontologically represent and standardize various entities and relations related to microbiomes, microbiome host organisms (e.g., human and mouse), and the interactions between the hosts and microbiomes at different conditions.

ontology/opmi.md

@@ -21,5 +21,3 @@ repository: https://github.com/OPMI/opmi
 preferredPrefix: OPMI
 domain: investigations
 ---
-
-The Ontology of Precision Medicine and Investigation (OPMI) aims to ontologically represent and standardize various entities and relations associated with precision medicine and related investigations at different conditions.

ontology/rs.md

@@ -36,5 +36,3 @@ preferredPrefix: RS
 depicted_by: http://rgd.mcw.edu/common/images/rgd_LOGO_blue_rgd.gif
 domain: organisms
 ---
-
-Ontology of rat strains

ontology/spd.md

@@ -30,5 +30,3 @@ publications:
   - id: https://doi.org/10.3390/d11100202
     title: "The Spider Anatomy Ontology (SPD)—A Versatile Tool to Link Anatomy with Cross-Disciplinary Data"
 ---
-
-An ontology for spider comparative biology including anatomical parts (e.g. leg, claw), behavior (e.g. courtship, combing) and products (i.g. silk, web, borrow).

ontology/wbbt.md

@@ -41,5 +41,3 @@ usages:
 activity_status: active
 repository: https://github.com/obophenotype/c-elegans-gross-anatomy-ontology
 ---
-
-A structured controlled vocabulary of the anatomy of <i>Caenorhabditis elegans</i>.

ontology/wbls.md

@@ -43,5 +43,3 @@ usages:
 activity_status: active
 repository: https://github.com/obophenotype/c-elegans-development-ontology
 ---
-
-A structured controlled vocabulary of the development of <i>Caenorhabditis elegans</i>.

ontology/wbphenotype.md

@@ -51,5 +51,3 @@ usages:
 activity_status: active
 repository: https://github.com/obophenotype/c-elegans-phenotype-ontology
 ---
-
-A structured controlled vocabulary of <i>Caenorhabditis elegans</i> phenotypes

tests/test_integrity.py

@@ -20,6 +20,7 @@
 DOI_PREFIX = "https://doi.org/"
 CHEMRXIV_DOI_PREFIX = "https://doi.org/10.26434/chemrxiv"
 
+
 def get_data():
     """Get ontology data."""
     ontologies = {}
@@ -32,6 +33,7 @@ def get_data():
 
         # Load the data like it is YAML
         data = yaml.safe_load("\n".join(lines[1:idx]))
+        data["long_description"] = "".join(lines[idx:])
         ontologies[data["id"]] = data
     return ontologies
 
@@ -83,8 +85,14 @@ def test_publications(self):
 
             for i, usage in enumerate(data.get("usages", [])):
                 for j, publication in enumerate(usage.get("publications", [])):
-                    self.assertIn("user", usage, msg=f"Malformed usage missing a user in {ontology}")
-                    with self.subTest(ontology=ontology, user=usage["user"], id=publication["id"]):
+                    self.assertIn(
+                        "user",
+                        usage,
+                        msg=f"Malformed usage missing a user in {ontology}",
+                    )
+                    with self.subTest(
+                        ontology=ontology, user=usage["user"], id=publication["id"]
+                    ):
                         self.assert_valid_publication_id(
                             publication,
                             msg=f"{ontology} usage {i} publication {j} has unexpected identifier: {publication['id']}",
@@ -127,7 +135,12 @@ def assert_valid_publication_id(self, publication, msg=None):
         )
 
         # Make sure that the unversioned DOI is used
-        if is_arxiv or is_biorxiv or is_medrxiv or identifier.startswith(CHEMRXIV_DOI_PREFIX):
+        if (
+            is_arxiv
+            or is_biorxiv
+            or is_medrxiv
+            or identifier.startswith(CHEMRXIV_DOI_PREFIX)
+        ):
             for v in range(1, 100):
                 self.assertFalse(
                     identifier.endswith(f".v{v}"), msg="Please use an unversioned DOI"
@@ -147,13 +160,45 @@ def test_schema_mandatory(self):
         }
         self.assertEqual(required - skip_keys, high_level - skip_keys)
 
+    @staticmethod
+    def skip_inactive(record) -> bool:
+        """Check if should skip for inactive records."""
+        return record.get("activity_status") != "active"
+
     def test_preferred_prefix(self):
         """Test all preferred prefixes."""
         for prefix, record in self.ontologies.items():
             with self.subTest(prefix=prefix):
-                if record.get("activity_status") != "active":
+                if self.skip_inactive(record):
                     continue
                 preferred_prefix = record.get("preferredPrefix")
                 self.assertIsNotNone(preferred_prefix)
                 self.assertLessEqual(2, len(preferred_prefix))
                 self.assertNotIn(" ", preferred_prefix)
+
+    def test_redundant_descriptions(self):
+        """Test that the description field is not redundant of the long form description."""
+        for prefix, record in self.ontologies.items():
+            if self.skip_inactive(record):
+                continue
+            description = record.get("description")
+            long_description = record["long_description"]
+            if description is None:
+                continue
+            with self.subTest(prefix=prefix):
+                self.assertNotEqual(
+                    _string_norm(description),
+                    _string_norm(long_description),
+                    msg=f"Effectively the same description was reused in the short and long-form field for {prefix}",
+                )
+
+
+def _string_norm(s: str) -> str:
+    return (
+        s.strip()
+        .lower()
+        .replace("\n", "")
+        .replace(" ", "")
+        .replace(".", "")
+        .replace("-", "")
+    )