Skip to content

Commit

Permalink
Formatter / DCAT / Mobility DCAT - improve accrual periodicity mapping.
Browse files Browse the repository at this point in the history
Cardinality:
  * ISO 0..n
  * DCAT 0..n
  * DCAT-AP 0..1
  * Mobility DCAT 1..1 (in ISO either use corresponding period eg. P0Y0M0DT1H0M0S or extend the codelist with the proper vocabulary)

  accrualPeriodicity mapping done using the ISO to Dublin core value mapping
  but additional checks are done when ISO records extended the codelist and
  may used the EU Publication Office frequency codes
  or the Mobility DCAT-AP update frequency codes.
  Domain specific codelists take priority over the DC or ISO codelists.

  eg.
  <mmi:MD_MaintenanceFrequencyCode codeListValue="15min"/>

  multipleAccrualPeriodicityAllowed is a parameter that can be set to true to allow multiple accrualPeriodicity values.
  Default to false for EU formatters. true for DCAT.
  • Loading branch information
fxprunayre committed Sep 20, 2024
1 parent 0e183c1 commit 12ffbed
Show file tree
Hide file tree
Showing 10 changed files with 786 additions and 56 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -79,7 +79,10 @@
</dcat:spatialResolutionInMeters>
</xsl:when>
<xsl:otherwise>
<xsl:comment>WARNING: Spatial resolution only supported in meters. <xsl:value-of select="concat(*/text(), ' ', */@uom)"/> is ignored (can be related to unknown unit or no conversion factor or not a decimal value).</xsl:comment>
<xsl:comment>WARNING: Spatial resolution only supported in meters.
<xsl:value-of select="concat(*/text(), ' ', */@uom)"/> is ignored (can be related to unknown unit or no
conversion factor or not a decimal value).
</xsl:comment>
</xsl:otherwise>
</xsl:choose>

Expand All @@ -103,6 +106,24 @@
Definition: The frequency at which a dataset is published.
Range: dcterms:Frequency (A rate at which something recurs)
Usage note: The value of dcterms:accrualPeriodicity gives the rate at which the dataset-as-a-whole is updated. This may be complemented by dcat:temporalResolution to give the time between collected data points in a time series.
Cardinality:
* ISO 0..n
* DCAT 0..n
* DCAT-AP 0..1
* Mobility DCAT 1..1 (in ISO either use corresponding period eg. P0Y0M0DT1H0M0S or extend the codelist with the proper vocabulary)
accrualPeriodicity mapping done using the ISO to Dublin core value mapping
but additional checks are done when ISO records extended the codelist and
may used the EU Publication Office frequency codes
or the Mobility DCAT-AP update frequency codes.
Domain specific codelists take priority over the DC or ISO codelists.
eg.
<mmi:MD_MaintenanceFrequencyCode codeListValue="15min"/>
multipleAccrualPeriodicityAllowed is a parameter that can be set to true to allow multiple accrualPeriodicity values.
Default to false for EU formatters. true for DCAT.
-->
<xsl:variable name="isoFrequencyToDublinCore"
as="node()*">
Expand All @@ -122,21 +143,64 @@
-->
</xsl:variable>

<xsl:template mode="iso19115-3-to-dcat"
match="mdb:identificationInfo/*/mri:resourceMaintenance/*/mmi:maintenanceAndUpdateFrequency">

<xsl:variable name="dcFrequency"
as="xs:string?"
select="$isoFrequencyToDublinCore[@key = current()/*/@codeListValue]"/>

<dct:accrualPeriodicity>
<dct:Frequency rdf:about="{if($dcFrequency)
then concat($europaPublicationBaseUri, 'frequency/', $dcFrequency)
else concat($isoCodeListBaseUri, */@codeListValue)}"/>
</dct:accrualPeriodicity>
</xsl:template>

<!--
http://publications.europa.eu/resource/authority/frequency
-->
<xsl:variable name="euUpdateFrquencyToUri" as="node()*">
<entry key="BIDECENNIAL">http://publications.europa.eu/resource/authority/frequency/BIDECENNIAL</entry>
<entry key="TRIDECENNIAL">http://publications.europa.eu/resource/authority/frequency/TRIDECENNIAL</entry>
<entry key="BIHOURLY">http://publications.europa.eu/resource/authority/frequency/BIHOURLY</entry>
<entry key="TRIHOURLY">http://publications.europa.eu/resource/authority/frequency/TRIHOURLY</entry>
<entry key="OTHER">http://publications.europa.eu/resource/authority/frequency/OTHER</entry>
<entry key="WEEKLY">http://publications.europa.eu/resource/authority/frequency/WEEKLY</entry>
<entry key="NOT_PLANNED">http://publications.europa.eu/resource/authority/frequency/NOT_PLANNED</entry>
<entry key="AS_NEEDED">http://publications.europa.eu/resource/authority/frequency/AS_NEEDED</entry>
<entry key="HOURLY">http://publications.europa.eu/resource/authority/frequency/HOURLY</entry>
<entry key="QUADRENNIAL">http://publications.europa.eu/resource/authority/frequency/QUADRENNIAL</entry>
<entry key="QUINQUENNIAL">http://publications.europa.eu/resource/authority/frequency/QUINQUENNIAL</entry>
<entry key="DECENNIAL">http://publications.europa.eu/resource/authority/frequency/DECENNIAL</entry>
<entry key="WEEKLY_2">http://publications.europa.eu/resource/authority/frequency/WEEKLY_2</entry>
<entry key="WEEKLY_3">http://publications.europa.eu/resource/authority/frequency/WEEKLY_3</entry>
<entry key="UNKNOWN">http://publications.europa.eu/resource/authority/frequency/UNKNOWN</entry>
<entry key="UPDATE_CONT">http://publications.europa.eu/resource/authority/frequency/UPDATE_CONT</entry>
<entry key="QUARTERLY">http://publications.europa.eu/resource/authority/frequency/QUARTERLY</entry>
<entry key="TRIENNIAL">http://publications.europa.eu/resource/authority/frequency/TRIENNIAL</entry>
<entry key="NEVER">http://publications.europa.eu/resource/authority/frequency/NEVER</entry>
<entry key="OP_DATPRO">http://publications.europa.eu/resource/authority/frequency/OP_DATPRO</entry>
<entry key="MONTHLY_2">http://publications.europa.eu/resource/authority/frequency/MONTHLY_2</entry>
<entry key="MONTHLY_3">http://publications.europa.eu/resource/authority/frequency/MONTHLY_3</entry>
<entry key="IRREG">http://publications.europa.eu/resource/authority/frequency/IRREG</entry>
<entry key="MONTHLY">http://publications.europa.eu/resource/authority/frequency/MONTHLY</entry>
<entry key="DAILY">http://publications.europa.eu/resource/authority/frequency/DAILY</entry>
<entry key="DAILY_2">http://publications.europa.eu/resource/authority/frequency/DAILY_2</entry>
<entry key="BIWEEKLY">http://publications.europa.eu/resource/authority/frequency/BIWEEKLY</entry>
<entry key="CONT">http://publications.europa.eu/resource/authority/frequency/CONT</entry>
<entry key="BIENNIAL">http://publications.europa.eu/resource/authority/frequency/BIENNIAL</entry>
<entry key="BIMONTHLY">http://publications.europa.eu/resource/authority/frequency/BIMONTHLY</entry>
<entry key="ANNUAL_2">http://publications.europa.eu/resource/authority/frequency/ANNUAL_2</entry>
<entry key="ANNUAL_3">http://publications.europa.eu/resource/authority/frequency/ANNUAL_3</entry>
<entry key="ANNUAL">http://publications.europa.eu/resource/authority/frequency/ANNUAL</entry>
</xsl:variable>

<!--
https://w3id.org/mobilitydcat-ap/update-frequency
-->
<xsl:variable name="mobilityDcatApUpdateFrequencyToUri"
as="node()*">
<entry key="1h">https://w3id.org/mobilitydcat-ap/update-frequency/1h</entry>
<entry key="1min">https://w3id.org/mobilitydcat-ap/update-frequency/1min</entry>
<entry key="10min">https://w3id.org/mobilitydcat-ap/update-frequency/10min</entry>
<entry key="12h">https://w3id.org/mobilitydcat-ap/update-frequency/12h</entry>
<entry key="15min">https://w3id.org/mobilitydcat-ap/update-frequency/15min</entry>
<entry key="2h">https://w3id.org/mobilitydcat-ap/update-frequency/2h</entry>
<entry key="24h">https://w3id.org/mobilitydcat-ap/update-frequency/24h</entry>
<entry key="3h">https://w3id.org/mobilitydcat-ap/update-frequency/3h</entry>
<entry key="3 months">https://w3id.org/mobilitydcat-ap/update-frequency/3-months</entry>
<entry key="30min">https://w3id.org/mobilitydcat-ap/update-frequency/30min</entry>
<entry key="5min">https://w3id.org/mobilitydcat-ap/update-frequency/5min</entry>
<entry key="6 months">https://w3id.org/mobilitydcat-ap/update-frequency/6-months</entry>
</xsl:variable>

<xsl:variable name="isoDurationToMobilityDcatApUpdateFrequency"
as="node()*">
Expand All @@ -155,21 +219,68 @@
<entry key="P0Y6M0DT0H0M0S">https://w3id.org/mobilitydcat-ap/update-frequency/6-months</entry>
</xsl:variable>


<xsl:param name="multipleAccrualPeriodicityAllowed"
as="xs:string"
select="'true'"/>
<xsl:variable name="allowMultipleAccrualPeriodicity"
as="xs:boolean"
select="xs:boolean($multipleAccrualPeriodicityAllowed)"/>

<xsl:template mode="iso19115-3-to-dcat"
match="mdb:identificationInfo/*/mri:resourceMaintenance/*/mmi:userDefinedMaintenanceFrequency">
match="mdb:identificationInfo/*/mri:resourceMaintenance">
<xsl:variable name="isFirstMaintenanceElement"
select="preceding-sibling::*[1]/name() != 'mri:resourceMaintenance'"/>
<xsl:if test="$isFirstMaintenanceElement">
<xsl:variable name="listOfFrequency"
as="node()*">
<xsl:for-each select="../mri:resourceMaintenance/*">
<xsl:for-each select="mmi:maintenanceAndUpdateFrequency[*/@codeListValue != '']">
<xsl:variable name="dcFrequency"
as="xs:string?"
select="$isoFrequencyToDublinCore[@key = current()/*/@codeListValue]"/>

<xsl:variable name="dcatFrequencyUri"
as="xs:string?"
select="$euUpdateFrquencyToUri[@key = current()/*/@codeListValue]"/>

<xsl:variable name="mobilityDcatApUpdateFrequencyUri"
as="xs:string?"
select="$mobilityDcatApUpdateFrequencyToUri[@key = current()/*/@codeListValue]"/>

<!-- Domain specific codelists take priority -->
<xsl:variable name="dcOrIsoFrequency"
as="xs:string?"
select="if ($mobilityDcatApUpdateFrequencyUri != '' or $dcatFrequencyUri != '') then ''
else if($dcFrequency)
then concat($europaPublicationBaseUri, 'frequency/', $dcFrequency)
else concat($isoCodeListBaseUri, */@codeListValue)"/>

<xsl:for-each select="($mobilityDcatApUpdateFrequencyUri, $dcatFrequencyUri, $dcOrIsoFrequency[. != ''])">
<dct:accrualPeriodicity>
<dct:Frequency rdf:about="{current()}"/>
</dct:accrualPeriodicity>
</xsl:for-each>
</xsl:for-each>

<xsl:variable name="dcMobilityFrequency"
as="xs:string?"
select="$isoDurationToMobilityDcatApUpdateFrequency[@key = current()/gco:TM_PeriodDuration]"/>
<xsl:for-each select="mmi:userDefinedMaintenanceFrequency">
<xsl:variable name="mobilityFrequencyFromPeriod"
as="xs:string?"
select="$isoDurationToMobilityDcatApUpdateFrequency[@key = current()/gco:TM_PeriodDuration]"/>

<xsl:if test="$mobilityFrequencyFromPeriod">
<dct:accrualPeriodicity>
<dct:Frequency rdf:about="{$mobilityFrequencyFromPeriod}"/>
</dct:accrualPeriodicity>
</xsl:if>
</xsl:for-each>
</xsl:for-each>
</xsl:variable>

<xsl:if test="$dcMobilityFrequency">
<dct:accrualPeriodicity>
<dct:Frequency rdf:about="{$dcMobilityFrequency}"/>
</dct:accrualPeriodicity>
<xsl:copy-of select="if ($allowMultipleAccrualPeriodicity) then $listOfFrequency else $listOfFrequency[1]"/>
</xsl:if>
</xsl:template>


<!--
RDF Property: dcat:temporalResolution
Definition: Minimum time period resolvable in the dataset.
Expand All @@ -179,7 +290,9 @@
-->
<xsl:template mode="iso19115-3-to-dcat"
match="mri:temporalResolution/*">
<dcat:temporalResolution rdf:datatype="http://www.w3.org/2001/XMLSchema#duration"><xsl:value-of select="text()"/></dcat:temporalResolution>
<dcat:temporalResolution rdf:datatype="http://www.w3.org/2001/XMLSchema#duration">
<xsl:value-of select="text()"/>
</dcat:temporalResolution>
</xsl:template>


Expand Down Expand Up @@ -219,9 +332,9 @@
<xsl:template mode="iso19115-3-to-dcat"
match="mri:extent/*/gex:geographicElement/gex:EX_GeographicBoundingBox">
<xsl:variable name="north" select="gex:northBoundLatitude/gco:Decimal"/>
<xsl:variable name="east" select="gex:eastBoundLongitude/gco:Decimal"/>
<xsl:variable name="east" select="gex:eastBoundLongitude/gco:Decimal"/>
<xsl:variable name="south" select="gex:southBoundLatitude/gco:Decimal"/>
<xsl:variable name="west" select="gex:westBoundLongitude/gco:Decimal"/>
<xsl:variable name="west" select="gex:westBoundLongitude/gco:Decimal"/>

<xsl:variable name="geojson"
as="xs:string"
Expand All @@ -235,7 +348,9 @@
<dct:spatial>
<rdf:Description>
<rdf:type rdf:resource="http://purl.org/dc/terms/Location"/>
<dcat:bbox rdf:datatype="http://www.opengis.net/ont/geosparql#geoJSONLiteral"><xsl:value-of select="$geojson"/></dcat:bbox>
<dcat:bbox rdf:datatype="http://www.opengis.net/ont/geosparql#geoJSONLiteral">
<xsl:value-of select="$geojson"/>
</dcat:bbox>
</rdf:Description>
</dct:spatial>
</xsl:template>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -122,8 +122,7 @@
* prov:wasGeneratedBy (Could be associated resource of type project?)
-->
<xsl:apply-templates mode="iso19115-3-to-dcat"
select="mdb:identificationInfo/*/mri:resourceMaintenance/*/mmi:maintenanceAndUpdateFrequency
|mdb:identificationInfo/*/mri:resourceMaintenance/*/mmi:userDefinedMaintenanceFrequency
select="mdb:identificationInfo/*/mri:resourceMaintenance
|mdb:identificationInfo/*/mri:spatialResolution/*/mri:distance
|mdb:identificationInfo/*/mri:temporalResolution/*
|mdb:identificationInfo/*/mri:extent/*/gex:geographicElement/gex:EX_GeographicBoundingBox
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,10 @@
<xsl:import href="../dcat/dcat-core.xsl"/>
<xsl:import href="eu-dcat-ap-core-dataset.xsl"/>

<xsl:param name="multipleAccrualPeriodicityAllowed"
as="xs:string"
select="'false'"/>

<!--
If true, all resource constraints are preserved in the output.
The first one is a license, others are dct:rights.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -88,6 +88,7 @@ public static Collection<String[]> data() throws Exception {

data.add(new String[]{"iso19115-3.2018-dcat-dataset.xml", "dcat", "", "iso19115-3.2018", "dataset-core.rdf"});
data.add(new String[]{"iso19115-3.2018-dcat-dataset.xml", "eu-dcat-ap", "", "iso19115-3.2018", "dataset-core.rdf"});
data.add(new String[]{"iso19115-3.2018-dcat-dataset.xml", "eu-dcat-ap", "?multipleAccrualPeriodicityAllowed=true", "iso19115-3.2018", "dataset-core-multipleAccrualPeriodicityAllowed.rdf"});
data.add(new String[]{"iso19115-3.2018-dcat-dataset.xml", "eu-geodcat-ap", "", "iso19115-3.2018", "dataset-core.rdf"});
data.add(new String[]{"iso19115-3.2018-dcat-dataset.xml", "eu-dcat-ap-mobility", "", "iso19115-3.2018", "dataset-core.rdf"});
data.add(new String[]{"iso19115-3.2018-dcat-dataset.xml", "eu-dcat-ap-hvd", "", "iso19115-3.2018", "dataset-core.rdf"});
Expand Down Expand Up @@ -151,7 +152,7 @@ public void checkFormatter() throws Exception {
}


// FileUtils.writeStringToFile(new File("/tmp/services/src/test/resources/org/fao/geonet/api/records/formatters/" + String.format("%s-%s-%s",
// FileUtils.writeStringToFile(new File("/tmp/services/src/test/resources/org/fao/geonet/api/records/formatters/new/" + String.format("%s-%s-%s",
// schema, formatter, checkfile)), actual.replaceFirst("urn:uuid/.*</dct:identifier>", "urn:uuid/{uuid}</dct:identifier>"), StandardCharsets.UTF_8);

Diff diff = DiffBuilder
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -454,6 +454,9 @@
<dct:accrualPeriodicity>
<dct:Frequency rdf:about="http://standards.iso.org/iso/19115/resources/Codelists/cat/codelists.xml#asNeeded"/>
</dct:accrualPeriodicity>
<dct:accrualPeriodicity>
<dct:Frequency rdf:about="https://w3id.org/mobilitydcat-ap/update-frequency/5min"/>
</dct:accrualPeriodicity>
<dct:accrualPeriodicity>
<dct:Frequency rdf:about="https://w3id.org/mobilitydcat-ap/update-frequency/15min"/>
</dct:accrualPeriodicity>
Expand Down Expand Up @@ -552,4 +555,4 @@
</foaf:Document>
</foaf:page>
</rdf:Description>
</rdf:RDF>
</rdf:RDF>
Original file line number Diff line number Diff line change
@@ -1,29 +1,18 @@
<mdb:MD_Metadata xmlns:mri="http://standards.iso.org/iso/19115/-3/mri/1.0"
xmlns:mda="http://standards.iso.org/iso/19115/-3/mda/1.0"
xmlns:mrl="http://standards.iso.org/iso/19115/-3/mrl/2.0"
xmlns:mrs="http://standards.iso.org/iso/19115/-3/mrs/1.0"
xmlns:mds="http://standards.iso.org/iso/19115/-3/mds/2.0"
xmlns:mrc="http://standards.iso.org/iso/19115/-3/mrc/2.0"
xmlns:gco="http://standards.iso.org/iso/19115/-3/gco/1.0"
xmlns:gfc="http://standards.iso.org/iso/19110/gfc/1.1"
xmlns:mrd="http://standards.iso.org/iso/19115/-3/mrd/1.0"
xmlns:cat="http://standards.iso.org/iso/19115/-3/cat/1.0"
xmlns:mco="http://standards.iso.org/iso/19115/-3/mco/1.0"
xmlns:mdt="http://standards.iso.org/iso/19115/-3/mdt/2.0"
xmlns:msr="http://standards.iso.org/iso/19115/-3/msr/2.0"
xmlns:mpc="http://standards.iso.org/iso/19115/-3/mpc/1.0"
xmlns:srv="http://standards.iso.org/iso/19115/-3/srv/2.1"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:lan="http://standards.iso.org/iso/19115/-3/lan/1.0"
xmlns:mac="http://standards.iso.org/iso/19115/-3/mac/2.0"
xmlns:gcx="http://standards.iso.org/iso/19115/-3/gcx/1.0"
xmlns:xlink="http://www.w3.org/1999/xlink"
xmlns:gex="http://standards.iso.org/iso/19115/-3/gex/1.0"
xmlns:gml="http://www.opengis.net/gml/3.2"
xmlns:mdb="http://standards.iso.org/iso/19115/-3/mdb/2.0"
xmlns:mex="http://standards.iso.org/iso/19115/-3/mex/1.0"
xmlns:cit="http://standards.iso.org/iso/19115/-3/cit/2.0"
xmlns:mas="http://standards.iso.org/iso/19115/-3/mas/1.0"
xmlns:mcc="http://standards.iso.org/iso/19115/-3/mcc/1.0"
xmlns:mmi="http://standards.iso.org/iso/19115/-3/mmi/1.0"
xmlns:mdq="http://standards.iso.org/iso/19157/-2/mdq/1.0"
Expand Down Expand Up @@ -591,11 +580,24 @@
codeList="http://standards.iso.org/iso/19115/resources/Codelists/cat/codelists.xml#MD_MaintenanceFrequencyCode"
codeListValue="monthly"/>
</mmi:maintenanceAndUpdateFrequency>
</mmi:MD_MaintenanceInformation>
</mri:resourceMaintenance>
<mri:resourceMaintenance>
<mmi:MD_MaintenanceInformation>
<mmi:maintenanceAndUpdateFrequency>
<mmi:MD_MaintenanceFrequencyCode
codeList="http://standards.iso.org/iso/19115/resources/Codelists/cat/codelists.xml#MD_MaintenanceFrequencyCode"
codeListValue="asNeeded"/>
</mmi:maintenanceAndUpdateFrequency>
</mmi:MD_MaintenanceInformation>
</mri:resourceMaintenance>
<mri:resourceMaintenance>
<mmi:MD_MaintenanceInformation>
<mmi:maintenanceAndUpdateFrequency>
<mmi:MD_MaintenanceFrequencyCode
codeList="http://standards.iso.org/iso/19115/resources/Codelists/cat/codelists.xml#MD_MaintenanceFrequencyCode"
codeListValue="5min"/>
</mmi:maintenanceAndUpdateFrequency>
<mmi:userDefinedMaintenanceFrequency>
<gco:TM_PeriodDuration>P0Y0M0DT0H15M0S</gco:TM_PeriodDuration>
</mmi:userDefinedMaintenanceFrequency>
Expand Down
Loading

0 comments on commit 12ffbed

Please sign in to comment.