-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Restructure claim transformers, make them more Rubyish (#1)
* Demo: Claim-type-specific transformers * Restructure claim transformers, make them more Rubyish
- Loading branch information
1 parent
76afedb
commit 0d111dd
Showing
47 changed files
with
868 additions
and
831 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,12 +0,0 @@ | ||
# This configuration was generated by | ||
# `rubocop --auto-gen-config` | ||
# on 2025-01-14 03:47:56 UTC using RuboCop version 1.69.2. | ||
# The point is for the user to remove these configuration records | ||
# one by one as the offenses are removed from the code base. | ||
# Note that changes in the inspected code, or installation of new | ||
# versions of RuboCop, may require this file to be generated again. | ||
|
||
# Offense count: 1 | ||
# Configuration parameters: CountComments, CountAsOne. | ||
Metrics/ModuleLength: | ||
Max: 269 | ||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file was deleted.
Oops, something went wrong.
16 changes: 16 additions & 0 deletions
16
lib/digital_scriptorium/transformers/acknowledgements_claim_transformer.rb
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,16 @@ | ||
# frozen_string_literal: true | ||
|
||
module DigitalScriptorium | ||
# Base transformer class providing a common interface for all transformers. | ||
class AcknowledgementsClaimTransformer < BaseClaimTransformer | ||
PREFIX = 'acknowledgements' | ||
|
||
def initialize(claim, _) | ||
super(claim, prefix: PREFIX) | ||
end | ||
|
||
def display_values | ||
[display_value(@claim.data_value)] | ||
end | ||
end | ||
end |
42 changes: 42 additions & 0 deletions
42
lib/digital_scriptorium/transformers/base_claim_transformer.rb
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,42 @@ | ||
# frozen_string_literal: true | ||
|
||
module DigitalScriptorium | ||
# Base transformer class providing a common interface for all transformers. | ||
class BaseClaimTransformer | ||
def initialize(claim, **kwargs) | ||
@claim = claim | ||
@prefix = kwargs[:prefix] | ||
end | ||
|
||
def display_values | ||
[] | ||
end | ||
|
||
def search_values | ||
[] | ||
end | ||
|
||
def facet_values | ||
[] | ||
end | ||
|
||
def extra_props | ||
{} | ||
end | ||
|
||
def display_value(recorded_value, in_original_script = nil, linked_terms = []) | ||
value = { 'recorded_value' => recorded_value } | ||
value['original_script'] = in_original_script if in_original_script | ||
value['linked_terms'] = linked_terms if linked_terms.any? | ||
value.to_json | ||
end | ||
|
||
def solr_props | ||
solr_props = {} | ||
solr_props["#{@prefix}_display"] = display_values if display_values.any? | ||
solr_props["#{@prefix}_search"] = search_values if search_values.any? | ||
solr_props["#{@prefix}_facet"] = facet_values if facet_values.any? | ||
solr_props.merge(extra_props) | ||
end | ||
end | ||
end |
18 changes: 18 additions & 0 deletions
18
lib/digital_scriptorium/transformers/date_claim_transformer.rb
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,18 @@ | ||
# frozen_string_literal: true | ||
|
||
module DigitalScriptorium | ||
# Base transformer class providing a common interface for all transformers. | ||
class DateClaimTransformer < QualifiedClaimTransformer | ||
include PropertyId | ||
|
||
PREFIX = 'date' | ||
|
||
def initialize(claim, export_hash) | ||
super(claim, export_hash, prefix: PREFIX, authority_id: PRODUCTION_CENTURY_IN_AUTHORITY_FILE) | ||
end | ||
|
||
def extra_props | ||
{ 'date_meta' => [@claim.data_value] } | ||
end | ||
end | ||
end |
16 changes: 16 additions & 0 deletions
16
lib/digital_scriptorium/transformers/dated_claim_transformer.rb
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,16 @@ | ||
# frozen_string_literal: true | ||
|
||
module DigitalScriptorium | ||
# Base transformer class providing a common interface for all transformers. | ||
class DatedClaimTransformer < BaseClaimTransformer | ||
PREFIX = 'dated' | ||
|
||
def initialize(claim, _export_hash) | ||
super(claim, prefix: PREFIX) | ||
end | ||
|
||
def facet_values | ||
[export_hash[@claim.entity_id_value]&.label('en')].compact | ||
end | ||
end | ||
end |
16 changes: 16 additions & 0 deletions
16
lib/digital_scriptorium/transformers/iiif_manifest_claim_transformer.rb
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,16 @@ | ||
# frozen_string_literal: true | ||
|
||
module DigitalScriptorium | ||
# Transformer for extracting links from relevant Digital Scriptorium claims. | ||
class IiifManifestClaimTransformer < LinkClaimTransformer | ||
PREFIX = 'iiif_manifest' | ||
|
||
def initialize(claim, _) | ||
super(claim, prefix: PREFIX) | ||
end | ||
|
||
def extra_props | ||
super.merge({ 'images_facet' => ['Yes'] }) | ||
end | ||
end | ||
end |
14 changes: 14 additions & 0 deletions
14
lib/digital_scriptorium/transformers/institution_claim_transformer.rb
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,14 @@ | ||
# frozen_string_literal: true | ||
|
||
module DigitalScriptorium | ||
# Base transformer class providing a common interface for all transformers. | ||
class InstitutionClaimTransformer < QualifiedClaimTransformer | ||
include PropertyId | ||
|
||
PREFIX = 'institution' | ||
|
||
def initialize(claim, export_hash) | ||
super(claim, export_hash, prefix: PREFIX, authority_id: HOLDING_INSTITUTION_IN_AUTHORITY_FILE) | ||
end | ||
end | ||
end |
12 changes: 12 additions & 0 deletions
12
lib/digital_scriptorium/transformers/institutional_record_claim_transformer.rb
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,12 @@ | ||
# frozen_string_literal: true | ||
|
||
module DigitalScriptorium | ||
# Transformer for extracting links from relevant Digital Scriptorium claims. | ||
class InstitutionalRecordClaimTransformer < LinkClaimTransformer | ||
PREFIX = 'institutional_record' | ||
|
||
def initialize(claim, _) | ||
super(claim, prefix: PREFIX) | ||
end | ||
end | ||
end |
14 changes: 14 additions & 0 deletions
14
lib/digital_scriptorium/transformers/language_claim_transformer.rb
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,14 @@ | ||
# frozen_string_literal: true | ||
|
||
module DigitalScriptorium | ||
# Transformer for converting qualified claims of Digital Scriptorium items into Solr fields. | ||
class LanguageClaimTransformer < QualifiedClaimTransformer | ||
include PropertyId | ||
|
||
PREFIX = 'language' | ||
|
||
def initialize(claim, export_hash) | ||
super(claim, export_hash, prefix: PREFIX, authority_id: LANGUAGE_IN_AUTHORITY_FILE) | ||
end | ||
end | ||
end |
13 changes: 4 additions & 9 deletions
13
lib/digital_scriptorium/transformers/link_claim_transformer.rb
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,15 +1,10 @@ | ||
# frozen_string_literal: true | ||
|
||
module DigitalScriptorium | ||
# Transformer for extracting links from relevant Digital Scriptorium items. | ||
class LinkClaimTransformer | ||
include PropertyId | ||
|
||
def self.transform(claim, config) | ||
solr_props = {} | ||
solr_props['images_facet'] = ['Yes'] if claim.data_value && claim.property_id == IIIF_MANIFEST | ||
solr_props["#{config['prefix']}_link"] = [claim.data_value] | ||
solr_props | ||
# Transformer for extracting links from relevant Digital Scriptorium claims. | ||
class LinkClaimTransformer < BaseClaimTransformer | ||
def extra_props | ||
{ "#{@prefix}_link" => [@claim.data_value] } | ||
end | ||
end | ||
end |
14 changes: 14 additions & 0 deletions
14
lib/digital_scriptorium/transformers/material_claim_transformer.rb
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,14 @@ | ||
# frozen_string_literal: true | ||
|
||
module DigitalScriptorium | ||
# Transformer for converting qualified claims of Digital Scriptorium items into Solr fields. | ||
class MaterialClaimTransformer < QualifiedClaimTransformer | ||
include PropertyId | ||
|
||
PREFIX = 'material' | ||
|
||
def initialize(claim, export_hash) | ||
super(claim, export_hash, prefix: PREFIX, authority_id: MATERIAL_IN_AUTHORITY_FILE) | ||
end | ||
end | ||
end |
62 changes: 10 additions & 52 deletions
62
lib/digital_scriptorium/transformers/name_claim_transformer.rb
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,65 +1,23 @@ | ||
# frozen_string_literal: true | ||
|
||
module DigitalScriptorium | ||
# Special-purpose transformer for name (P14) claims | ||
class NameClaimTransformer | ||
# Transformer for name (P14) claims. | ||
# NOTE: Name claims produce fields prefixes derived from the value of their role (P15) qualifiers | ||
# (owner, author, scribe, artist, agent). | ||
class NameClaimTransformer < QualifiedClaimTransformerWithFacetFallback | ||
include PropertyId | ||
|
||
def self.transform(claim, export_hash) | ||
# TODO: Log a warning to stdout if this qualifier is missing, it's a mistake that should be addressed | ||
return {} unless claim.qualifiers_by_property_id? ROLE_IN_AUTHORITY_FILE | ||
|
||
prefix = get_role_prefix(claim, export_hash) | ||
recorded_value = claim.data_value | ||
original_script = claim.qualifiers_by_property_id(IN_ORIGINAL_SCRIPT)&.first&.data_value&.value | ||
linked_terms = get_linked_terms(claim, export_hash) | ||
|
||
build_solr_props(prefix, recorded_value, original_script, linked_terms) | ||
def initialize(claim, export_hash) | ||
super(claim, export_hash, prefix: role_prefix(claim, export_hash), authority_id: NAME_IN_AUTHORITY_FILE) | ||
end | ||
|
||
def self.build_solr_props(prefix, recorded_value, original_script, linked_terms) | ||
linked_term_labels = get_labels(linked_terms) | ||
|
||
{ | ||
"#{prefix}_display" => [{ | ||
'recorded_value' => recorded_value, | ||
'original_script' => original_script, | ||
'linked_terms' => linked_terms.any? ? linked_terms : nil | ||
}.compact.to_json], | ||
"#{prefix}_search" => ([recorded_value, original_script].compact + linked_term_labels).uniq, | ||
"#{prefix}_facet" => linked_term_labels.any? ? linked_term_labels : [recorded_value] | ||
} | ||
end | ||
|
||
def self.get_linked_terms(claim, export_hash) | ||
linked_terms = [] | ||
|
||
claim.qualifiers_by_property_id(NAME_IN_AUTHORITY_FILE)&.each do |qualifier| | ||
authority_id = qualifier.entity_id_value | ||
authority = export_hash[authority_id] | ||
linked_terms << get_linked_term(authority) if authority | ||
end | ||
|
||
linked_terms.uniq | ||
end | ||
|
||
def self.get_linked_term(authority) | ||
term = { 'label' => authority.label('en') } | ||
wikidata_id = authority.claims_by_property_id(WIKIDATA_QID)&.first&.data_value | ||
wikidata_uri = wikidata_id && "https://www.wikidata.org/wiki/#{wikidata_id}" | ||
term['source_url'] = wikidata_uri | ||
term.compact | ||
end | ||
|
||
def self.get_role_prefix(claim, export_hash) | ||
def role_prefix(claim, export_hash) | ||
role_entity_id = claim.qualifiers_by_property_id(ROLE_IN_AUTHORITY_FILE).first.entity_id_value | ||
role_item = export_hash[role_entity_id] | ||
role_label = role_item.label('en') | ||
role_label.downcase.split.last | ||
end | ||
|
||
def self.get_labels(linked_terms) | ||
linked_terms.map { |term| term['label'] }.uniq | ||
role_label.split.last.downcase | ||
rescue e | ||
raise "Error fetching role qualifier: #{e}" | ||
end | ||
end | ||
end |
20 changes: 20 additions & 0 deletions
20
lib/digital_scriptorium/transformers/note_claim_transformer.rb
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,20 @@ | ||
# frozen_string_literal: true | ||
|
||
module DigitalScriptorium | ||
# Base transformer class providing a common interface for all transformers. | ||
class NoteClaimTransformer < BaseClaimTransformer | ||
PREFIX = 'note' | ||
|
||
def initialize(claim, _) | ||
super(claim, prefix: PREFIX) | ||
end | ||
|
||
def display_values | ||
[display_value(@claim.data_value)] | ||
end | ||
|
||
def search_values | ||
[@claim.data_value] | ||
end | ||
end | ||
end |
20 changes: 20 additions & 0 deletions
20
lib/digital_scriptorium/transformers/physical_description_claim_transformer.rb
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,20 @@ | ||
# frozen_string_literal: true | ||
|
||
module DigitalScriptorium | ||
# Base transformer class providing a common interface for all transformers. | ||
class PhysicalDescriptionClaimTransformer < BaseClaimTransformer | ||
PREFIX = 'physical_description' | ||
|
||
def initialize(claim, _) | ||
super(claim, prefix: PREFIX) | ||
end | ||
|
||
def display_values | ||
[display_value(@claim.data_value)] | ||
end | ||
|
||
def search_values | ||
[@claim.data_value] | ||
end | ||
end | ||
end |
14 changes: 14 additions & 0 deletions
14
lib/digital_scriptorium/transformers/place_claim_transformer.rb
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,14 @@ | ||
# frozen_string_literal: true | ||
|
||
module DigitalScriptorium | ||
# Transformer for converting qualified claims of Digital Scriptorium items into Solr fields. | ||
class PlaceClaimTransformer < QualifiedClaimTransformer | ||
include PropertyId | ||
|
||
PREFIX = 'place' | ||
|
||
def initialize(claim, export_hash) | ||
super(claim, export_hash, prefix: PREFIX, authority_id: PLACE_IN_AUTHORITY_FILE) | ||
end | ||
end | ||
end |
Oops, something went wrong.