Gem breaking on extraneous attributes

[opoudjis]

In pursuit of metanorma/metanorma#75, I created the following document, as instructed:

= X
:doctype: standard
:glossarist-dataset: dataset:../isotc211-glossary/geolexica

== Terms and definitions

glossarist::import[dataset]

This is the barest of bare minimums of integration.

I get:

/Users/nickn/.rbenv/versions/3.3.3/lib/ruby/gems/3.3.0/gems/glossarist-2.0.10/lib/glossarist/model.rb:19:in `public_send': undefined method `status=' for an instance of Glossarist::LocalizedConcept (NoMethodError)

      public_send("#{name}=", value)
      ^^^^^^^^^^^
	from /Users/nickn/.rbenv/versions/3.3.3/lib/ruby/gems/3.3.0/gems/glossarist-2.0.10/lib/glossarist/model.rb:19:in `set_attribute'
	from /Users/nickn/.rbenv/versions/3.3.3/lib/ruby/gems/3.3.0/gems/glossarist-2.0.10/lib/glossarist/model.rb:15:in `block in initialize'
	from /Users/nickn/.rbenv/versions/3.3.3/lib/ruby/gems/3.3.0/gems/glossarist-2.0.10/lib/glossarist/model.rb:15:in `each_pair'
	from /Users/nickn/.rbenv/versions/3.3.3/lib/ruby/gems/3.3.0/gems/glossarist-2.0.10/lib/glossarist/model.rb:15:in `initialize'
	from /Users/nickn/.rbenv/versions/3.3.3/lib/ruby/gems/3.3.0/gems/glossarist-2.0.10/lib/glossarist/concept.rb:66:in `initialize'
	from /Users/nickn/.rbenv/versions/3.3.3/lib/ruby/gems/3.3.0/gems/glossarist-2.0.10/lib/glossarist/model.rb:11:in `new'
	from /Users/nickn/.rbenv/versions/3.3.3/lib/ruby/gems/3.3.0/gems/glossarist-2.0.10/lib/glossarist/model.rb:11:in `new'
	from /Users/nickn/.rbenv/versions/3.3.3/lib/ruby/gems/3.3.0/gems/glossarist-2.0.10/lib/glossarist/concept_manager.rb:58:in `load_localized_concept'
	from /Users/nickn/.rbenv/versions/3.3.3/lib/ruby/gems/3.3.0/gems/glossarist-2.0.10/lib/glossarist/concept_manager.rb:42:in `block in load_concept_from_file'
	from /Users/nickn/.rbenv/versions/3.3.3/lib/ruby/gems/3.3.0/gems/glossarist-2.0.10/lib/glossarist/concept_manager.rb:41:in `each'
	from /Users/nickn/.rbenv/versions/3.3.3/lib/ruby/gems/3.3.0/gems/glossarist-2.0.10/lib/glossarist/concept_manager.rb:41:in `load_concept_from_file'
	from /Users/nickn/.rbenv/versions/3.3.3/lib/ruby/gems/3.3.0/gems/glossarist-2.0.10/lib/glossarist/concept_manager.rb:24:in `block in load_from_files'
	from <internal:dir>:411:in `glob'
	from /Users/nickn/.rbenv/versions/3.3.3/lib/ruby/gems/3.3.0/gems/glossarist-2.0.10/lib/glossarist/concept_manager.rb:20:in `load_from_files'
	from /Users/nickn/.rbenv/versions/3.3.3/lib/ruby/gems/3.3.0/gems/glossarist-2.0.10/lib/glossarist/managed_concept_collection.rb:75:in `load_from_files'

And indeed, status is not defined in either Concept, or LocalizedConcept; only entry_status is defined in the latter.

status in the localized-concept YAMLs of isotc211-glossary is set to be "valid".

I don't know anything about the termbases we are publishing, and I'm not going to: I only care about Metanorma. What I do see though is that random YAML is going to be fed into this gem from all sorts of sources, and this gem must be defensive about what it reads in that YAML, or it simply cannot be used.

Which means that "fix the isotc211-glossary dataset" is not the right response to this, even if you end up doing that anyway. (And I would not: saying a term is valid is entirely legit.) The right response is to anticipate that the YAML that will be read in by this gem will contain all sorts of extraneous fields, and to make sure that Glossarist ignores them, rather than naively trying to shove them into its model.

I do need to add that if glossarist-ruby and metanorma-plugin-glossarist are going to be made available to the Metanorma stack, there is an obligation to do integration testing, including with such new termbases as we publish. We should not be making https://github.com/geolexica/isotc211-glossary available without ensuring that glossarist-ruby can read it: that is our organisational responsibility, as publishers of both.

[opoudjis]

FWIW, this issue is not occurring with isotc204-glossary. A different bug is, and I will be reporting that too.

[HassanAkbar]

@opoudjis Glossarist supports a config file which can be used to define extra attributes that are not in the concept model.

For that we just need to create a glossarist.yaml file and add extension_attributes in it. Here is an example of glossarist.yaml in isotc211.geolexica.org repo.

[opoudjis]

Ok, so:

• If I use that new file, this issue goes away
• All files that will be consumed by :glossarist-dataset: must have extension_attributes provided where needed
• This behaviour will be documented somewhere—in addition to me mentioning it in metanorma.org

Correct?

[HassanAkbar]

@opoudjis

All files that will be consumed by :glossarist-dataset: must have extension_attributes provided where needed

Correct, We can have a glossarist.yaml file in the root folder from where the command is executed. It will have a list of all the extension attributes needed for all the files in the dataset.

This behaviour will be documented somewhere—in addition to me mentioning it in metanorma.org

Yes, I will update the documentation, once I finalized the current task related to lutaml/lutaml-model#2. I've added a ticket for that here -> #105

[opoudjis]

And glossarist.yaml tells it that status is ok as a key? OK, but what about nested hashes? Surely you're going to want some sort of JSON Path in there? This doesn't look very scalable. Not to mention that the config file should probably be named as an argument in the :glossarist-dataset: attribute: hardcoding the file name is not a good idea.

Anyway, the document is compiling now...

[ronaldtse]

So this problem will be fixed when we migrate to lutaml-model. I just ran into this exact same problem when trying to use a Glossarist dataset that can be loaded by Paneron.