tagUsage calculation in AT corpus

[matyaskopp]

AT corpus has wrong numbers in tagUsage in /project/corpora/Parla/ParlaMint/ParlaMint-full/Data/Corpora folder:

• ParlaMint-AT.TEI.ana/ParlaMint-AT.ana.xml
• ParlaMint-AT.TEI/ParlaMint-AT.xml
• Sample-ParlaMint-AT.TEI.ana/ParlaMint-AT.ana.xml
• Sample-ParlaMint-AT.TEI.ana/ParlaMint-AT.ana.xml

All corpus files look like this:

ParlaMint/Data/ParlaMint-AT/ParlaMint-AT.xml

Lines 112 to 128 in 392e2ee

	<tagsDecl><!--These numbers do not reflect the size of the sample!-->
	<namespace name="http://www.tei-c.org/ns/1.0">
	<tagUsage gi="body" occurs="1197"/>
	<tagUsage gi="desc" occurs="1197"/>
	<tagUsage gi="div" occurs="1197"/>
	<tagUsage gi="gap" occurs="1197"/>
	<tagUsage gi="incident" occurs="1197"/>
	<tagUsage gi="kinesic" occurs="1197"/>
	<tagUsage gi="note" occurs="1197"/>
	<tagUsage gi="pb" occurs="1197"/>
	<tagUsage gi="seg" occurs="1197"/>
	<tagUsage gi="text" occurs="1197"/>
	<tagUsage gi="time" occurs="1197"/>
	<tagUsage gi="u" occurs="1197"/>
	<tagUsage gi="vocal" occurs="1197"/>
	</namespace>
	</tagsDecl>

And component files:

ParlaMint/Data/ParlaMint-AT/ParlaMint-AT_2005-03-31-022-XXII-NRSITZ-00100.xml

Lines 110 to 126 in 392e2ee

	<tagsDecl><!--These numbers do not reflect the size of the sample!-->
	<namespace name="http://www.tei-c.org/ns/1.0">
	<tagUsage gi="text" occurs="1"/>
	<tagUsage gi="body" occurs="1"/>
	<tagUsage gi="div" occurs="1"/>
	<tagUsage gi="note" occurs="1"/>
	<tagUsage gi="pb" occurs="1"/>
	<tagUsage gi="u" occurs="1"/>
	<tagUsage gi="seg" occurs="1"/>
	<tagUsage gi="kinesic" occurs="1"/>
	<tagUsage gi="vocal" occurs="1"/>
	<tagUsage gi="incident" occurs="1"/>
	<tagUsage gi="gap" occurs="1"/>
	<tagUsage gi="desc" occurs="1"/>
	<tagUsage gi="time" occurs="1"/>
	</namespace>
	</tagsDecl>

I guess that the finalization script does not calculate these numbers and only AT set 1 into component files

[matyaskopp]

Now I see:

ParlaMint/Scripts/parlamint2distro.pl

Line 131 in 819add4

$scriptFinal = "$Bin/parlamint2final.xsl";

ParlaMint/Scripts/parlamint2final.xsl

Line 21 in 819add4

- inserts tagCounts in root (taken from component files and not changed there!)

parlamint2final is not calculating tagUsage

---

tagUsage calculation is implemented in

ParlaMint/Scripts/parlamint-add-common-content.xsl

Line 12 in 819add4

- calculate tagUsage

which is not used in the finalization

[TomazErjavec]

I thought everybody computes their tagUsages, but notied AT a couple of day ago myself.
I now inserted your calculation into finalize but it is a doomed effort, because I change the countable markup for ES-GA and now also IS (names without words, a but which floated to the top only in the MTed corpus), hm. I guess we should do my fixings first, and then just use add-common (although my version of add-common does things yours doesn't:).
Would you dare try it, or is that too much to hope for, I'm afraid of introducing even more confusion!
Or maybe we live with the fact that tagusages will be slightly off for 3.0, and hope to do better in 3.1?

[TomazErjavec]

Discussion on this continues in #675, closing this one.