Querying Relationship Annotations for Rhetoric Text Structure Analysis

[yanirmr]

Hello,

I'm embarking on a project focused on tagging relationships between entities to annotate the structure of rhetorical texts. The goal is to identify and analyze structures such as claims, rhetorical questions, facts, opinions, declines of hypothetical claims, regard for other speakers, and more. While I believe I have a schema that suits my needs, I have a few queries regarding the post-annotation phase:

1. Querying the Tagged Dataset: How can I query the tagged dataset once the annotation is complete? Are there built-in tools within INCEpTION, or would I need to export the data and use external tools to analyze the annotations?

2. Examples: Are there any exemplary projects or datasets that demonstrate the usability of relationship annotations, especially in the context of rhetorical text structures? Any references would be greatly appreciated.

3. Analysis of Structures and Sequences: I'm interested in analyzing the frequency of certain structures and sequences in the annotated data. Is there a recommended approach for this within INCEpTION, or should I consider external tools or scripts?

4. Effort Estimation: Given my objective to identify specific rhetorical patterns based on the annotations, how much effort would be required to query and analyze the data? Does INCEpTION have any built-in features to assist in this, or should I be prepared to implement a custom querying system?

Any guidance or insights would be invaluable. Thank you for your time and expertise.

Best regards,

[reckart]

Querying

INCEpTION includes a search sidebar that allows you to search within the annotations of the current user. Beyond that, there is currently no functionality to query the data.

One way to query the data would be to export the project in CAS XMI (XML 1.0) format and then read the data using the dkpro-cassis Python library with a script that extracts interesting properties and turns them into a suitable report. You might find the INCEpTALYTICS repo a useful starting point for such efforts.

Alternatively, you could consider converting the data into a format that could be imported e.g. into BlackLab or ANNIS. The search built into INCEpTION is currently powered by MTAS, but I am considering switching to BlackLab in the future.

Relationships / rhethorical structure

I am not aware of any publicly available dataset which used INCEpTION for RST annotations. You might consider having a look at the semantic frame annotation example project. It showcases not the relation layers but rather how to use link features - which you might find useful alternative to relations.

Frequency analysis

See answer regarding querying above.

Effort estimation

You need to be prepared to convert / analyze your data externally. Currently, the assumption is that the kinds of analyses that people may want to do on their data may vary greatly and it would be quite some effort to implement and maintain a highly flexible system within INCEpTION that would cover all imaginable cases. That said, I would be interested in learning more about the kinds of analyses that people do want to perform to get a better idea of whether there are certain kinds of basic analyses which might be added directly into INCEpTION.

[yanirmr]

Hello Richard,

Thank you for your detailed response to my queries. I'd like to address a few points to ensure we're aligned.

Firstly, as you might recall, I have experience working with INCEpTION. However, this is my maiden venture into tagging relationships, and my primary goal is to create a corpus for analysis rather than training a learning model. This presents new challenges for me.

Regarding exporting data, I've come to understand that it's impractical to expect everything to reside within INCEpTION given the multifaceted and evolving needs. I'm now looking into viable query tools, or potentially building one if needed.

On the technical side, while I'm quite proficient with tools and libraries, our current project scope limits the amount of time and resources we can allocate to development. This necessitates a solution that's easy to implement and doesn't heavily rely on a programmer's extended involvement.

I appreciate your suggestion of the INCEpTALYTICS repo. However, my initial impressions indicate that it provides a rather elementary statistical overview and lacks support for advanced functionalities. I also noticed its limited maintenance activity, which concerns me given that our project is set to span over the next year or two. Such indicators raise red flags for long-term viability.

While I've not delved into tools like BlackLab or ANNIS yet, I'm open to exploring them and will be testing their capabilities soon.

I'm specifically interested in analyzing the structural sequences of entities and the relationships between them. These weren't highlighted in your previous message, so any guidance or references about tools or methods to achieve this would be invaluable.

Considering INCEpTION's limitations, while some were familiar to me, others became more apparent after your explanation. I'm assessing the impact of these limitations on our project timeline and objectives.

Lastly, I want to emphasize that we're aiming for a long-term solution. With our project's duration spanning over the next couple of years, it's crucial for us to adopt tools and methods that remain relevant and supported throughout this period.

Thank you again for your insights. Looking forward to any further guidance or recommendations you might have.

Best regards.