story/VOGRE-7 #15

Girik1105 · 2024-11-05T15:10:40Z

Guidelines for Pull Requests

If you haven't yet read our code review guidelines, please do so, You can find them here.

Please confirm the following by adding an x for each item (turn [ ] into [x]).

I have removed all code style changes that are not necessary (e.g. changing blanks across the whole file that don’t need to be changed, adding empty lines in parts other than your own code)
I am not making any changes to files that don’t have any effect (e.g. imports added that don’t need to be added)
I do not have any sysout statements in my code or commented out code that isn’t needed anymore
I am not reformatting any files in the wrong format or without cause.
I am not changing file encoding or line endings to something else than UTF-8, LF
My pull request does not show an insane amount of files being changed although my ticket only requires a few files being changed
I have added Javadoc/documentation where appropriate
I have added test cases where appropriate
I have explained any part of my code/implementation decisions that is not be self-explanatory

Please provide a brief description of your ticket

Get rid of celery and submit quaduples on request
Description

Currently, quadruples are submitted via Celery on a cron job. Celery should be completely removed. Instead a project should have a button that will submit quadrupes to Quadriga. This ticket includes multiple tasks:
Quadruples need to have a flag (if they don’t already) that indicates if they ahve been submitted or not
Quaduples need to be submitted to the new Quadriga 2.0
Only quadruples with complete relationships (in which concepts have been resolved and added to conceptpower) should be submitted.

VOGRE-7

Are there any other pull requests that this one depends on?

VOGRE-14

Anything else the reviewer needs to know?

Since VOGRE-14 has url changes which also affect annotator and annotator factory this PR will require changes once VOGRE-14 merged.

…ngo update

…ngo inlinecss) and added collectstatic

…luding templates

…to fix import for text tor repository

…ates

…ory text import for importing and repository text for details

… import function

Removed obselete code

jdamerow · 2024-11-28T01:57:23Z

annotations/middleware.py

I don't think this is the right approach. It is likely to cause memory issues once there are sufficiently many relationsets. It also seems unnecessary. The only time we care about the readiness of a relation to be submitted is when we want to submit them or maybe when we look at a text. So it really only needs to be checked when the page shows all relations, and then only the relations that are relevant (not all of them).

I don't think this is the right approach. It is likely to cause memory issues once there are sufficiently many relationsets. It also seems unnecessary. The only time we care about the readiness of a relation to be submitted is when we want to submit them or maybe when we look at a text. So it really only needs to be checked when the page shows all relations, and then only the relations that are relevant (not all of them).

If we only show the relations that can be submitted, then the user will not be able to see all relations that they have created on the text. Can we still show the relations that cannot be submitted with the disabled checkbox? I'll change the way we check for ready relations.

jdamerow · 2024-11-28T02:00:35Z

annotations/utils.py

+        }
+    }
+
+


these two functions are specific to the quadriga api, aren't they? They should not be in the annotations util file.

jdamerow · 2024-11-28T02:01:17Z

annotations/views/rest_views.py

+            return Response({'error': 'You are not authorized to submit this RelationSet.'},
+                            status=status.HTTP_403_FORBIDDEN)
+
+        if relationset.status != 'ready_to_submit':


jdamerow · 2024-11-28T02:02:02Z

annotations/views/rest_views.py

+            return Response({'error': 'You are not authorized to submit this RelationSet.'},
+                            status=status.HTTP_403_FORBIDDEN)
+
+        if relationset.status != 'ready_to_submit':


and everywhere else where these strings are used

jdamerow · 2024-11-28T02:02:27Z

concepts/tasks.py

+    #     logger.debug("Resolve concept failed: %s" % str(E))
+    #     return 
+    # logger.debug("Resolved concept %s" % manager.instance.uri)
+    pass


remove commented out code

jdamerow · 2024-11-28T02:02:59Z

vogon/__init__.py

@@ -1,5 +1,4 @@
 from __future__ import absolute_import

 # This will make sure the app is always imported when
-# Django starts so that shared_task will use this app.
-from .celery import app as celery_app
+# Django starts so that shared_task will use this app.


are these comments still valid?

jdamerow · 2024-11-28T02:03:40Z

Resolve conflicts please

…file

… Remove middleware, complete get queryset method

diging-jenkins · 2024-12-11T18:08:17Z

Can one of the admins verify this patch?

jdamerow · 2024-12-12T20:44:56Z

Resolve conflicts please

jdamerow · 2025-01-17T18:34:13Z

.env_app-example

@@ -15,3 +15,5 @@ export BASE_URL="http://localhost:8000/"
 # Make sure APP_ROOT variable ends with a /
 # this is the prefix that vogon will be deployed under (e.g. "vogon-2" for deployment at my-server.org/vogon-2)
 export APP_ROOT=
+
+export QUADRIGA_ENDPOINT=


this needs to have an example (as a comment), e.g http://quadriga.instance/quadriga/...whatever else needs to be here/

jdamerow · 2025-01-17T18:34:30Z

annotations/annotators.py

-from annotations.models import TextCollection, VogonUserDefaultProject, Text
-from urllib.parse import urlparse
-import chardet
+from annotations.models import TextCollection, Text


is this needed here?

jdamerow · 2025-01-17T18:40:46Z

annotations/quadriga.py

+        "metadata": {
+            "type": "appellation_event",
+            "interpretation": concept.uri,
+            "termParts": [


there should be one term part for every piece of annotated text in an appellation (I think currently there can only be one term part as vogon doesn't allow highlighting multiple parts that are not together for one appellation). Position needs to be the position of the first character of the term part.

jdamerow · 2025-01-17T18:41:04Z

annotations/quadriga.py

+            "termParts": [
+                {
+                    "position": 1,
+                    "expression": concept.label or "",


expresssion is the annotated text

jdamerow · 2025-01-17T18:42:53Z

annotations/quadriga.py

+
+        if obj and obj.uri not in nodes:
+            obj_id = get_node_id()
+            nodes[obj_id] = build_concept_node(obj, user, creation_time, source_uri)


what about nested relations, in which subject or object are a relation?

jdamerow · 2025-01-17T18:44:08Z

annotations/views/rest_views.py

+            graph_data = generate_graph_data(relationset, user)
+
+            response = requests.post(endpoint, json=graph_data, headers=headers)
+            response.raise_for_status()


all the quadriga related things should happen in a quadriga file or class

jdamerow · 2025-01-17T18:44:50Z

vogon/__init__.py

@@ -1,5 +1,4 @@
 from __future__ import absolute_import

 # This will make sure the app is always imported when
-# Django starts so that shared_task will use this app.
-from .celery import app as celery_app
+# Django starts so that shared_task will use this app.


Girik1105 and others added 30 commits August 9, 2024 17:59

[VOGRE-2] Changed from github auth to local auth

d6db439

Refined local auth settings

ee7966e

Made auth templates pretty

2d8f1dd

Fixed allauth setting, user bool error view(because of 1.8 to 2.2 dja…

b5ac38b

…ngo update

Fixed local env settings, Changed Email config in settings

0daa957

[VOGRE-2] Removed DS_Store

257d6a2

[VOGRE-2] Added missing stylesheet, fixed module Not Found error (dja…

50002d0

…ngo inlinecss) and added collectstatic

Merge branch 'develop' into story/VOGRE-2

b8fdfbc

[VOGRE-2] DS_Store

2036802

Added Missing stylesheets

9d8c2fe

[VOGRE-2] Added account email required setting

571835a

Merge branch 'develop' into story/VOGRE-2

7a3167d

[VOGRE-3] Removed goat endpoints and replaced it with Conceptpower

cbc0e75

[VOGRE-4] Added conceptpower creds in settings

9c3a18d

[VOGRE-3] Merged develop

7875785

[VOGRE-3] Added variables to env-example, fixed previous merge files

10c20f1

[VOGRE-3] Added import texts to vogon using citesphere

afeaa35

[VOGRE-3] Repository model changes and working details page

dfe83fb

[VOGRE-3] Repository views for collections and collection changed inc…

7a86c66

…luding templates

[VOGRE-3] Working views: repository groups, collections, items, need …

c8f63e3

…to fix import for text tor repository

[VOGRE-3] Workflow working groups -> collections -> items -> repo item

0b00565

[VOGRE-3] Refactored code with correct id names in views, urls, templ…

857a7d4

…ates

[VOGRE-3] Fixed repository search, split repository text into reposit…

4e40f83

…ory text import for importing and repository text for details

[VOGRE-3] Add to project complete

b71ec91

[VOGRE-3] removed debug print statement

e70c80a

[VOGRE-3] Giles util function to pull erxtracted data and use in text…

25539a9

… import function

[VOGRE-3] Giles data successfully being added to texts

c166b18

[VOGRE-3] Individual texts reaching annotation page

886aea6

[VOGRE-3] fixing js errors for annotations to work

140a943

[VOGRE-3] Annotations WORKKKKKK

86a445b

Girik1105 added 4 commits November 20, 2024 16:04

[VOGRE-7]

21de855

Removed obselete code

[VOGRE-7] removed obselete code

14cd882

[VOGRE-7] removed all functions with todo to retire

25cc5b8

[VOGRE-7] Removed obselete file

29ef265

Girik1105 marked this pull request as draft November 21, 2024 16:36

[VOGRE-7] Removed celery instances in concepts app

9ce5df5

jdamerow requested changes Nov 28, 2024

View reviewed changes

jdamerow closed this Nov 28, 2024

Girik1105 added 8 commits December 6, 2024 13:44

[VOGRE-7] Merge develop

6f59886

[VOGRE-7] Used model static variables

fc3873f

[VOGRE-7] shifted quadriga specific functions from utils to quadriga …

250426e

…file

[VOGRE-7] Comments

5f5e972

[VOGRE-7] Removed tasks.py celery file

a5a6370

[VOGRE-7] Updated queryset to check for readiness when called - TODO:…

4f64d6d

… Remove middleware, complete get queryset method

[VOGRE-7] Added tokenize in serializers, tested without middleware

429b67d

[VOGRE-7] Got rid of middleware

4336950

Girik1105 reopened this Dec 11, 2024

[VOGRE-7] Removed commented code

9c2def5

jdamerow closed this Dec 12, 2024

Girik1105 added 3 commits December 13, 2024 07:39

[VOGRE-7] Merged develop

f6db813

[VOGRE-7] Migrations

754be0b

Merge branch 'develop' into story/VOGRE-7

c2f1bf4

Girik1105 reopened this Dec 13, 2024

Girik1105 marked this pull request as ready for review December 13, 2024 15:40

Merge branch 'develop' into story/VOGRE-7

80530ca

jdamerow requested changes Jan 17, 2025

View reviewed changes

jdamerow closed this Jan 17, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

story/VOGRE-7 #15

story/VOGRE-7 #15

Girik1105 commented Nov 5, 2024 •

edited

Loading

jdamerow Nov 28, 2024

Girik1105 Dec 6, 2024

jdamerow Nov 28, 2024

jdamerow Nov 28, 2024

jdamerow Nov 28, 2024

jdamerow Nov 28, 2024

jdamerow Nov 28, 2024

jdamerow Jan 17, 2025

jdamerow commented Nov 28, 2024

diging-jenkins commented Dec 11, 2024

jdamerow commented Dec 12, 2024

jdamerow Jan 17, 2025

jdamerow Jan 17, 2025

jdamerow Jan 17, 2025

jdamerow Jan 17, 2025

jdamerow Jan 17, 2025

jdamerow Jan 17, 2025

jdamerow Jan 17, 2025

story/VOGRE-7 #15

story/VOGRE-7 #15

Conversation

Girik1105 commented Nov 5, 2024 • edited Loading

Guidelines for Pull Requests

Please provide a brief description of your ticket

Are there any other pull requests that this one depends on?

Anything else the reviewer needs to know?

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jdamerow commented Nov 28, 2024

diging-jenkins commented Dec 11, 2024

jdamerow commented Dec 12, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Girik1105 commented Nov 5, 2024 •

edited

Loading