Encoded characters not uploaded correctly

[simonrihm]

When using some special characters in properties of type string and uploading the data to the knowledge graph, the literals will not reflect those characters correctly. For example, a species with rdfs:label "Cu(NO3)2·2.5H2O" will be uploaded as Cu(NO3)2·2.5H2O (note the additional  character).

This seems to be an encoding issue as it is the case only for characters that are encoded differently between UTF-8 and Windows-1252, see https://www.w3schools.com/tags/ref_urlencode.ASP
I assume it happens when the rdflib.Graph is serialized to build an update query here:

TheWorldAvatar/JPS_BASE_LIB/python_wrapper/twa/kg_operations/sparql_client.py

Line 230 in 13f303c

update = f"""DELETE {{ {g_to_delete.serialize(format='nt')} }}

The serialized string is then executed as query here:

TheWorldAvatar/JPS_BASE_LIB/python_wrapper/twa/kg_operations/sparql_client.py

Line 128 in 13f303c

self.kg_client.executeUpdate(update)

Here is a minimal example to reproduce this issue:

from rdflib import Graph, Literal, URIRef
from rdflib.namespace import RDFS
from twa.kg_operations import PySparqlClient

g=Graph()
g.add((URIRef('http://www.theworldavatar.com/test'), RDFS.label, Literal("X·Y’")))
update = f"""INSERT DATA {{ {g.serialize(format='nt')} }}"""
print(update)

kg_client = PySparqlClient(
    "http://00.000.000.00:0000/blazegraph/namespace/test/sparql",
    "http://00.000.000.00:0000/blazegraph/namespace/test/sparql"
)
kg_client.perform_update(update)

[jb2197]

As discussed, encoding-decoding special characters when interacting with blazegraph using twa (previously py4jps) is a known issue that requires investigation on how py4j handles encoding-decoding, see #667

Given the time constraint of various projects, a workaround could be used for the time being (assuming you queried literal from the blazegraph and saved it to python str object your_string):

your_string.encode('ISO-8859-1').decode('utf-8')

In this specific case, it will be "X·Yâ\x80\x99".encode('ISO-8859-1').decode('utf-8') that returns "X·Y’"