Skip to content

Commit

Permalink
[FIX] tools: classify \uFEFF as whitespace for translations
Browse files Browse the repository at this point in the history
Versions
--------
- 15.0
- 16.0
- 17.0
- saas-17.1
- saas-17.2

Issue
-----
Commit 9426ee5 introduced the \uFEFF character to web_editor as a
zero-width non-breaking whitespace. When this gets added to a HTML node,
and processed for translation, it throws an "empty document" error.

Cause
-----
When passed to the `get_text_content` function, the call to
`html.fromstring('\uFEFF').text_content()` throws an error.
\uFEFF is not technically classified as whitespace, so the `nonspace`
function which attempts to prevent processing empty documents doesn't
catch it.

Solution
--------
Instead of the `isspace` method, use a regex which matches on all
whitespace as well as \uFEFF.

To be applied on stable versions while the origins of stray ZWNBSPs
get tackled on master.

opw-3957259

closes odoo#171578

X-original-commit: 5e70361
Signed-off-by: Raphael Collet <[email protected]>
Signed-off-by: Levi Siuzdak <[email protected]>
  • Loading branch information
lvsz committed Jul 2, 2024
1 parent 1160a4f commit 4be86b2
Showing 1 changed file with 2 additions and 1 deletion.
3 changes: 2 additions & 1 deletion odoo/tools/translate.py
Original file line number Diff line number Diff line change
Expand Up @@ -176,6 +176,7 @@ def translate_attrib_value(node):
)

avoid_pattern = re.compile(r"\s*<!DOCTYPE", re.IGNORECASE | re.MULTILINE | re.UNICODE)
space_pattern = re.compile(r"[\s\uFEFF]*") # web_editor uses \uFEFF as ZWNBSP


def translate_xml_node(node, callback, parse, serialize):
Expand All @@ -189,7 +190,7 @@ def translate_xml_node(node, callback, parse, serialize):

def nonspace(text):
""" Return whether ``text`` is a string with non-space characters. """
return bool(text) and not text.isspace()
return bool(text) and not space_pattern.fullmatch(text)

def translatable(node):
""" Return whether the given node can be translated as a whole. """
Expand Down

0 comments on commit 4be86b2

Please sign in to comment.