Skip to content

Commit

Permalink
Escape double quotes in fulltext to avoid issues with JSON format
Browse files Browse the repository at this point in the history
  • Loading branch information
valearna committed Jan 12, 2024
1 parent 94762b0 commit f8fffdd
Showing 1 changed file with 1 addition and 0 deletions.
1 change: 1 addition & 0 deletions src/backend/api/endpoints/curator_dashboard.py
Original file line number Diff line number Diff line change
Expand Up @@ -171,6 +171,7 @@ def get_text_from_pdfs(self, paper_id):
fulltext = fulltext.replace('-\n', '')
fulltext = fulltext.replace('.\n', '. ')
fulltext = fulltext.replace('\n', ' ')
fulltext = fulltext.replace('"', '\"')
sentences = sent_tokenize(fulltext)
sentences = [sent for sent in sentences if np.average([len(w) for w in sent.split(' ')]) > 2]
fulltext = fulltext.replace('\n', ' ')
Expand Down

0 comments on commit f8fffdd

Please sign in to comment.