Incorporation of dataset descriptions

[pierrepo]

Both zenodo et figshare offer a description field for each dataset. This field may or may not be filled by users but could contain relevant information regarding the content of the dataset.

This issue lists and discusses prerequisites regarding the incorporation of dataset descriptions within the collected data.

• Convert \n characters to space. This will convert a structured paragraph into a single line.
• Remove \t characters to space. This will avoid conflict with the TSV output format, where tabulation is used as a separator.
• Clean HTML tags. Descriptions in Zenodo are formatted in HTML rich-text format. We should remove all these tags.

[pierrepo]

We could use the BeautifulSoup Python library to remove HTML tags. See:

• https://www.geeksforgeeks.org/remove-all-style-scripts-and-html-tags-using-beautifulsoup/?ref=lbp
• https://stackoverflow.com/questions/16206380/python-beautifulsoup-how-to-remove-all-tags-from-an-element