Skip to content

Commit

Permalink
refactor: improve fetch_url_via_curl
Browse files Browse the repository at this point in the history
  • Loading branch information
sigoden committed Jul 17, 2024
1 parent 2ee2473 commit d449cb8
Showing 1 changed file with 2 additions and 2 deletions.
4 changes: 2 additions & 2 deletions tools/fetch_url_via_curl.sh
Original file line number Diff line number Diff line change
Expand Up @@ -8,8 +8,8 @@ set -e
main() {
# span and div tags are dropped from the HTML https://pandoc.org/MANUAL.html#raw-htmltex and sed removes any inline SVG images in image tags from the Markdown content.
curl -fsSL "$argc_url" | \
pandoc -f html-native_divs-native_spans -t gfm-raw_html | \
sed -E 's/!\[.*?\]\((data:image\/svg\+xml[^)]+)\)//g' \
pandoc -f html-native_divs-native_spans -t gfm-raw_html --wrap=none | \
sed -E 's/!\[[^]]*\]\([^)]*\)//g' \
>> "$LLM_OUTPUT"
}

Expand Down

0 comments on commit d449cb8

Please sign in to comment.