Skip to content

Crawl sync api fix

Compare
Choose a tag to compare
@j-mendez j-mendez released this 21 Apr 17:05
1efc75e

This release brings fixing the thread handling of async task with the client that is established on the main thread.
Crawl speed is improved drastically due to the incorrect handling between the client pool releasing between threads.

What's Changed

  • chore(log): add log util by @j-mendez in #31
  • feat(regex): add optional regex black listing by @j-mendez in #36
  • perf(crawl): improve crawl link exclusion by @j-mendez in #37
  • perf(parsing): add async parallel page handling by @j-mendez in #38
  • perf(client): fix blocking and async mixture by @j-mendez in #39

Full Changelog: v1.5.1...v1.6.0