Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Move Firecrawl from v0 to v1 API #12571

Closed
3 of 5 tasks
nickscamara opened this issue Jan 9, 2025 · 2 comments · Fixed by #12574
Closed
3 of 5 tasks

Move Firecrawl from v0 to v1 API #12571

nickscamara opened this issue Jan 9, 2025 · 2 comments · Fixed by #12574
Labels
💪 enhancement New feature or request

Comments

@nickscamara
Copy link
Contributor

nickscamara commented Jan 9, 2025 •
edited
Loading

Self Checks

  • I have searched for existing issues search for existing issues, including closed ones.
  • I confirm that I am using English to submit this report (我已阅读并同意 Language Policy).
  • [FOR CHINESE USERS] 请务必使用英文提交 Issue，否则会被关闭。谢谢！:）
  • Please do not modify this template :) and fill in all the required fields.

1. Is this request related to a challenge you're experiencing? Tell me about your story.

Currently Dify uses v0 sdk for Firecrawl, while v1 is the most up-to-date and is a lot more stable and performant - causing a lot less bugs.

Suggested by @mogery.

@ftonato will take care of the migration and will be opening a pr later this week.

2. Additional context or comments

No response

3. Can you help us with this feature?

  • I am interested in contributing to this feature.
@nickscamara
Copy link
Contributor Author

Related issue on #8324 but don't think it got updated as far as I can tell.

@dosubot dosubot bot added the 💪 enhancement New feature or request label Jan 9, 2025
@mogery
Copy link

mogery commented Jan 9, 2025

The PR on that issue updated a different code path I think.

ftonato pushed a commit to ftonato/dify that referenced this issue Jan 9, 2025
refactor: Update Firecrawl to use v1 API
7a29db2 
- Updated API endpoints from v0 to v1 for scrape and crawl methods.
- Enhanced error handling in `scrape_url` and `check_crawl_status` methods.
- Introduced helper methods _extract_common_fields and _format_crawl_status_response for better code organization and readability.

Closes: langgenius#12571
@ftonato ftonato mentioned this issue Jan 9, 2025
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
💪 enhancement New feature or request
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

refactor: Update Firecrawl to use v1 API ftonato/dify
2 participants
@nickscamara @mogery