-
Notifications
You must be signed in to change notification settings - Fork 2
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Updates fcc crawl for new fcc site #126
base: master
Are you sure you want to change the base?
Conversation
@matthewdking Thanks again for your amazing effort taking this on! Bit concerned about the performance hit of waiting for the entire freecodecamp profile to load all it's resources before we process the page but I don't have a great alternative. So for now I'm happy to go with Puppeteer. Hopefully one day freecodecamp will sort out a public api and someone will update prereq-check to use it... |
@matthewdking I've only been able to do minimal testing, but the code looks good - as soon as the tests are sorted I'm happy to approve the merge. |
2bf2c7d
to
a4c0625
Compare
Codecov Report
@@ Coverage Diff @@
## master #126 +/- ##
==========================================
- Coverage 93.47% 91.59% -1.89%
==========================================
Files 18 18
Lines 322 333 +11
==========================================
+ Hits 301 305 +4
- Misses 21 28 +7
Continue to review full report at Codecov.
|
a4c0625
to
f0f2344
Compare
fixes #125
Included in this PR:
request-promise
in the crawl of FCC with puppeteer headless browser. The reason for this was down to the data we needed not being accessible on the first call to the site. If anyone knows a neater way to do it open to suggestions 😄Will update tests for this in the next few days.
Be good to get this in before FAC15 application deadline (August 17th)