FanFicFare v4.4.0

Installation instructions:

Calibre Plugin:

Install attached plugin zip file, or use Calibre's 'Get plugins' feature.

Command line interface:

FanFicFare is delivered as a pip Python package.

pip install FanFicFare
or
pip install --upgrade FanFicFare
...to update.

(pip will automatically install dependencies.)

Run fanficfare -h from command-line.

Changes:

• Add http_proxy and https_proxy settings. Should be unnecessary, also taking from Calibre.
• Fix for XF2 stories using author img as cover when absolute URL.
• XF: data-url or data-src for lazyload images. Closes #713
• adapter_royalroadcom: Collect numWords. #712
• XenForo2 sites (SB&SV) collect estimatedWords. #712
• Update packaged urllib3 and certifi modules.
• Pre-v5 Calibre fix for nsapa_proxy.
• Record error message for new downloads too since Failed Chapters are now a thing.
• adapter_wwwnovelallcom - Updates for site changes. 'translator' no longer available?
• New Site: worldofx.de (German language X-files eFiction)
• Fixes to nsapa_proxy from nsapa #711
• Add link to wiki/BrowserCacheFeature in Cloudflare error. #708
• Treat img url contains '.svg?' same as endswith('.svg')--Calibre image processing chokes hard on SVG. #709
• Implement use_browser_cache for ficbook.net #704
• Forgot to import traceback in browser cache changes.
• Skip Failures to read cache entries for Chrome Simple Cache and Firefox Cache2 instead of error.
• Fix date reading for XenForo forums sites
• Change defaults.ini to correctly show default values for remove_spoilers, legend_spoilers
• Collect 'fandoms' for adapter_scribblehubcom.
• PI: Only update author link (AKA authorUrl) in Calibre if changed. Only really affects older versions of Calibre. Calibre does the same starting around Sept 2020.

FanFicFare v4.3.0

Installation instructions:

Calibre Plugin:

Install attached plugin zip file, or use Calibre's 'Get plugins' feature.

Command line interface:

FanFicFare is delivered as a pip Python package.

pip install FanFicFare
or
pip install --upgrade FanFicFare
...to update.

(pip will automatically install dependencies.)

Run fanficfare -h from command-line.

Changes:

• quotev.com: use_cloudscraper:true by default.
• Update translations
• adapter_bdsmlibrarycom: Set author Anonymous when author not found instead of sys.exit(). Closes #696
• Fix XF authorUrl and author_avatar_cover feature. Closes #695
• Catch exceptions in word count.
• Add CLI --color option for warns and fails. Closes #692
• Report browser cache load fail as such.

FanFicFare v4.2.0

Installation instructions:

Calibre Plugin:

Install attached plugin zip file, or use Calibre's 'Get plugins' feature.

Command line interface:

FanFicFare is delivered as a pip Python package.

pip install FanFicFare
or
pip install --upgrade FanFicFare
...to update.

(pip will automatically install dependencies.)

Run fanficfare -h from command-line.

Changes:

• Add New Site: novelfull.com (#688 & #689) - Thanks, AlexRiina!
• Need to set logger.setLevel() again with import changes to see debugs in plugin.
• import changes for arch linux system plugins - As submitted by eli-schwartz
• Remove site: fictionpad.com, completely different site, looks like generic parked?
• Remove site: www.thundercatsfans.org, no longer efiction, now static pages and PDFs
• Remove site: www.thepetulantpoetess.com, no longer efiction, URLs like OTW but different
• Remove site: www.potterfics.com, "Potterfics.com has closed its doors forever"
• Remove site: fanfic.castletv.net, DNS there, no server, last successful 2018-10-21
• Remove site: deandamage.com, site there, efiction broken, last successful 2018-12-04
• Remove site: www.deepinmysoul.net, moved to deepinmysoul.nl and changed software--not eFiction anymore, <100 stories, all old
• Include code for nsapa_proxy for dealing with Captchas (by nsapa -- coming soon!)
• Some other import cleanup.
• Updates from cloudscraper 1.2.58
• Use newer Calibre image processing, add JPG quality setting - thanks, Malloc Voidstar
• Add use_cloudscraper:true under [www.ficbook.net] to defaults.ini
• Fix for adapter_literotica story URL oddities. Again.
• Fix for site change: adapter_fictionmaniatv
• Get storyId for sugarquillnet and siyecouk from the parsed query string. From mcepl
• Fix for adapter_literotica changing URLs on author page yet again.
• Add [harrypotterfanfiction.com] slow_down_sleep_time:1 -- Site blocking fast downloads.

FanFicFare v4.1.0

Installation instructions:

Calibre Plugin:

Install attached plugin zip file, or use Calibre's 'Get plugins' feature.

Command line interface:

FanFicFare is delivered as a pip Python package.

pip install FanFicFare
or
pip install --upgrade FanFicFare
...to update.

(pip will automatically install dependencies.)

Run fanficfare -h from command-line.

Changes:

• adapter_literotica - Update for recent site change and fix first chapter - Thanks, davidfor!
• adapter_fictionlive - Fix off-by-one error in most_recent_chunk / add_chapter_url interaction, closes #672 Thanks, HazelSh!
• Fixes for literotica sites changes. Issue #671
• Fix for include_dice_rolls when multiple fieldsets.
• Check for img 'failedtoload' before trying to fetch on updates.
• Issue with fiction.live setting in defaults[fiction.live] overriding personal[www.fiction.live]. Could use a more general solution if I can think of one.
• minor changes to track fictionlive website updates - Thanks, HazelSh #668
• Fix show_timestamps option in adapter_fictionlive
• Add include_dice_rolls option
• Include error for continue_on_chapter_error in log
• Put 'Change theme to Classic' back in adapter_storiesonlinenet
• Remove some dup imports/code, thanks akshgpt7. Closes #663
• use_ssl_unverified_context:true ignored when use_clouadscraper:true
• Fixes for ancient 'import *' getting broken by removing unused imports in base_writer -- Fixes "name 're' is not defined" with HTML output.

FanFicFare v4.0.2

Bug Fix Release

Installation instructions:

Calibre Plugin:

Install attached plugin zip file, or use Calibre's 'Get plugins' feature.

Command line interface:

FanFicFare is delivered as a pip Python package.

pip install FanFicFare
or
pip install --upgrade FanFicFare
...to update.

(pip will automatically install dependencies.)

Run fanficfare -h from command-line.

Changes:

• Fix for BG job race conditions.
• Fix writer_txt import removeAllEntities
• Update plugin about.html
• Fix reduce_zalgo not imported.
  (See v4.0.0 also)

FanFicFare v4.0.0

Installation instructions:

Calibre Plugin:

Install attached plugin zip file, or use Calibre's 'Get plugins' feature.

Command line interface:

FanFicFare is delivered as a pip Python package.

pip install FanFicFare
or
pip install --upgrade FanFicFare
...to update.

(pip will automatically install dependencies.)

Run fanficfare -h from command-line.

Changes:

• Major refactoring/rewrite of network fetching and caching layer using requests library.
• use_browser_cache feature added -- Read fanfiction.net stories from your Chrome or FireFox browser cache in addition / instead of the network. Many thanks to Sidney Markowitz for the stellar help with this. See BrowserCacheFeature Wiki Page.
• Add additional_images setting for html & epub formats. Close #648
• Fix for 'Updating' (anonymous?) author in adapter_wuxiaworldsite. For #657
• Allow tweak_fg_sleep etc with any site, add defaults settings for basexenforo, fictionalley and fictionpress.
• Drag/drop of 'emails'--look for story URL in Content-Base header for Thunderbird RSS 'emails'.
• Apply 'email' proc flag when processing dragged .eml files--which are emails.
• Update translations.
• Fix adapter_webnovelcom for some site changes. Closes #644
• Allow tweak_fg_sleep etc with any site, add defaults settings for basexenforo, fictionalley and fictionpress.
• fanficauthors.net doesn't use login anymore. Closes #651
• BG % done by stories AND chapters.
• Restore multi-process background jobs as one process per site.
• Don't count continue_on_chapter_error chapters when checking vs 'new chapters'.
• More properly get msg payload fromemail drag and drop. Closes #645
• Give adapter_archiveofourownorg a get_section_url() for quicker rejects checks from email.
• Convert adapter_spikeluvercom to base_efiction after site changes.
• Remove 'extrasleep' feature--it was ugly--increase slow_down_sleep_time for the couple sites that used it. We can trust the users, right?
• Remove httplib._MAXHEADERS workaround for royalroad.com - not happening anymore.
• Fixes for py2/py3 & a stray INI space.
• Fix for Reject List to equate story URLs with and without titles, for XenForo and ffnet. Otherwise you can end up with dups in Reject List and not recognize new URLs are already being rejected. Also fixes [storyUrl] sections without titles for ffnet.
• Fix fictionpress.net and retain Reject List entries for removed sites.
• Get cloudscraper 1.2.56 changes, but keep py2 compat.
• Fix for chapter error report. Closes #641
• adapter_fanfictionnet: Start keeping story title part of storyUrl.

FanFicFare v3.29.0

fanfiction.net re-enabled

This version re-enables fanfiction.net support.

Installation instructions:

Calibre Plugin:

Install attached plugin zip file, or use Calibre's 'Get plugins' feature.

Command line interface:

FanFicFare is delivered as a pip Python package.

pip install FanFicFare
or
pip install --upgrade FanFicFare
...to update.

(pip will automatically install dependencies.)

Run fanficfare -h from command-line.

Changes:

• Add better chapter error reporting, refactor proceed_question code.
• Add plugin options to mark success/failed/chapter error individually.
• Clear metadata cache after adapter metadata fetch. Cached metadata values may not be replace_metadata processed if fetched before their conditional dependencies. Revealed by AO3 one-shots using title for chapter name.
• Re-enable ffnet and bump up sleep times for same.
• ffnet fpcom, continue_on_chapter_error:true by default, increase sleep times.
• First rough version of reporting continue_on_chapter_error chapters.
• Report chapter_error in custom error column and marked (when configured).
• Update to cloudscraper v1.2.52
• Increase times between retries on fetch error.
• Tweek ffnet/fpcom sleep times again.
• slow_down_sleep_time: randomize between 0.5 time and 1.5 time.
• Report cloudscraper exceptions as such, plus hide the potentially misleading 'opensource' part.
• adapter_fanficsme: do an extra fetch before login for cookie(?) Closes #633
• adapter_webnovelcom: Fixes for site changes. Closes #629
• quotev.com: site change in date parse, use utf8:ignore as first encoding choice. Closes #625

FanFicFare v3.28.0

fanfiction.net disabled

This version disables fanfiction.net support. The site is blocking our traffic and working around it has become prohibitively difficult.

We can hope that the situation will improve and fanfiction.net support will be returned, but for now, this version gives a more accurate error message when it fails and stops throwing extra traffic at the fanfiction.net servers.

Installation instructions:

Calibre Plugin:

Install attached plugin zip file, or use Calibre's 'Get plugins' feature.

Command line interface:

FanFicFare is delivered as a pip Python package.

pip install FanFicFare
or
pip install --upgrade FanFicFare
...to update.

(pip will automatically install dependencies.)

Run fanficfare -h from command-line.

Changes:

• Disable adapter_fanfictionnet with warning about site blocking.
• Catch exception from emails not decoding, skip & logger.error().
• Add a fake get_image_size() method for when no image processing available. Closes #621 (CLI only)
• Change adapter_twilightednet to https
• Change for adapter_fanfictionnetadapter_fanfictionnet to make skip_author_cover work again.
• Make included certifi and requests use same tmp file code and store under calibre tmp dir for cleanup.
• Add append_datepublished_to_storyurl option for storiesonline.net, finestories.com, scifistories.com only.

FanFicFare v3.27.0

Installation instructions:

Calibre Plugin:

Install attached plugin zip file, or use Calibre's 'Get plugins' feature.

Command line interface:

FanFicFare is delivered as a pip Python package.

pip install FanFicFare
or
pip install --upgrade FanFicFare
...to update.

(pip will automatically install dependencies.)

Run fanficfare -h from command-line.

Changes:

• Add use_cloudscraper code to use with CloudFlare sites, like fanfiction.net now does.
• Changes to allow email chapter update URLs to work in adapter_wattpadcom
• Updates for many included dependencies

This version includes the module cloudscraper and dependencies and is configured to use it for fanfiction.net and fictionpress.com.

This probably won't help with ffnet in older Calibre versions due to CloudFlare detecting that older versions of OpenSSL bundled in them. If an older version works for you, great, but don't expect it to.

FFF does still work in older Calibre versions for other sites.

I have tested both CLI in Python3 and plugin in Calibre v5.7.2 on Win10, Mac(BigSur) and Linux.

FanFicFare v3.26.0

Installation instructions:

Calibre Plugin:

Install attached plugin zip file, or use Calibre's 'Get plugins' feature.

Command line interface:

FanFicFare is delivered as a pip Python package.

pip install FanFicFare
or
pip install --upgrade FanFicFare
...to update.

(pip will automatically install dependencies.)

Run fanficfare -h from command-line.

Changes:

• Use refresh_screen=False when calling Reading List add/remove, refresh book_ids.
• Clear user_agent for literotica.com
• Fix finding imgs in existing epubs. Closes #608 Many thanks to Epicpkmn11 for finding this before I released.
• Fixes for adapter_fictionlive, thanks HazelSh
• Make default CSS for fiction.live and royalroad.com use em instead of pt, thanks HazelSh
• Fix for genre change in adapter_royalroadcom
• Fix for adapter_storiesonlinenet 'access' issue.
• Don't error on w/o class in replace_failed_smilies_with_alt_text
• Allow tab to leave edit boxes.
• Revert(ish) 'fix' for ffnet covers--they fixed it.
• Change convert_inline_images default to false.
• Fix http/https matching in identifiers:url search.
• Allow tags without src attr in epub to update
• Fix for ffnet cover images changed? Or broken?
• Warn, not crash on when cover_min_size fails.
• Add 2nd dup-story URL check after fetching metadata for when story URL changes.
• Fix adapter_thesietchcom for site change
• Add (optional, default on) convert support for data:image in-line s.
• Adding pass-through support for data:image in-line s.
• XenForo: Fixes for extra /s in normalized URLs.
• Set default cover_min_size: 10,10 to avoid spacer images as covers.
• New Site: squidgeworld.org - shares code with AO3.
• adapter_wuxiaworldco: added status #592 from Rikkitp
• Don't use polyglot, not included until Calibre3. Remove some debug output.
• adapter_scribblehubcom: Accept chapter URLs.
• adapter_wwwnovelallcom: Accept chapter URLs, must change to true storyId/URL after.
• Make adapter_novelupdatescc share code with adapter_wuxiaworldco
• adapter_literotica: Keep language domains & use for language metadata. #588
• adapter_alternatehistorycom now uses same thread group HTML as XenForo2. Closes #590
• adapter_wuxiaworldco/adapter_novelupdatescc: Re-add Don't include grayed out 'In preparation' chapters