Skip to content

edelkas/wikiascrap

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

12 Commits
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Scraps a Wikia (now Fandom) site into an XML file using the Special:Export
feature on every page. The full list of pages can be found on Special:AllPages.

OPTIONS:

Options can be configured by editing the variables at the start of the code.

* PAGES: This will enable or disable the download of articles.
* FULL: This will enable or disable the download of all revisions (not just
  the current version of each article), and additionally, it will fetch pages
  from all namespaces other than Main: Talk pages, User pages, etc.
* FILES: This will enable or disable the download of all files, which will
  be exported to a matching folder.

NOTE: Due to recent Wikia changes, downloading files requires authentication,
      since Wikia redirects anonymous users from "File:" pages to the first
      normal page the file is used. Therefore, to download files, you need to
      fill in the TOKEN variable at the start of the code with a valid token.

      This token is the value of the cookie named "access_token" from the
      domain "fandom.com". To obtain it, simply authenticate in a
      Fandom account manually, and then use the cookie browser of your
      browser to find it. On some browsers, like Firefox, you may need
      to open developer tools, like the Storage Inspector.

About

Export a Wikia (now Fandom) page to XML

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages