diff --git a/.github/workflows/ci.yaml b/.github/workflows/ci.yaml
new file mode 100644
index 00000000..bf1e63d5
--- /dev/null
+++ b/.github/workflows/ci.yaml
@@ -0,0 +1,30 @@
+name: ci
+on:
+ push:
+ branches:
+ - master
+ - main
+ - mkdocs
+permissions:
+ contents: write
+jobs:
+ deploy:
+ runs-on: ubuntu-latest
+ steps:
+ - uses: actions/checkout@v4
+ - name: Configure Git Credentials
+ run: |
+ git config user.name github-actions[bot]
+ git config user.email 41898282+github-actions[bot]@users.noreply.github.com
+ - uses: actions/setup-python@v4
+ with:
+ python-version: 3.x
+ - run: echo "cache_id=$(date --utc '+%V')" >> $GITHUB_ENV
+ - uses: actions/cache@v3
+ with:
+ key: mkdocs-material-${{ env.cache_id }}
+ path: .cache
+ restore-keys: |
+ mkdocs-material-
+ - run: pip install "mkdocs-material[imaging]"
+ - run: mkdocs gh-deploy --force
diff --git a/.gitignore b/.gitignore
index 0d7d8ac2..620f6162 100644
--- a/.gitignore
+++ b/.gitignore
@@ -7,4 +7,5 @@ keys.json
test/tmp/
*.lock
.nyc_output/
-coverage
\ No newline at end of file
+coverage
+.cache
\ No newline at end of file
diff --git a/CNAME b/CNAME
new file mode 100644
index 00000000..58d0fbd2
--- /dev/null
+++ b/CNAME
@@ -0,0 +1 @@
+papermemory.org
\ No newline at end of file
diff --git a/Readme.md b/Readme.md
index bb0cc7ab..f3ee8df5 100644
--- a/Readme.md
+++ b/Readme.md
@@ -45,6 +45,10 @@ This browser extension allows you to automatically store research papers you rea
+Share ideas 💡 in [issues](https://github.com/vict0rsch/PaperMemory/issues) and love with [stars](https://github.com/vict0rsch/PaperMemory/stargazers) ⭐️:)
+
+
+
@@ -92,18 +96,14 @@ This browser extension allows you to automatically store research papers you rea
[About finding published papers from preprints](#preprints)
-### Privacy
+## Privacy
All the data collected is stored locally on your computer and you can export it to `json` it from the options page.
-### Feature requests
+## Feature requests
I'm regularly adding feature ideas in the [issues](https://github.com/vict0rsch/PaperMemory/issues). Feel free to go upvote the ones you'd like to see happen or submit your own requests.
-### Duplicates
-
-As of version `0.5.3`, PaperMemory finds and merges duplicates based on their titles only. If you visit a paper which has the same title as an existing paper, it will not be added as a new paper, rather "attached" to the existing one. However, if the existing paper does not have a known publication venue and the new paper does, then they are switched to favour the one with a known publication venue.
-
## Customization
Customize features in the menu:
@@ -121,60 +121,11 @@ Switch between Light and Dark mode
----
-Share ideas 💡 in [issues](https://github.com/vict0rsch/PaperMemory/issues) and love with [stars](https://github.com/vict0rsch/PaperMemory/stargazers) ⭐️:)
+## Keyboard Shortcuts
+
-## Keyboard Navigation
-
-* **Keyboard** navigation:
- * Open the popup:
- * `cmd + shift + e` on Mac
- * `ctrl + shift + m` on Linux & Windows
- * See the extension's `User Guide` for customization
- * Open the Memory
- * `a` from the popup's home will open it
- * navigate to the bottom left button with `tab` and hit `enter`
- * Search
- * Search field is automatically focused on memory open
- * Navigate to the top input with `tab` or `shift + tab`
- * Navigate papers
- * `tab` will iterate through papers down the list
- * `shift + tab` will go back up the list
- * Edit a paper
- * Press `e` to _edit_ the paper's metadata: tags, code and note when the paper is *focused* (from click or keyboard `tab` navigation)
- * Navigate through fields with `(shift+) tab`: tags and note if you're on a paper's page.
- * Press `enter` to _open_ a focused paper (*focus* an existing tab with the paper or *create a new tab* to the paper's pdf if it's not open already)
- * `backspace` to _delete_ a focused paper (a confirmation will be prompted first don't worry 👮♀️)
- * Close Memory or Menu
- * `esc` closes the memory (or the menu -- **not** in Firefox)
-* **Search**
- * In a paper's authors, title and note.
- * Split queries on spaces: `gan im` will look for: _all papers whose (title OR author) contain ("gan" AND "im")_
- * In a paper's code link
- * Start the search query with `c:` to only search code links
- * Paper years
- * Start the search query with `y: ${year}`, optionally with `,` separating requested years or starting with `>` or `<` to filter paper published after/before a given year (stricly)
- * `y: 20,21,22` will display papers published in `2020 OR 2021 OR 2022`
- * `y: <2015` will display papers published before (strictly) `2015`
- * `y: >19` will display papers published after (strictly) `2019`
- * Paper tags
- * Start the search query with `t:` to filter by tags
- * `t: gan` will look for _all papers whose tag-list contains at least 1 tag containing "gan"_
- * `t: gan tim` will look for _all papers whose tag-list contains (at least 1 tag containing "gan") AND (at least 1 tag containing "tim")_
-* **Export** your memory as json file (in the extension's Menu)
-
-### Keyboard Shortcuts
-
-By default, and _if_ they are not already attributed, the following keyboard shortcuts are available:
-
-* `cmd + shift + e` will open PaperMemory's popup on a Mac, `ctrl + shift + m` on Linux & Windows.
-* `cmd/ctrl + shift + s` will try to download the pdf of the current paper. If you have enabled this feature in the menu, the paper will be downloaded in your PaperMemoryStore (see thee advanced options page for more info)
-* `cmd/ctrl + shift + l` will trigger the manual parsing of a paper if you have disabled automatic paper detection in the menu
-
-All those shortcuts can be changed/set from `chrome://extensions/shortcuts` (Chrome, Brave, Edge) or `about:addons` (Firefox).
-
-## Guides
+## User Guides
@@ -198,211 +149,24 @@ In the extension's `options` (right click on the icon or in the popup's menu) yo
-## Install from source (Brave & Chrome) - -* Download/clone the repo - * If you want to try a feature that's not currently merged to `master` you need to first change the repository's branch you want to try -* Go to Chrome/Brave's extension manager -* Enable developer mode -* Click on "load unpackaged extension" -* Select the downloaded folder :) +## Install from source ## Preprints There currently exists, to my knowledge, no centralized source for matching a preprint to its subsequent published article. This makes it really hard to try and implement best practices in terms of citing published papers rather than their preprint. -My approach with PaperMemory is to try and notify you that a publication likely exists by utilizing the `note` field. You will occasionally notice `Accepted @ X` in a Paper's notes. This will be added automatically if you are on a known published venue's website (as Nature, PMLR or NeurIPS) but also from: -* [PapersWithCode.com](https://paperswithcode.com) - * A query is sent to their [api](https://paperswithcode.com/api/v1/docs/) from an Arxiv ID or a paper's plain text title if it's not an Arxiv paper - * As PaperMemory retrieves code, it also looks for a `proceeding` field in PWC's response. - * If it exists and is not `null` then it is expected to look like `${conf}-${year}-${month}`. - * In this case a note is added to the paper: `Accepted @ ${conf} ${year} -- [paperswithcode.com]` -* [dblp.org](https://dblp.org) - * A query is sent to their [api](https://dblp.org/faq/How+to+use+the+dblp+search+API.html) for an exact paper title match - * The oldest `hit` in the response which is not a preprint (`hit.venue !== "CoRR"`) is used - * If such a match is found, a note is added as: `Accepted @ ${venue} ${year} -- [dblp.org]` - * In this case, **the original Arxiv bibtex data is overwritten to use DBLP's** - * Try for instance [Domain-Adversarial Training of Neural Networks](http://arxiv.org/pdf/1505.07818v4) - * Note that DBLP journals may use ISO4 abbreviations -* [SemanticScholar.org](https://www.semanticscholar.org/) - * A query is sent to their [api](https://www.semanticscholar.org/product/api) for an exact paper title match - * Up to 50 relevant papers are returned in `response.data` as an `Array` - * In case of a match, the venue should not be `"ArXiv"` - * If there's a match and its venue is not Arxiv then `match.venue` and `match.year` are used to create a note: `Accepted @ {venue} ({year}) -- [semanticscholar.org]` -* [CrossRef.org](https://crossref.org) - * A query is sent to their [api](https://api.crossref.org/swagger-ui/index.html) for an exact paper title match - * The response *must* contain an `event` field with a `name` attribute. If it does not it'll be ignored. - * If it does, a note is added as: `Accepted @ ${items.event.name} -- [crossref.org]` - * Try for instance [Attention-Guided Generative Adversarial Networks for Unsupervised Image-to-Image Translation](http://arxiv.org/pdf/1903.12296v3) - -There's room for improvement here^, please contact me (an issue will do) if you want to help - -## Discovering Code repositories - -PaperMemory uses the PapersWithCode API in order to discover code repositories. If the paper being added to the Memory is from Arxiv, PaperMemory will use PWC's `arxiv_id` search field. Otherwise it will query per title. PaperMemory then expects exactly `1` result from the API. Any different `count` in the response will make PaperMemory consider there is no match. - -If a match is found, the selected repo is the official (if it exists) one with the most stars (customizable in the Advanced Options). - -Here's an example return value from PWC's API - -```json -"https://paperswithcode.com/api/v1/papers/?title=climategan" - -{ - "count": 1, - "next": null, - "previous": null, - "results": [ - { - "id": "climategan-raising-climate-change-awareness", - "arxiv_id": "2110.02871", - "nips_id": null, - "url_abs": "https://arxiv.org/abs/2110.02871v1", - "url_pdf": "https://arxiv.org/pdf/2110.02871v1.pdf", - "title": "ClimateGAN: Raising Climate Change Awareness by Generating Images of Floods", - "abstract": "...", - "authors": ["..."], - "published": "2021-10-06", - "conference": "climategan-raising-climate-change-awareness-1", - "conference_url_abs": "https://openreview.net/forum?id=EZNOb_uNpJk", - "conference_url_pdf": "https://openreview.net/pdf?id=EZNOb_uNpJk", - "proceeding": "iclr-2022-4" - } - ] -} - -"https://paperswithcode.com/api/v1/papers/climategan-raising-climate-change-awareness/repositories/" - -{ - "count": 2, - "next": null, - "previous": null, - "results": [ - { - "url": "https://github.com/cc-ai/climategan", - "owner": "cc-ai", - "name": "climategan", - "description": "Code and pre-trained model for the algorithm generating visualisations of 3 climate change related events: floods, wildfires and smog. ", - "stars": 25, - "framework": "pytorch", - "is_official": true - }, - { - "url": "https://github.com/cc-ai/mila-simulated-floods", - "owner": "cc-ai", - "name": "mila-simulated-floods", - "description": "", - "stars": 7, - "framework": "pytorch", - "is_official": true - } - ] -} - -``` - -## Ad-Hoc PDFs - -If you want to associate a pdf to a paper in your PaperMemoryStore that is not downloaded from PaperMemory (for instance, it is behind a paywall), you can **download it to `~/Downloads/PaperMemoryStore/`** (maybe you're reading the paper on Sci-Hub you thief 🙃) provided the file's name contains the **paper's title** (not counting non-alphanumeric characters and case-insensitive). - -Because of limitations Browser Extensions face, a file **must** be _downloaded from the Browser to the folder_ in order to be detected. This means that if someone sends you a pdf over Slack or if you move a file to the PaperMemoryStore folder for instance, you won't be able to match it to a PaperMemory item. - -## FAQ - -.json
or .bib
bibliography, or only export papers with a given tag. More details in #89
-<0.5.3
(May 2022) you may have duplicate papers in your Memory. The easiest way to resolve this is to go to a duplicate's page, then delete it from memory and refresh the page. The automatic de-duplication features in version 0.5.3
should handle the matching if the 2 papers have the same title.>=0.5.3
and the 2 papers have the same title, this may be a problem with PaperMemory and you should open an issue here on Github.+![Popup](assets/popup.png){ align=left .popup-display } +![Popup](assets/menu.png){ align=right .popup-display } +
+ + + +The following :fontawesome-solid-toggle-on: items are displayed in PaperMemory's :octicons-gear-16: Popup Menu and can be changed anytime (close and re-open popup to see changes). + +--- + +### Arxiv enhancements + +`Add PDF direct download button` + +: Add the blue animated button to download the PDF to your Downloads without opening it. + +`Add Markdown Link section` + +: In the right-most column on ArXiv.org, add the copy-able box with a :material-language-markdown-outline: Markdown-ready link `[$title]($url)`. + +`Add BibTex citation section` + +: In the right-most column on ArXiv.org, add the copy-able box with a :tabler-math-function: BibTex citation. + +--- + +### User interface + +`Display notification when a paper is added` + +: Enable/Disable the red sliding notification displayed by PaperMemory when a Paper is parsed or a new code repository is found. + +`Use Dark Mode` + +: Choose between light and dark mode for the Popup, Advanced Options and BibMatcher. + +`Directly open the Memory when not on a Paper's page` + +: This feature assumes that your primary reason to open PaperMemory -- when you're not on a paper's page -- is to look for a paper in your Memory and therefore directly opens it without you cliking on the :material-folder-multiple-outline: button or pressing ++a++. + +`Change paper tab titles` + +: By default, PaperMemory will replace your browser's tab titles with the paper's title so they are more easily searched and bookmarked. + +`Add a link to SciRate in the popup` + +: Add a :tabler-scirate: icon to your papers to open directly on [SciRate](https://scirate.com). + +`Add a link to arXiv Vanity in the popup` + +: Add a :tabler-arxiv-vanity: icon to your papers to open directly on [ArXiv Vanity](https://www.arxiv-vanity.com/). + +`Add a link to ar5iv in the popup` + +: Add a :tabler-ar5iv: icon to your papers to open directly on [ar5iv](https://ar5iv.labs.arxiv.org/). + +`Add a link to the HuggingFace paper page in the popup (Arxiv only)` + +: Add a :tabler-hf-logo-transparent-darktheme: icon to your papers to open directly on [HuggingFace Papers](https://huggingface.co/papers). + + +### Functionalities + +`Include a paper's venue and year when available in the Markdown link` + +: Choose between `[$title]($url)` and `[$title -- $venue ($year)]($url)` when creating the :material-language-markdown-outline: Markdown links for a paper (only if a venue is known). + +`Favor PDFs for paper links (otherwise, links will point to a paper's abstract page)` + +: Whenever a link is involved (:material-language-markdown-outline:, :tabler-external-link:, :octicons-link-24: etc.) prefer linking to the online PDF (if known) over the default HTML/Abstract page. + +`Only record papers on their pdf pages, not from their abstracts` + +: Prevent PaperMemory from recording *all* papers, but only those of which you open the PDF. + +`Disable automatic recording of papers. You will have to manually open the popup and click a button` + +: Prevent PaperMemory from recording *any* paper, you will have to trigger the parsing from the Popup or the [Keyboard Shortcut](./configuration.md#keyboard-shortcuts). + +`When hitting Enter on a memory item, open a local PDF file when one is available instead of the the link to the paper.` + +: Choose to open the local PDF file you have in your `Downloads/` (if any) instead of opening the online version of the PDF, when pressing ++enter++ on a paper in your Memory. + +`Download pdfs to PaperMemory's folder in your Downloads to be able to open them instead of re-downloading.` + +: Enable the `PaperMemoryStore/` folder in your `Downloads/` to enable PaperMemory to handle local files instead of re-downloading online PDFs every time. + +`Enable parsing of arbitrary website by adding a button in the popup.` + +: Adds the `Parse Current Website` button when on an unknown web page (*i.e.* not detected as a paper provider) to record arbitrary web pages to your Memory. + +## Advanced Options + ++![Popup](assets/advanced.png){ .advanced-screenshot } +
+ +More customization options and features are available in the **Advanced Options page**: + +* Right click on the extension's button in your browser toolbar and select *Options* +* Open the Popup Menu, scroll down and click on *Options page*. +* Paste one of those URLs in your address bar + ```bash + # Chrome / Brave / Edge: + chrome-extension://ehchlpggdaffcncbeopdopnndhdjelbc/src/options/options.html + # Firefox: + moz-extension://05b6ef8a-7314-4c1d-a519-80f48a5794a6/src/options/options.html + ``` + + +`PapersWithCode preferences` + +: Choose whether or not you enable unofficial repositories and if you have a preferred (Tensorflow / PyTorch) Deep Learning framework. + +`Auto-tagging` + +: Define a list of regexs to apply to `title` and `author` fields to automatically tag papers in your Memory. + +`Preprint matching` + +: Trigger the preprint-matching process for your current memory: this will search for published versions of your ArXiv preprints. PaperMemory will continuously look for publications when you visit a preprint's ArXiv page, but this feature bulks the process for all preprints in your memory. + +`Select Sources` + +: Select the paper sources you want PaperMemory to parse papers from. + +`Data Management` + +: Manage your data: + + * export your memory as `.bib` or `.json` files + * import an existing `.json` memory you have backed-up + * export a bibliography by tag + * import a list of URLs into your memory. + +`Online synchronization` + +: Enable [Online Synchronization](features.md#online-synchronization) by providing a Github Personal Access Token. \ No newline at end of file diff --git a/docs/faq.md b/docs/faq.md new file mode 100644 index 00000000..53045588 --- /dev/null +++ b/docs/faq.md @@ -0,0 +1,89 @@ +--- +description: PaperMemory's Frequently Asked Questions +--- + +# FAQ + + +### Why does PaperMemory require access to all urls? + +Because Chrome & Brave will disable an extension by default when it auto-updates with new permissions. In this case, any new addition of a paper source will require new permissions to access the data necessary to parse the paper data and will therefore disable the extension until users re-enable it. + +### The button to go from an abstract to its pdf does not work + +In general there can be 2 reasons: + +1. It's a bug, it can happen, I'm sorry about this: you should raise an issue, and ideally even provide a fix in a Pull Resquest +2. More likely, the pdf is behind a paywall and the standard `abstract -> pdf` path was redirected by the venue's website + + +### How do you match Arxiv.org pre-prints to actual publications? + + +It's all there: [preprints](features.md#preprint-matching) 😃 + +Contributions and ideas on how to improve the process and potentially add publication sources from titles or arxiv `id` are welcome! + +### Where does PaperMemory store my data? + +It's all stored locally in your browser's local storage. If you want to transfer data to a new browser/computer, use the export/import tools in the extension's options. + +### How can I export all or some papers? + +Check out the [Advanced Options](./configuration.md#advanced-options)'s *Data Management* section, you can export: + +* your full PaperMemory data as `.json` file +* your full bibliography as `.bib` file (or `.json`) +* papers based on the tags you'be assigned them as a `.bib` file + +### Can I reference my PDF in PaperMemory? + +If you want to associate a PDF file to a paper in your PaperMemoryStore that is not downloaded from PaperMemory (for instance, it is behind a paywall), you can **download it to `~/Downloads/PaperMemoryStore/`** (maybe you're reading the paper on Sci-Hub you thief 🙃) provided the file's name contains the **paper's title** (not counting non-alphanumeric characters and case-insensitive). + +Because of limitations Browser Extensions face, a file **must** be _downloaded from the Browser to the folder_ in order to be detected. This means that if someone sends you a pdf over Slack or if you move a file to the PaperMemoryStore folder for instance, you won't be able to match it to a PaperMemory item. + +### How do I resolve duplicates? + +* If you installed PaperMemory `<0.5.3` (May 2022) you may have duplicate papers in your Memory. The easiest way to resolve this is to go to a duplicate's page, then delete it from memory and refresh the page. The automatic de-duplication features in version `0.5.3` should handle the matching _if_ the 2 papers have the same title. +* If you still see duplicates with PaperMemory `>=0.5.3` _and_ the 2 papers have the same title, this may be a problem with PaperMemory and you should open an issue here on Github. + + +### Can I access the memory full-screen? + +Sure! In the extension popup's menu, there's a link at the bottom to the full-page memory. You can also just go to this url (Thanks @kaixin96!): + +```bash +# Copy-paste this url (Chrome / Brave / Edge): +chrome-extension://ehchlpggdaffcncbeopdopnndhdjelbc/src/fullMemory/fullMemory.html +# Firefox: +moz-extension://05b6ef8a-7314-4c1d-a519-80f48a5794a6/src/fullMemory/fullMemory.html +``` + +### Can I exclude a paper source? + +Yep. In the extension popup's advanced options page: right-click the extension's icon and click on `Options`. Or click on the link at the bottom of the popup menu. Or go to this url (Thanks @kaixin96!): + +```bash +# Copy-paste this url (Chrome / Brave / Edge): +chrome-extension://ehchlpggdaffcncbeopdopnndhdjelbc/src/options/options.html +# Firefox: +moz-extension://05b6ef8a-7314-4c1d-a519-80f48a5794a6/src/options/options.html +``` + +### Are there data backups? + +In case there's a bad bug or you mess up when overwriting your memory, PaperMemory preforms weekly backups (locally as always), which you can retrieve with a little coding: + +1. Open the extension's options (either right clicking its icon, or from the extension's menu, at the bottom) +2. On the options page, open the Javascript Console with `cmd/ctrl + alt + i` or `right click > Inspect` +3. Do the following in the Javascript Console: + +```javascript +const backups = await getStorage("weeklyBackups") +console.log(Object.keys(backups)) // this shows you available backup dates +const overwrite = backups["+ +
+ ++ +
+ +## Arxiv Enhancements + +PaperMemory enhances paper pages on ArXiv.org to make its parsing abilities readily available without even opening the popup: + +* Displays the publication venue if one exists +* Allows you to directly download a paper to your [PaperMemoryStore](#local-file-storage) + +![Arxiv enhancements](assets/display.png) + + + +## Memory: Parsing papers + +### Venues + +In order to parse papers from the websites you visit, PaperMemory contains a custom function for each individual paper provider (ArXiv, ScienceDirect, OpenReview, Nature, Science, FrontiersIn, etc.). + ++[:fontawesome-brands-chrome: Chrome / :simple-brave: Brave / :fontawesome-brands-edge: Edge](https://chromewebstore.google.com/detail/paper-memory/hmebhknlgddhfbbdhgplnillngljgmdi?pli=1){ .md-button .md-button--primary } +
++ [:fontawesome-brands-firefox-browser: Firefox](https://addons.mozilla.org/en-US/firefox/addon/paper-memory/){ .md-button .md-button--primary } +
+{"use strict";/*!
+ * escape-html
+ * Copyright(c) 2012-2013 TJ Holowaychuk
+ * Copyright(c) 2015 Andreas Lubbe
+ * Copyright(c) 2015 Tiancheng "Timothy" Gu
+ * MIT Licensed
+ */var Ha=/["'&<>]/;Un.exports=$a;function $a(e){var t=""+e,r=Ha.exec(t);if(!r)return t;var o,n="",i=0,s=0;for(i=r.index;i