Skip to content

meedstrom/org-node

Repository files navigation

org-node

MELPA

News

On [2024-09-19], the git repo history was REWRITTEN! You may have had to delete and reinstall the package. (Depends on your package manager.) Sorry about that.

What was rewritten? I’ve erased the ;; Package-Version: comment line in file contents, because of Emacs 30’s new (use-package :vc) keyword, which defaults to installing the oldest commit with a given Package-Version, not the newest commit.

Even if that’s 5 years old. This user-unfriendly change breaks with what has been a near-consensus in the community for over a decade, and will surprise and break some innocent’s initfiles every week for the next two decades. Please chime in at the mailing list discussion!

(For more reading, you can check out some long related discussions about the untrustworthiness of Package-Version, and that’s leaving aside the fact the ecosystem doesn’t really make use of the concept of versioned releases in a way that matters, and when some tools pretend it does instead of going with the newest commit, they make it less stable for all.)

Lesson learned – when you create a package, never commit a Package-Version line! If Emacs 30 has its way, that line will curse your repo for the rest of time.

Background

What’s all this

I found org-roam too slow, so I made quickroam. And that idea spun off into this package, a standalone thing. I hope it’s also easier to pick up than org-roam.

  • If you were using org-roam, there is nothing to migrate. You can use both packages. It’s exactly the same on-disk format: “notes” are identified by their org-id.

    With optional shims, you can even skip syncing the org-roam DB and continue using its rich backlinks buffer and org-roam-capture!

    In pursuit of being “just org-id”, this package has no equivalent setting to org-roam-directory – it just looks up org-id-locations.

  • If you were not using org-roam, maybe think of it as somewhat like org-recent-headings beefed-up to the extent that you won’t need other methods of browsing.

    If you were the sort of person to prefer ID-links over file links or any other type of link, you’re in the right place! Now you can rely on IDs, and—if you want—stop worrying about filenames, directories and subtree hierarchies. As long as you’ve assigned an ID to a heading, you can find it later.

What’s a “node”?

My life can be divided into two periods ”before org-roam” and ”after org-roam”. I crossed a kind of gap once I got a good way to link between my notes. It’s odd to remember when I just relied on browsing subtrees and folders – what a strange way to work!

I used to lose track of things I had written, under some forgotten heading in a forgotten file in a forgotten directory. The org-roam method let me find and build on my own work, instead of recreating my work all the time.

At the core, all the “notetaking packages” (orgrr/zk/zetteldeft/org-roam/denote/howm/minaduki/…) try to help you with this: make it easy to link between notes and explore them.

Right off the bat, that imposes two requirements: a method to search for notes, since you can’t link to something you can’t search for, and a design-choice about what kinds of things should turn up as search hits. What’s a “note”?

Just searching for Org files is too coarse. Just searching for any subtree anywhere brings in too much clutter.

Here’s what org-roam invented. It turns out that if you limit the search-hits to just those files and subtrees you’ve deigned to assign an org-id – which roughly maps to everything you’ve ever thought it was worth linking to – it filters out the noise excellently.

Once a subtree has an ID you can link to, it’s a “node” because it has joined the wider graph, the network of linked nodes. I wish the English language had more distinct sounds for the words “node” and “note”, but to clarify, I’ll say “ID-node” when the distinction matters.

Features

A comparison of three similar systems, which permit relying on org-id and don’t lock you into the all-too-seductive (for developers) concept of “one-note-per-file”.

Featureorg-roamorg-nodeorg-super-links
Backlinksyesyesyes
Node search and insertyesyes– (suggests org-ql)
Node aliasesyesyes
Node exclusionyeslimitednot applicable
Rich backlinks bufferyesyes (org-roam’s)
Customize how backlinks shownyesyes (org-roam’s)yes
Reflinksyesyes (as backlinks)
Ref searchyesyes (as aliases)not applicable
Org 9.5 @citations as refsyesyesnot applicable
Support org-ref v3yeslimitednot applicable
Support org-ref v2yesnot applicable
Work thru org-roam-captureyesyes?
Work thru org-captureyes?
Daily-nodesyesyes
Node seriesyes
Show backlinks in same windowyesyes
Cooperate with org-super-linksyesnot applicable
Fix link descriptionsyes
List dead linksyes
Rename file when title changesyes
Warn about duplicate titlesyes
Principled “related-section”yes
Refileyes
Untitled notes
Support roam: linksyes– (wontfix)not applicable
Can have separate note pilesyes– (wontfix)not applicable
Some query-able cacheEmacSQLhash tables
Async cache rebuildyesnot applicable
Time to cache my 3000 nodes2m 48s0m 01snot applicable
Time to save file w/ 400 nodes5–10sinstant?
Time to open minibuffer1–3sinstantnot applicable

Setup

Install

Assuming your package manager knows about MELPA, add this initfile snippet:

(use-package org-node
  :after org
  :config (org-node-cache-mode))

If you are an org-roam user, you’ll want the following module as well. Please check its README to make org-node work with org-roam side-by-side.

(use-package org-node-fakeroam
  :defer)

Both are recent additions to MELPA, so you may have to refresh the recipe list first:

  • On package.el: M-x package-refresh-contents RET
  • On Elpaca: C-u M-x elpaca-update-menus RET
  • On Straight (I think): M-x straight-fetch-all RET

Quick start

If you’re new to these concepts, fear not. The main things for day-to-day operation are two verbs: “find” and “link”.

Pick some short keys and try them out.

(keymap-set global-map "M-s M-f" #'org-node-find)
(keymap-set org-mode-map "M-s M-i" #'org-node-insert-link)

To browse config options, type M-x customize-group RET org-node RET.

Final tip: there’s no separate command for creating a new node! Reuse one of the commands above, then type the name of a node that doesn’t exist. Try it and see what happens!

Backlinks

What are backlinks?

Backlinks are the butter on the bread of your notes. If you’ve ever seen a “What links here” section on some webpage, that’s exactly what it is. Imagine seeing that, all the time. The following sections outline two general ways to do so.

Backlink solution 1: Borrow org-roam’s backlink buffer

As a Roam user, you can keep using M-x org-roam-buffer-toggle.

If it has been slow, or saving files has been slow, org-node-fakeroam gives you some new ways to keep Roam’s DB data fresh, circumventing Roam’s “autosync mode”.

Backlink solution 2: Print inside the file

I rarely have the screen space to display a backlink buffer. Because it needs my active involvement to keep visible, I go long periods seeing no backlinks. This solution can be a great complement (or even stand alone).

Option 2A: Let org-node manage a :BACKLINKS: property

For a first-time run, type M-x org-node-backlink-fix-all. (Don’t worry, if you change your mind, you can undo with M-x org-node-backlink-regret.)

Then start using the minor mode org-node-backlink-mode, which keeps these properties updated. Init snippet:

(add-hook 'org-mode-hook #'org-node-backlink-mode)

NOTE: I hadn’t even realized this, but people who don’t use visual-line-mode may not find this solution very scalable, since Org properties must stay on one line. This marks the second time I distribute software that presumes soft-wrapped text buffers :-)

Option 2B: Let org-super-links manage a :BACKLINKS:...:END: drawer

I think the following should work. Totally untested, let me know!

(add-hook 'org-node-insert-link-hook #'org-node-convert-link-to-super)

Bad news: this is currently directed towards people who used org-super-links from the beginning, or people who are just now starting to assign IDs, as there is not yet a command to add new BACKLINKS drawers in bulk to preexisting nodes. (org-super-links#93)

Misc

Managing org-id-locations

I find unsatisfactory the config options in org-id (Why? See Taking ownership of org-id), so org-node gives you an extra way to feed data to org-id, making sure we won’t run into “ID not found” situations.

Example setting:

(setq org-node-extra-id-dirs
      '("~/org/"
        "~/Syncthing/"
        "/mnt/stuff/"))

Do a M-x org-node-reset and see if it can find your notes now.

Org-capture

You may have heard that org-roam has its own special set of capture templates: the org-roam-capture-templates.

People who understand the magic of capture templates, they may take this in stride. Me, I never felt confident using a second-order abstraction over an already leaky abstraction I didn’t fully understand.

Can we just use vanilla org-capture? That’d be less scary. The answer is yes!

The secret sauce is (function org-node-capture-target):

(setq org-capture-templates
      '(("i" "Capture into ID node"
         plain (function org-node-capture-target) nil
         :empty-lines-after 1)

        ("j" "Jump to ID node"
         plain (function org-node-capture-target) nil
         :jump-to-captured t
         :immediate-finish t)

        ;; Sometimes handy after `org-node-insert-link' to
        ;; make a stub you plan to fill in later
        ("q" "Make quick stub ID node"
         plain (function org-node-capture-target) nil
         :immediate-finish t)))

With that done, you can optionally configure the everyday commands org-node-find & org-node-insert-link to outsource to org-capture when they try to create new nodes:

(setq org-node-creation-fn #'org-capture)

Node series

Do you already know about “daily-notes”? Then get started with a keybinding such as:

(keymap-set global-map "M-s s" #'org-node-series-dispatch)

and configure org-node-series-defs if needed. See wiki.

What are series?

It’s easiest to explain series if we use “daily-notes” (or “dailies”) as an example of a series.

Roam’s idea of a “daily-note” is the same as an org-journal entry: a file/entry where the title is just today’s date.

You don’t need software for that basic idea, only to make it extra convenient to navigate them and jump back and forth in the series.

Thus, fundamentally, any “journal” or “dailies” software are just operating on a sorted series to navigate through. You could have series about, let’s say, historical events, Star Trek episodes, your school curriculum…

Define more series in the variable org-node-series-defs. Already included is a definition that approximates the org-roam-dailies defaults, but I encourage you to override it to suit your tastes. Wiki.

You may be taken aback that defining a new series requires writing 5 lambdas, but once you get the hang of it, you can often reuse those lambdas.

Future

A future version will likely bring convenient wrappers that let you define a series in 1-2 lines.

It’s also possible we just redesign this completely. Input welcome. How would you like to define a series? Where should the information be stored?

An analogue to org-roam-node-display-template?

To customize how the nodes look in the minibuffer, configure org-node-affixation-fn:

M-x customize-variable RET org-node-affixation-fn RET

You may also want to set org-node-alter-candidates to t.

Completion-at-point

To complete words at point into known node titles:

(org-node-complete-at-point-mode)
(setq org-roam-completion-everywhere nil) ;; Prevent Roam's variant

Grep

If you have Ripgrep installed on the computer and consult installed on Emacs, you can use this command to grep across all your Org files at any time.

(keymap-set global-map "M-s M-g" #'org-node-grep) ;; Requires consult

This is can be a real power tool for mass edits. Say you want to rename some Org tag :math: to :Math: absolutely everywhere. Then you could follow a procedure such as:

  1. Use org-node-grep and type :math:
  2. Use embark-export (see embark)
  3. Use wgrep-change-to-wgrep-mode (see wgrep)
  4. Do a query-replace (M-%) to replace all :math: with :Math:
  5. Type C-c C-c to apply the changes

Let org-open-at-point detect refs

Say there’s a link to a web URL, and you’ve forgotten you also have a node listing that exact URL in its ROAM_REFS property.

Wouldn’t it be nice if, clicking on that link, you automatically visit that node first instead of being sent to the web? Here you go:

(add-hook 'org-open-at-point-functions
          #'org-node-try-visit-ref-node)

Limitation: TRAMP

Working with files over TRAMP is unsupported for now. Org-node tries to be very fast, often nulling file-name-handler-alist, which TRAMP needs.

The best way to change this is to file an issue to show you care :-)

Limitation: Encryption

Encrypted nodes probably won’t be found. Same as above, file an issue.

Limitation: Unique titles

If two ID-nodes exist with the same title, one of them disappears from minibuffer completions.

That’s just the nature of completion. Other packages such as Roam have the same limitation. Much can be said for embracing the uniqueness constraint, and org-node will print messages telling you about title collisions.

Anyway… there’s a workaround. Assuming you leave org-node-affixation-fn at its default setting, just add to initfiles:

(setq org-node-alter-candidates t)

This lets you match against the node outline path and not only the title, which resolves most conflicts given that the most likely source of conflict is subheadings in disparate files, that happen to be named the same. Some people make this trick part of their workflow.

NB: this workaround won’t help the in-buffer completions provided by org-node-complete-at-point-mode, but with a light peppering of luck this isn’t something you’ll ever have to notice.

Limitation: Excluding notes

The option org-node-filter-fn works well for excluding TODO items that happen to have an ID, and excluding org-drill items and that sort of thing, but beyond that, it has limited utility because unlike org-roam, child ID nodes of an excluded node are not excluded!

So let’s say you have a big archive file, fulla IDs, and you want to exclude all of them from appearing in the minibuffer. Putting a :ROAM_EXCLUDE: t at the top won’t do it. As it stands, what I’d suggest is to use the file name.

While a big selling point of IDs is that you avoid depending on filenames, it’s often pragmatic to let up on purism just a bit :-) It works well for me to filter out any file or directory that happens to contain “archive” in the name – see the last line here:

(setq org-node-filter-fn
      (lambda (node)
        (not (or (org-node-get-todo node) ;; Ignore headings with todo state
                 (member "drill" (org-node-get-tags node)) ;; Ignore :drill:
                 (assoc "ROAM_EXCLUDE" (org-node-get-properties node))
                 (string-search "archive" (org-node-get-file-path node))))))

Limitation: Org-ref

Org-node supports the Org 9.5 @citations, but not fully the aftermarket org-ref &citations that emulate LaTeX look-and-feel, since it nearly doubles my scan time if I amend org-link-plain-re to match all of org-ref-cite-types.

What works is bracketed Org-ref v3 citations that start with “cite”, e.g. [[citep:...]], [[citealt:...]], [[citeauthor:...]], since org-node-parser.el is able to pick them up for free. What doesn’t work is e.g. [[Citealt*:...]] since it doesn’t start with “cite”, nor plain citep:... since it is not wrapped in brackets.

If you need more of Org-ref, you have at least two options:

  • Use org-roam - see discussions on boosting its performance here and here
  • Get your elbows dirty and try to update the archived branch “orgref”, see relevant commit.

Toolbox

Basic commands:

  • org-node-find
  • org-node-insert-link
  • org-node-insert-transclusion
  • org-node-insert-transclusion-as-subtree
  • org-node-visit-random
  • org-node-refile
  • org-node-series-dispatch
    • Browse node series – see README
  • org-node-extract-subtree
    • A bizarro counterpart to org-roam-extract-subtree. Export the subtree at point into a file-level node, leave a link where the subtree was, and show the new file as current buffer.
  • org-node-nodeify-entry
    • (Trivial) Give an ID to the subtree at point, and run the hook org-node-creation-hook
  • org-node-insert-heading
    • (Trivial) Shortcut for org-insert-heading + org-node-nodeify-entry
  • org-node-grep
    • (Requires consult) Grep across all known Org files.
  • org-node-fakeroam-show-roam-buffer
    • A different way to invoke the Roam buffer: display the buffer or refresh it if it was already visible. And a plot twist, if it was not visible, do not refresh until the second invocation.
      • Useful if you have disabled the automatic redisplay, because the Roam command org-roam-buffer-toggle is not meant for that.

Rarer commands:

  • org-node-lint-all-files
    • Can help you fix a broken setup: it runs org-lint on all known files and generates a report of syntax problems, for you to correct at will. Org-node assumes all files have valid syntax, though many of the problems reported by org-lint are survivable.
  • org-node-list-dead-links
    • List links where the destination ID could not be found
  • org-node-list-reflinks
    • List citations and non-ID links
  • org-node-rewrite-links-ask
    • Look for link descriptions that got out of sync with the corresponding node title, then prompt at each link to update it
  • org-node-rename-file-by-title
    • Auto-rename the file based on the current #+title
      • Works as an after-save-hook! Does nothing until you configure org-node-renames-allowed-dirs.
      • Please note that if your filenames have datestamp prefixes, it is important to get org-node-datestamp-format right or it may clobber a pre-existing datestamp.
  • org-node-backlink-fix-all
    • Update BACKLINKS property in all nodes
  • org-node-list-feedback-arcs
  • org-node-rename-asset-and-rewrite-links
    • Interactively rename an asset such as an image file and try to update all Org links to them. Requires wgrep.
      • NOTE: For now, it only looks for links inside the root directory that it prompts you for, and sub and sub-subdirectories and so on – but won’t find a link outside that root directory.

        Like if you have Org files under /mnt linking to assets in /home, those links won’t be updated. Neither if you choose ~/org/subdir as the root directory will links in ~/org/file.org be updated.

Appendix

Appendix I: Rosetta stone

API cheatsheet between org-roam and org-node.

Actionorg-roamorg-node
Get ID near point(org-roam-id-at-point)(org-id-get nil nil nil t)
Get node at point(org-roam-node-at-point)(org-node-at-point)
Get list of files(org-roam-list-files)(org-node-list-files)
Prompt user to pick a node(org-roam-node-read)(org-node-read)
Get backlink objects(org-roam-backlinks-get NODE)(org-node-get-backlinks NODE)
Get reflink objects(org-roam-reflinks-get NODE)(org-node-get-reflinks NODE)
Get title(org-roam-node-title NODE)(org-node-get-title NODE)
Get title of file where NODE is(org-roam-node-file-title NODE)(org-node-get-file-title NODE)
Get title or name of file where NODE is(org-node-get-file-title-or-basename NODE)
Get name of file where NODE is(org-roam-node-file NODE)(org-node-get-file-path NODE)
Get ID(org-roam-node-id NODE)(org-node-get-id NODE)
Get tags(org-roam-node-tags NODE)(org-node-get-tags NODE), no inheritance
Get outline level(org-roam-node-level NODE)(org-node-get-level NODE)
Get whether this is a subtree(zerop (org-roam-node-level NODE))(org-node-get-is-subtree NODE)
Get char position(org-roam-node-point NODE)(org-node-get-pos NODE)
Get properties(org-roam-node-properties NODE)(org-node-get-properties NODE), no inheritance
Get subtree TODO state(org-roam-node-todo NODE)(org-node-get-todo NODE)
Get subtree SCHEDULED(org-roam-node-scheduled NODE)(org-node-get-scheduled NODE)
Get subtree DEADLINE(org-roam-node-deadline NODE)(org-node-get-deadline NODE)
Get subtree priority(org-roam-node-priority NODE)(org-node-get-priority NODE)
Get outline-path(org-roam-node-olp NODE)(org-node-get-olp NODE)
Get ROAM_REFS(org-roam-node-refs NODE)(org-node-get-refs NODE)
Get ROAM_ALIASES(org-roam-node-aliases NODE)(org-node-get-aliases NODE)
Get ROAM_EXCLUDE(assoc "ROAM_EXCLUDE" (org-node-get-properties NODE)), no inheritance
Ensure fresh data(org-roam-db-sync)(org-node-cache-ensure t t)

About

A notetaking system like Roam using Emacs Org-mode

Resources

License

Stars

Watchers

Forks