Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make bibtex-completion (more) interactive #361

Closed

Conversation

bdarcus
Copy link

@bdarcus bdarcus commented Feb 23, 2021

Introduction

Over the last year, there has been a lot of activity on packages that are built around the emacs completing-read API. Designed for compatibility and modularity, but in collaboration, these tools together provide users flexibility and excellent functionality.

This PR provides an alternative approach to (really just adapts code from) #355 as a means to offer bibtex-completion functionality to emacs vertically-oriented completion systems using completing-read, such as icomplete-vertical and selectrum (see #353).

The changes are minor, adding only about 70 very simple LOC. It follows @minad's advice to make the core functions interactive, using a new bibtex-completion--read function, which allows for configurable completion systems.

By default, this function uses completing-read, which is already used in bibtex-completion in a few places.

But the intention is that one could use another function there, as the default function is bound to bibtex-completion-read-backend via fset.

So ivy-bibtex and helm-bibtex could, if there was value in this, instead use functions based on ivy-read and helm-read respectively (though this appears to be unnecessary, as they seem to work fine as is). One could even write a different completing-read function, if one wanted to modify the behavior locally, or in a separate package.

Also adds a keymap; bibtex-completion-map.

I do think this demonstrates how small a footprint this sort of change would introduce, while providing more flexibility for users and developers, but without impacting the packages that depend on it.

Status

Everything seems to work correctly.

The basic bibtex-completion--read function works, and all of the action functions included in the keymap now use it.

Here's a screenshot showing the most basic, default, installation, with icomplete-vertical:

Screenshot from 2021-02-28 11-21-50

With this setup, you only have access to the default command; there are no other actions/commands available.

Adding embark, however, adds that support, and so is strongly recommended. Binding the keymap to embark-act like so ...

(setf (alist-get 'bibtex embark-keymap-alist) 'bibtex-completion-map)

... will get you quick and easy access to additional actions, as well as the ability to snapshot sub collections and act on those as with ivy-occur.

A full set of completing-read compatible packages to get, IMHO, the best experience using this includes the following:

Here is with the above installed and configured (though consult is not strictly necessary).

Screenshot from 2021-03-01 16-20-49

Next Steps?

I could use help on the following (which will be valuable even if we go with #355 instead):

  1. Reviewing the code.
  2. Testing, in general, but also to see in particular that I am correct this doesn't impact ivy-bibtex and helm-bibtex
  3. I'd also like to come up with a good example configuration, both with embark (looking into embark-direct-action-mode, tablist, and hiding the general embark commands) and without.
  4. I'm wondering how to filter on items that have notes or pdfs . See Add "=has-link=" to bibtex-completion search string #363.
  5. Would we benefit from adding an annotation function (see this comment)? For now, I have a placeholder comment in the code. If yes, that might suggest a slightly different UI for completing-read; say putting some of the metadata in annotation, and having the candidate display itself narrower than the full frame width?

Finally, while I think out-of-scope for this PR:

  1. Evaluating whether it could be helpful to have helm-bibtex and ivy-bibtex use an adapted bibtex-completion--read, and if so, implementing that. As I say above, the PR so far does not impact the functionality of the helm-bibtex and ivy-bibtex actions. The bibtex-completion commands also work, at least within helm (I haven't tested withivy).

Testing

If you want to test this, but don't have the selectrum et al packages installed, a quick and simple option I recommend using emacs -Q is one of the selectrum test scripts, preferably the selectrum-prescient one.

All you need to do, then, is eval this code:

(require 'embark)
(load "~/Code/helm-bibtex/bibtex-completion.el")
(setf (alist-get 'bibtex embark-keymap-alist) 'bibtex-completion-map)
(setq bibtex-completion-bibliography "~/org/bib/academic.bib")

Upon running one of the commands, and narrowing the candidates, you can then run embark-act (in the test scripts, it is bound to M-o).

Concerns

@tmalsburg has raised some reasonable concerns, which we can summarize as:

  1. impact on code maintainability (what this PR is intending to test/demonstrate)
  2. what to do about command names that weren't designed to be user facing?

#355 (comment)
#355 (comment)

Issue 1 I can't really assess, but is obviously the more serious issue. Hopefully this PR addresses this concern.

On 2, the function names are mostly OK, with two obvious issues:

  1. We have one function with "PDF" while all others are all lowercase. I addressed that with a888ea5.
  2. bibtex-completion-open-any; I actually think it's fine, and more importantly, have a hard time coming up with something better (maybe omit the "any"?). But if we do, we could use the same solution as 1?

I suppose if we wanted to be aggressive, we could also think about:

  • bibtex-completion-open-any -> bibtex-completion-open
  • bibtex-completion-open-url-or-doi -> bibtex-completion-open-link
  • bibtex-completion-open-pdf (per change noted above)

And for that matter consolidating around "insert" and "open" as the key action verbs.

Not sure, but it seems like something like make-obsolete could work?

;; we want new-name interactive, but `old-name` NOT interactive and deprecated
(defun old-name ()
  (message "old name"))

(defun new-name ()
  (interactive)
  (message "new name"))

(make-obsolete 'old-name 'new-name "2.0")

;; old-name function call still works
(old-name)

Relation to #355

These would be functionally the same; just different command name prefixes.

@mtreca started #355, but this code is further-developed and feature-complete. I have merged the two code bases here as an experiment.

Decisions

After working on this, I have concluded this should come down to a choice between two options:

  1. merge this PR here
  2. we develop a separate project (I don't see any benefit to anyone to host it here)

Each option is workable, I think, and each has trade-offs.

I suppose there's also a third option, which is merging this into an experimental branch for longer term consideration, and also doing the separate package now.

@bdarcus bdarcus changed the title Make bibtex-completion action-functions full commands Make bibtex-completion action-functions full commands, provide completing-read support Feb 23, 2021
@bdarcus bdarcus force-pushed the interactive-bibtex-completion branch 2 times, most recently from bb3d3e6 to 389ccf3 Compare February 23, 2021 22:01
@bdarcus bdarcus force-pushed the interactive-bibtex-completion branch from 389ccf3 to 71429d2 Compare February 24, 2021 00:01
bibtex-completion.el Outdated Show resolved Hide resolved
bibtex-completion.el Outdated Show resolved Hide resolved
bibtex-completion.el Outdated Show resolved Hide resolved
@bdarcus bdarcus force-pushed the interactive-bibtex-completion branch 2 times, most recently from 99f5349 to 233686c Compare February 24, 2021 22:17
@bdarcus
Copy link
Author

bdarcus commented Feb 25, 2021

Going to copy @myshevchuk here too, just in case he has time for input.

Makes core action-functions interactive by adding a new
`bibtex-completion--read` function, with configurable completion
systems. The default uses `completing-read`.
@bdarcus bdarcus force-pushed the interactive-bibtex-completion branch from 233686c to 4813891 Compare February 25, 2021 21:35
bibtex-completion.el Outdated Show resolved Hide resolved
bibtex-completion.el Outdated Show resolved Hide resolved
@bdarcus bdarcus force-pushed the interactive-bibtex-completion branch 2 times, most recently from 25a604f to 4fa15d0 Compare February 27, 2021 13:22
bibtex-completion.el Outdated Show resolved Hide resolved
bibtex-completion.el Outdated Show resolved Hide resolved
@bdarcus bdarcus marked this pull request as ready for review February 27, 2021 18:10
Change to cl-loop, and propertize candidates string to use
bibtex-completion-candidates, but to display using
bibtex-completion-format-entry.
@bdarcus bdarcus force-pushed the interactive-bibtex-completion branch from 0ebfea9 to 8516cfa Compare February 28, 2021 09:06
@bdarcus bdarcus changed the title Make bibtex-completion action-functions full commands, provide completing-read support Make bibtex-completion interactive Feb 28, 2021
@bdarcus bdarcus changed the title Make bibtex-completion interactive Make bibtex-completion (more) interactive Feb 28, 2021
@tmalsburg
Copy link
Owner

I'm dealing with a number of emergencies that require my full attention (nothing life threatening). I will therefore mute this and related threads for a week or two. This doesn't mean that I lost interest. Thanks for your understanding.

@bdarcus
Copy link
Author

bdarcus commented Mar 12, 2021

So here's what I'm thinking:

We should close PR for now.

Why?

I think the ideas embedded in it are sound, but for sure it would involve a change in the relations among the three packages here.

So I'm pretty sure you won't want to consider that for the short-term, and you may not even want to adopt it for the long-term.

Meanwhile, I've gotten comfortable enough with all of this to get much farther along on the independent project, which might benefit from that independence (with its own repo, issue tracker, discussion board, etc.).

OTOH, the code is here for later reference. So if you should want to incorporate the ideas in future releases, it will be easy enough to do. It's all pretty simple, after all, but there would be some decisions to make, on questions already raised (command names, for example, and whether worthwhile to refactor helm-bibtex and ivy-bibtex).

Let me know if I have that right when you get a chance @tmalsburg.

@bdarcus
Copy link
Author

bdarcus commented Mar 21, 2021

Also, I added some code to allow people to optionally use icon fonts for the pdf and note (and link, which I added) symbols, but I couldn't get bibtex-completion to use them, perhaps because these are propertized strings? I was wanting to experiment with whether it was possible to use the icon fonts, and ensure proper alignment with them. If the latter is true, it could be nice to allow this with bibtex-completion et al.

As in:

(setq! bibtex-completion-pdf-symbol (all-the-icons-icon-for-file "foo.pdf" :face ;'all-the-icons-dred))

EDIT: this on its own won't work, as at least all-the-icons fonts aren't monospaced. So in my code, I turned these into alists, so that the cdr can specify an alternate face for the icon, as a way to preserve alignment.

(defcustom bibtex-actions-icon
  `((pdf .      (,bibtex-completion-pdf-symbol . " "))
    (note .     (,bibtex-completion-notes-symbol . " "))
    (link .     (,bibtex-actions-link-symbol . " ")))
  "Configuration alist specifying which symbol or icon to pick for a bib entry.
This leaves room for configurations where the absense of an item
may be indicated with the same icon but a different face."
  :group 'bibtex-actions
  :type '(alist :key-type string
                :value-type (choice (string :tag "Icon"))))

@bdarcus
Copy link
Author

bdarcus commented Mar 25, 2021

I'm going to close this, and encourage instead merging the other smaller PRs I've submitted (though the title shortening one is the least important to me; just thought it might be helpful to someone).

As I said above, I do think the ideas here are worth considering longer term.

@tmalsburg
Copy link
Owner

I see there is a separate package now (bibtex-actions) that implements a lot of the ideas discussed above. Looks great and I'm looking forward to trying it out.

@bdarcus
Copy link
Author

bdarcus commented Apr 8, 2021 via email

@linwaytin
Copy link

First thanks @tmalsburg for this great package!!
I use it a lot when writing reports and articles.

Recently I switched from ivy to selectrum with consult.
The only function I haven't found a replacement is this package.
@mohkale has already had some nice code.
Do you mind to merge your code into this package or make it a separate package?
I really like to see this function so that I can totally get rid of the ivy dependence. (It's great. I just don't want to have too many packages on my system.)

@bdarcus
Copy link
Author

bdarcus commented May 21, 2021 via email

@bdarcus
Copy link
Author

bdarcus commented May 21, 2021

In regards to this, I'm not sure about how to approach this in a completing-read way but consult has a marvelous narrowing feature where you can quickly discard a subset of completing-read entries interactively. For eg. with consult-imenu you can enter f and it only shows function imenu entries. That may be useful for this.

@mohkale - I forgot about this post of your's above. How do you see this applying to bibtex-actions? On what would you narrow?

Daniel actually submitted a patch to emacs to add grouping support, which was merged yesterday.

@mohkale
Copy link
Contributor

mohkale commented May 21, 2021

@bdarcus

On what would you narrow?

You could narrow on what you specified (presence of PDF, or notes). I've actually never needed this but I just set it up so that you can narrow on the kind of bibtex entry it is.

(require 'bibtex-completion)

(defvar consult-bibtex-history+ nil
  "History for `consult-bibtex+'.")

(defvar consult-bibtex-default-action+ 'consult-bibtex-insert-pdftools-link
  "The default action for the `consult-bibtex+` command.")

(defvar consult-bibtex-narrow+
  '((?b . "Book")
    (?u . "Unpublished")
    ;; Fallback for when none of the others apply.
    (?o . "Other"))
  "Narrowing configuration for `consult-bibtex+'.")

(defun consult-bibtex-candidates+ ()
  (cl-loop
   for cand in (bibtex-completion-candidates)
   with cand-str = nil
   do (setq cand-str
            (bibtex-completion-format-entry (cdr cand) (1- (frame-width))))
   ;; Add a `consult--type' property for narrowing support.
   do (add-text-properties 0 1
                           `(consult--type
                             ,(or
                               (when-let ((type (bibtex-completion-get-value "=type=" cand)))
                                 (car (rassoc (capitalize type) consult-bibtex-narrow+)))
                               (car (rassoc "Other" consult-bibtex-narrow+)))
                             consult--candidate
                             ,(bibtex-completion-get-value "=key=" cand))
                           cand-str)
   collect cand-str))

(defun consult-bibtex-read-entry+ (&optional arg local-bib)
  (when arg
    (bibtex-completion-clear-cache))
  (bibtex-completion-init)
  (let* ((candidates (consult-bibtex-candidates+))
         (preselect
          (when-let ((key (bibtex-completion-key-at-point)))
            (cl-find-if (apply-partially #'string-equal key) candidates))))
    (consult--read candidates
                   :prompt (format "BibTeX entries%s: " (if local-bib " (local)" ""))
                   :require-match t
                   :category 'bibtex-completion
                   :lookup 'consult--lookup-candidate
                   :default preselect
                   :title (consult--type-title consult-bibtex-narrow+)
                   :narrow (consult--type-narrow consult-bibtex-narrow+)
                   :history '(:input consult-bibtex-history+))))

(defun consult-bibtex+ (&optional arg local-bib)
  (interactive "P")
  (when-let ((entry (consult-bibtex-read-entry+ arg local-bib)))
    (if consult-bibtex-default-action+
        (funcall-interactively consult-bibtex-default-action+ entry)
      (warn "`consult-bibtex-default-action+' is unassigned."))))

Now if I run the command you can see entries of type book are grouped together and entries of type unpublished are also grouped together. I've set it up so anything without a type I'd like to narrow on gets put into the Other group.

Screenshot_20210521_114735

And I can use consult to narrow onto a specific group type such that none of the other candidates are visible.

Screenshot_20210521_115436

This is of course a very simplistic example. I can also set it up so that I can narrow on notes or pdfs (I plan to now). But as a demonstration it should be enough.

While narrowing is consult specific, you can group candidates with completing-read by using the metadata property.

(completing-read "Prompt: "
                 (lambda (str pred action)
                   (if (eq action 'metadata)
                       `((x-title-function . GROUP-FUNCTION))
                     (complete-with-action action cands str pred))))

Daniel actually submitted a patch to emacs to add grouping support, which was merged yesterday.

Incidentally I don't think the group-function minad submitted allows narrowing. All it does is let completion frontends like selectrum or bibtex group candidates together based on it, which could be enough to meet what you intended. Also I think this has been around for a while, minads recent changes were just changing the name of x-title-function to something like group-function (you can look that up I'm sure).

EDIT:

I've added pdf and note based narrowing, now I can quickly see only the bibtex entries that have note files.

Screenshot_20210521_121913

@bdarcus
Copy link
Author

bdarcus commented May 21, 2021

You could narrow on what you specified (presence of PDF, or notes).

That's what I was thinking.

So it's another way to handle what we now do with "initial-value" filters.

emacs-citar/citar#43

I've actually never needed this but I just set it up so that you can narrow on the kind of bibtex entry it is.

Got it.

Incidentally I don't think the group-function minad submitted allows narrowing. All it does is let completion frontends like selectrum or bibtex group candidates together based on it, which could be enough to meet what you intended. Also I think this has been around for a while, minads recent changes were just changing the name of x-title-function to something like group-function (you can look that up I'm sure).

Yes; all correct. But it now formalizes grouping as a part of completing-read.

Edit: see, for example, this commit or vertico.

I've added pdf and note based narrowing, now I can quickly see only the bibtex entries that have note files.

What do you think of the UX compared to the approach I took in bibtex-actions? Do you see an advantage?

@mohkale
Copy link
Contributor

mohkale commented May 21, 2021

@bdarcus

What do you think of the UX compared to the approach I took in bibtex-actions? Do you see an advantage?

I don't really use bibtex-completion, although I'm loving all the progress you've made with it so far. I feel like your implementation is better for regular commands. If users use bibtex-actions-open-pdf directly as an interactive command, instead of through embark, your filtering approach is clearly better. Users shouldn't be presented with bibtex entries that don't have PDFs when their trying to open PDFs. Arguably there could be some issues where a user thinks an entry does have a PDF when it doesn't, and they keep trying to find it through a filtered command only to realise it doesn't exist, but that's a configuration issue and I don't expect developers to account for all of those. The main advantage of narrowing (as described here) is that it's reversable, and the goal is flexible. You run one command to view all bibtex entries, then you quickly filter down to the ones you want. If you want to open PDFs you narrow to only files that don't have PDFs. If the one your after doesn't exist you can remove the narrowing and verify that it exists as an entry, but doesn't have a PDF. Personally I prefer the single entry point philosophy, one command, a million ways to use it 😄. It's subjective.

@bdarcus
Copy link
Author

bdarcus commented May 21, 2021

The main advantage of narrowing (as described here) is that it's reversable, and the goal is flexible. You run one command to view all bibtex entries, then you quickly filter down to the ones you want. If you want to open PDFs you narrow to only files that don't have PDFs. If the one your after doesn't exist you can remove the narrowing and verify that it exists as an entry, but doesn't have a PDF.

I'll have to experiment with that consult-based filtering.

We also tried to address this issue here: emacs-citar/citar#120.

This uses a combination of "future" history and "predefined" search terms.

But I'm not sure if this was the best approach or not, how it compares to your approach, etc.

Personally I prefer the single entry point philosophy, one command, a million ways to use it

I have thought about a single entry point, but not sure the point given the current design, given one could just write an alias to whatever you want and use that with embark.

@oantolin
Copy link

oantolin commented May 21, 2021

@mohkale

Personally I prefer the single entry point philosophy, one command, a million ways to use it 😄. It's subjective.

If you have a bunch of commands that work on the same type of objects, Embark let's you pick any single one of them and think of it as the "single entry point". For example, before writing Embark I used to use find-file, delete-file, copy-file, rename-file, etc. Now I think of find-file as the entry point and run the others as embark actions.

So for bibtex-actions, whichever command is the the one you use most often, think of that one as the entry point and then run the others as actions.

@mohkale
Copy link
Contributor

mohkale commented May 22, 2021

So for bibtex-actions, whichever command is the the one you use most often, think of that one as the entry point and then run the others as actions.

@oantolin That's exactly what I do and meant by a millions "ways to use it". Apologies if that wasn't clear enough, although thanks for clarifying the point. 😄.

@oantolin
Copy link

oantolin commented May 22, 2021

Sorry, @mohkale, I think I read too quickly and didn't realize that's what you meant!

@linwaytin
Copy link

https://github.com/bdarcus/bibtex-actions

@bdarcus Can this package load the .bib file specified in a .tex file automatically, llike ivy-bibtex-with-local-bibliography? This is the function I'm using.

@bdarcus
Copy link
Author

bdarcus commented May 22, 2021

https://github.com/bdarcus/bibtex-actions

@bdarcus Can this package load the .bib file specified in a .tex file automatically, llike ivy-bibtex-with-local-bibliography? This is the function I'm using.

I don't actually know.

I use bibtex-completion currently for file loading, but I'm not sure if that extends to this.

Do you know @mohkale?

@bdarcus
Copy link
Author

bdarcus commented May 22, 2021

BTW, I've been active in helping get native org-cite functionality added to org-mode. This sort of feature would probably ideally be generalized to include org and also pandoc files, including csljson bib files.

@mohkale
Copy link
Contributor

mohkale commented May 22, 2021

Do you know @mohkale?

Nope. I rarely use the in-file bibtex stuff and when I do it's only one or two of them. Is there a reason you don't want to create a propper bibtex file for them @linwaytin? In my case I keep everything I've ever read or intend to read in bibtex files because that makes it easier to reference them and pull up my notes for them. You can always make a local bibtex file for whatever your working on and use directory-local-variables to make bibtex-completion look in there as well.

@oantolin
Copy link

oantolin commented May 22, 2021

@mohkale I believe @linwaytin is talking about using a bibtex file, and is wondering if bibtex-actions can figure out which bibtex file it is by looking for a \bibliography{use-this-one} in the current buffer, instead of having to set a variable.

@mohkale
Copy link
Contributor

mohkale commented May 22, 2021

Oh, my bad, I read that too quickly 😓.

I don't think it can, this seems more like like the kinda thing local-variables should be used for.

@oantolin
Copy link

I don't think it can, this seems more like like the kinda thing local-variables should be used for.

I disagree, looking at the current buffer to figure out which bibtex file to use seems like an obvious and great solution. Something simple like this might be enough:

(defvar old--bibtext-files nil)

(defun toggle-local-bibliography ()
  "Toggle whether to use the local bibliography."
  (interactive)
  (if old--bibtext-files
      (progn
        (setq bibtex-files old--bibtext-files
              old--bibtext-files nil)
        (message "Using global bibliography %s" bibtex-files))
    (save-excursion
      (goto-char (point-min))
      (if (search-forward-regexp "\\\\bibliography{\\([^}]+\\)}" nil t)
          (let ((bib (expand-file-name (concat (match-string 1) ".bib"))))
            (setq old--bibtext-files bibtex-files bibtex-files (list bib))
            (message "Using %s" bib))
        (message "No local bibliography found")))))

@oantolin
Copy link

I think that advanced bibtex users tend to have a few large centralized bibtex files, with all the references they might use, maybe one file per subfield or something like that.; but many naive bibtex users (and I'm still at this stage!), tend to use one bib file per paper. The naive strategy seems inferior in the long run, but at least at first, the need to fix all the keys to be unique and keep things tidy seems overwhelming and it just feels easier to reduce the scope of the problem to the references of a single paper. People following the naive strategy tend to copy and paste references from the bib files of older papers into the new bib file, and then just search for the remaining references they need as they go along.

People using the one bib-file-per-paper approach probably don't even have a good setting for bibtex-files and would always want the local bibliography.

@linwaytin
Copy link

Yes, I am not an advanced bibtex user. I currently use a bib file for a project.
Thanks for all your answers.

@bdarcus
Copy link
Author

bdarcus commented May 25, 2021

I played a bit with narrowing using consult-buffer @mohkale.

The main advantage of narrowing (as described here) is that it's reversable, and the goal is flexible.

Yes, but so are initial-values and future history. One can simply delete the initial-value from the prompt, for example.

The narrowing UI is arguably a bit more elegant though.

@mohkale
Copy link
Contributor

mohkale commented May 25, 2021

Yes, but so are initial-values and future history. One can simply delete the initial-value from the prompt, for example.

Yes. I seem to have misunderstood how that works there. I thought the collection was filtered before completing-read but it looks like its filtered afterwards in which case their mostly equivalent.

@bdarcus
Copy link
Author

bdarcus commented May 26, 2021

Yes. I seem to have misunderstood how that works there. I thought the collection was filtered before completing-read but it looks like its filtered afterwards in which case their mostly equivalent.

Here's how the "presets" feature works. If you add this ...

(setq bibtex-actions-presets '("book" "article" "inbook"))

... you can then access those by doing M-n from the prompt.

It doesn't, however, match against the type field, of course, but against the whole candidate string.

But the advantage is you could set those to a regexp to grab, for example, all articles published since 2019, or whatever.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants