Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow format-entry function to take optional template parameter #367

Closed
wants to merge 12 commits into from

Conversation

bdarcus
Copy link

@bdarcus bdarcus commented Mar 24, 2021

Add:

  • optional template parameter to bibtex-completion-format-entry
  • a new bibtex-completion-process-display-format function

Remove:

  • the internal format var and init setq (so to decouple bibtex-completion-format-entry from bibtex-completion-init)

Also, bump version to 1.1, so other packages can specify this requirement (since they would break if they relied on this change, but loaded an earlier version).


Screenshot from 2021-03-26 11-04-25

I want to do as title says, without breaking anything of course ...

(setq myfoo-formats-suffix "(${=key=:15}) ${=type=:12}:${tags:30}")

The obvious and clean solution is to modify bibtex-completion-format-entry to take an optional template parameter.

The screenshot above is with this PR (also shows has-note filtering as mentioned in #363). The "suffix" content on the right is using a different face from the main display, and that content generated by a different template.

This involves a net neutral change in LOC.

@bdarcus bdarcus changed the title allow format-entry function to take optional template parameter WIP allow format-entry function to take optional template parameter Mar 24, 2021
bibtex-completion.el Outdated Show resolved Hide resolved
To start, let's just mirror the main display-formts defcustom, and set
it nil.

If more flexibility is needed, can always turn this into a more general
alist variable.
@bdarcus bdarcus marked this pull request as ready for review March 25, 2021 17:17
@bdarcus bdarcus changed the title WIP allow format-entry function to take optional template parameter allow format-entry function to take optional template parameter Mar 25, 2021
bibtex-completion.el Outdated Show resolved Hide resolved
@bdarcus bdarcus changed the title allow format-entry function to take optional template parameter Allow format-entry function to take optional template parameter Mar 27, 2021
@@ -352,6 +360,11 @@ more types can slow down resolution for large biblioraphies.")
"Stores `bibtex-completion-display-formats' together with the \"used width\" of each format string.
This is set internally.")


(defvar bibtex-completion-display-formats-suffix-internal nil
Copy link
Author

@bdarcus bdarcus Mar 28, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
(defvar bibtex-completion-display-formats-suffix-internal nil
(defvar bibtex-completion--display-formats-suffix-internal nil

I was wanting to make these symbol name private using the -- convention, but have not since it's not currently used anywhere else in the code. Would be nice to address this at some point, as it can confusing to navigate it all without that convention.

@bdarcus
Copy link
Author

bdarcus commented Mar 28, 2021

To update you @tmalsburg; here's what I've done on the PRs:

First, I closed #361, after concluding it didn't make sense now. It's easy enough to come back to at any point.

Instead, I submitted three small ones, all of them backward compatible; in order of importance:

  1. this one
  2. Add =has-link= and bibtex-completion-link-symbol #366 to add =has-link=
  3. Add shorten-title function #364 to add a shorten-title function; may or may not make sense

EDIT: On this PR, I've linked in an issue from selectrum, where I'm trying to sort out a detail in my implementation. I think this PR relates to that, but see my line comment below. If I'm right, that would allow arbitrary templates, and so more flexibility for formatting of different strings.

@@ -412,24 +427,32 @@ Also sets `bibtex-completion-display-formats-internal'."
(user-error "Bibliography file %s could not be found" file)))
(bibtex-completion-normalize-bibliography))

;; Pre-calculate minimal widths needed by the format strings for
;; various entry types:
(setq bibtex-completion-display-formats-internal
Copy link
Author

@bdarcus bdarcus Mar 29, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was also just wondering: couldn't this be set within the format-entry function? Like this?

  (let* ((processed-template
          (bibtex-completion--process-display-formats template))

That would then allow to decouple this function from the init function, so give more flexibility?

Basically, ideally I'm wanting to be able to pass in whatever arbitrary template, so I can do things like this:

(bibtex-completion-format-entry
  candidate
  (1- (frame-width))
  bibtex-actions-template-suffix)

Copy link
Author

@bdarcus bdarcus Apr 3, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did a quick comparison of the existing bibtex-completion-format-entry, and my modified one, which uses the above approach.

ELISP> (benchmark-run 100
  (bibtex-actions--format-entry
   "aclu2011"
   (1- (frame-width))
   bibtex-actions-display-template))
(0.008320599000000001 0 0.0)

ELISP> (benchmark-run 100
  (bibtex-completion-format-entry
   "aclu2011"
   (1- (frame-width))))
(0.027423597 0 0.0)

They're both fast, but using this approach is even faster, though that could be because I changed mapcar to cl-loop?

So I've just pushed a commit that implements this, and then a second one that removes the now unneeded "internal" format variables.

a6dfc52
ba8e97f


Interestingly, my bibtex-actions--get-candidates function, which transforms the output of bibtex-completion-candidates, is really slow by comparison, even though the UI in selectrum seems fast.

ELISP> (benchmark-run 100 (bibtex-completion-candidates))
(1.018316332 2 0.2629074330000023)

ELISP> (benchmark-run 100 (bibtex-actions--get-candidates))
(20.117633309 43 6.017887700999999)

I ended up with a solution where I don't use the bc candidate string at all, but instead use format-entry directly there, along with adding some "invisible" metadata.

Maybe I make up elsewhere what I lose here?

bdarcus added a commit to emacs-citar/citar that referenced this pull request Mar 30, 2021
Add suffix UI, using the same configuration method as for the main display formatting.

Also add annotation-function, to provide the same UI pre-Emacs 28..

To enable this, I have adapted some code from bibtex-completion. I will 
that code, or greatly simplify it, if and when this PR is merged:

tmalsburg/helm-bibtex#367

Also, change 'bibtex-actions-icons' to 'bibtex-actions-symbols'.

Closes #44
bdarcus added a commit to emacs-citar/citar that referenced this pull request Mar 30, 2021
Add suffix UI, using the same configuration method as for the main display formatting.

Also add annotation-function, to provide the same UI pre-Emacs 28..

To enable this, I have adapted some code from bibtex-completion. I will
remove that code, or greatly simplify it, if and when this PR is merged:

tmalsburg/helm-bibtex#367

Also, change 'bibtex-actions-icons' to 'bibtex-actions-symbols'.

Closes #44
@@ -24,7 +24,7 @@

;; A BibTeX backend for completion frameworks

;; There are currently two fronends: helm-bibtex and ivy-bibtex.
;; There are currently two frontends: helm-bibtex and ivy-bibtex.
Copy link
Author

@bdarcus bdarcus Apr 5, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Now three ;-)

Suggested change
;; There are currently two frontends: helm-bibtex and ivy-bibtex.
;; There are currently three frontends: helm-bibtex, ivy-bibtex and bibtex-actions.

@@ -4,7 +4,7 @@
;; Justin Burkett <[email protected]>
;; Maintainer: Titus von der Malsburg <[email protected]>
;; URL: https://github.com/tmalsburg/helm-bibtex
;; Version: 1.0.0
;; Version: 1.1.0
Copy link
Author

@bdarcus bdarcus Apr 5, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually, I'm a little unclear how MELPA versioning works.

If it's based on git tags, then since all these packages use the same repo, doesn't this need to be 2.1?

I don't think that's the case, though; I think their script pulls the version directly from the package header.

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, that's my understanding as well (but I might be wrong).

@tmalsburg
Copy link
Owner

Could you please briefly explain which problem this PR is supposed to address? And if the purpose is to format the entry list in various different ways, can't you just let-bind bibtex-completion-display-formats temporarily and then call the existing functions with that binding? This way you can flexibly change the layout without changing anything in bibtex-completion.

@bdarcus
Copy link
Author

bdarcus commented Apr 8, 2021

... if the purpose is to format the entry list in various different ways,

That is indeed the purpose.

can't you just let-bind bibtex-completion-display-formats temporarily and then call the existing functions with that binding? This way you can flexibly change the layout without changing anything in bibtex-completion.

I hadn't even considered that possibility. I'm kind of learning elisp as I go.

How would that work?

For comparison, here is what my locally-modified let* code looks like, where "main" uses one template and "suffix" uses another:

   (let* ((pdf (if (assoc "=has-pdf=" (cdr candidate)) " has:pdf"))
          (note (if (assoc "=has-note=" (cdr candidate)) "has:note"))
          (link (if (assoc "doi" (cdr candidate)) "has:link"))
          (citekey (bibtex-completion-get-value "=key=" candidate))
          (candidate-main
           (bibtex-actions--format-entry
            candidate
            (1- (frame-width))
            bibtex-actions-template))
          (candidate-suffix
           (bibtex-actions--format-entry
            candidate
            (1- (frame-width))
            bibtex-actions-template-suffix))

@tmalsburg
Copy link
Owner

tmalsburg commented Apr 8, 2021

Temporarily let-binding configuration variables is an elisp technique that I also needed to get used to. Nothing that I was familiar with from other languages. It allows for a lot of flexibility but can also become messy and result in code that's difficult to read and debug.

Specifically, what I was thinking of is something like this:

(let ((bibtex-completion-display-formats (your desired format)))
  (bibtex-completion-format-entry entry width))

With this setup, the entry will be formatted not with the globally configured layout but with the one specified in the let-binding.

@bdarcus
Copy link
Author

bdarcus commented Apr 8, 2021

Specifically, what I was thinking of is something like this:

(let ((bibtex-completion-display-formats (your desired format)))
  (bibtex-completion-format-entry entry width))

So is what you're thinking here to have a kind of format-entry wrapper that does that let-binding? Like:

(defun bibtex-actions--format-entry (entry width template)
  ;; here do the let-binding
  (bibtex-completion-format-entry ...)

@bdarcus
Copy link
Author

bdarcus commented Apr 8, 2021

So is what you're thinking here to have a kind of format-entry wrapper that does that let-binding?

Hmm ... would that even work though, given that now there's bibtex-completion-display-formats-internal in the init function, which assumes a single template?

@tmalsburg
Copy link
Owner

So is what you're thinking here to have a kind of format-entry wrapper that does that let-binding? Like:

To be honest, I'm not sure what exactly you're trying to achieve. I was thinking that you want to render the two halves of the table (left and right) separately. My idea was that you could call bibtex-completion-format-entry two times, once for the left-hand side and once for the right-hand side, and then concatenate the two halves to form the complete table. But perhaps I'm misunderstanding what you'd like to achieve.

@bdarcus
Copy link
Author

bdarcus commented Apr 8, 2021 via email

@tmalsburg
Copy link
Owner

Can't you render the table in one go and then apply font styling for each half each line separately?

@bdarcus
Copy link
Author

bdarcus commented Apr 9, 2021

Can't you render the table in one go and then apply font styling for each half each line separately?

That's another idea I didn't consider.

But wouldn't that be more complicated?

Concatenatng two template strings is simple, of course.

But how do I know where to attach the face (I just have one for the "suffix" section) to the string?

And what happens if one or both templates are alists rather than strings?

@tmalsburg
Copy link
Owner

But how do I know where to attach the face (I just have one for the "suffix" section) to the string?

I'm not super familiar with faces but I would guess that you can assign faces to a ranges of strings. So you'd have to assign one fact to the first n characters and the another to the remainder of the line. Should be too difficult.

And what happens if one or both templates are alists rather than strings?

I'm not sure what you mean. It would help to see some sketch in code.

@bdarcus
Copy link
Author

bdarcus commented Apr 9, 2021

I would guess that you can assign faces to a ranges of strings. So you'd have to assign one fact to the first n characters and the another to the remainder of the line.

IC. So grab n from the "display-formats-internal" variable.

Does it matter more generally (beyond faces) that that n would only represent one template?

I'm not sure what you mean. It would help to see some sketch in code.

It's not a huge deal; I just mean, say the user has this in their config:

(setq bibtex-completion-display-formats
    '((article       . "${year:4} ${author:36} ${title:*} ${journal:40}")
      (inbook        . "${year:4} ${author:36} ${title:*} Chapter ${chapter:32}")
      (incollection  . "${year:4} ${author:36} ${title:*} ${booktitle:40}")
      (inproceedings . "${year:4} ${author:36} ${title:*} ${booktitle:40}")
      (t             . "${year:4} ${author:36} ${title:*}")))

(setq bibtex-actions-suffix-format '((t . "${=has-pdf=:1}${=has-note=:1} ${=type=:7}")))

That would require some logic to pull the right "display-formats" template for the entry to concatenate with the "suffix" template.

On reflection, it's not complicated, but it is additional code to write.

I do have this all working well with my forked format-entry function (which requires the template argument), and the logic is straightforward.

I guess we've established that I have options, though, including just keeping my code as is.

What's your concern about merging this? That you don't see the need, and worry about something breaking?

@tmalsburg
Copy link
Owner

I don't see why bibtex-actions-suffix-format is needed. Can't you simply integrate the suffix into bibtex-completion-display-formats? The idea being that bibtex-completion renders the lines for you and then your own code adds the faces which are specific to your package and not needed in helm/ivy-bibtex.

What's your concern about merging this? That you don't see the need, and worry about something breaking?

Multiple things: I don't see the need, within bibtex-completion at least. It seems more natural to have the code for styling the lines with faces withing your package where it's actually needed. The added complexity will also make it harder to maintain this code base (which is already challenging).

@bdarcus
Copy link
Author

bdarcus commented Apr 9, 2021

Multiple things: I don't see the need, within bibtex-completion at least. It seems more natural to have the code for styling the lines with faces withing your package where it's actually needed. The added complexity will also make it harder to maintain this code base (which is already challenging).

I'm totally fine with you closing this.

My code is licensed same as your's, so if I do something useful you want to borrow at some point, that will be easy to do.

But to offer counterpoints to consider:

  1. Sure, but the formatting function is hard-coded to accept only a single template, which gives third-party front-ends like mine a lot less flexibility, without any obvious advantage.
  2. I'm biased of course, but I think the changes make it easier to understand and maintain; it's actually simpler, I would say. Here, formatting is entirely contained within the formatting function, without any dependence on some global variable that's created by the init function. The init function also become simpler.

But as I say, I won't take it personally if you disagree, and appreciate you having found the time to look into these ideas I've submitted!

@tmalsburg
Copy link
Owner

Sure, but the formatting function is hard-coded to accept only a single template, which gives third-party front-ends like mine a lot less flexibility,

I must be missing something because, as far as I understand, it should be easy to achieve the layout you show in the screenshot with the current facilities (by combining the main and suffix template and then apply faces in a second step). I don't see what is gained by adding the ability to supply multiple template.

it's actually simpler, I would say. Here, formatting is entirely contained within the formatting function, without any dependence on some global variable that's created by the init function. The init function also become simpler.

I agree that the precalculation isn't super elegant but there was a reason to do it that way: If we precaculate the formatting at init, we do it just once. If we calculate the formatting in bibtex-completion-format-entry (as I think this PR does), it will be done for every single entry which would be slower.

@bdarcus
Copy link
Author

bdarcus commented Apr 11, 2021

If we precaculate the formatting at init, we do it just once. If we calculate the formatting in bibtex-completion-format-entry (as I think this PR does), it will be done for every single entry which would be slower.

Yes, that's true.

But OTOH, my benchmarking shows the format-entry function runs a bit faster here (though I assume that might be because of cl-loop rather than this?). I think this was wrong, but see below.

I could always move that back though. (actually, I don't think I can)

@bdarcus
Copy link
Author

bdarcus commented Apr 12, 2021

On this:

it should be easy to achieve the layout you show in the screenshot with the current facilities (by combining the main and suffix template and then apply faces in a second step). I don't see what is gained by adding the ability to supply multiple template.

That may work in this particular circumstance, but:

  1. I don't see why I should be forced into that particular choice. It's easier for me to continue using my forked format-entry function instead at this point.
  2. where I started with this, this approach wouldn't have worked.

So, for example, if someone wants to implement the suffix-like content using affixation and/or annotation (which is what I originally did), you do need multiple templates. That's where this PR really came from, and a number of people I've seen start to do something like what I have done have wanted to use affixation and/or annotation. Either way, you need multiple templates to do that.

But on reflection, even for my case, I still don't think the format-entry function should hard-code a template.

Hence why I still think this is a good idea :-).

@bdarcus
Copy link
Author

bdarcus commented Apr 13, 2021

I could always move that back though. (actually, I don't think I can)

Edit:

Actually, it would be possible. Could just move the processed-template call to a variable on the calling function, and also restore the global internal variable for default template. That would preserve the flexibility, but maintain compatibility (though the format-entry calls probably would probably need adjustment on the front-ends) and performance.

Here's the adjustment on my end: emacs-citar/citar@ce73dbe

Actually, I guess I should have finished my coffee first. There's a dumb bug, fixed here: emacs-citar/citar@419b438.

A quick benchmark shows that change in bibtex-actions--format-candidates function speeds up dramatically.

 (benchmark-run 100 (bibtex-actions--format-candidates))
(5.492027389 29 2.0400067539999998)

... whereas before the change, it was closer to a minute for the run of 100.

@bdarcus
Copy link
Author

bdarcus commented Apr 13, 2021

A potentially bigger problem, though I am unsure how big:

emacs-citar/citar#67

Obviously wouldn't happen if I followed your suggested approach, @tmalsburg, but can you help me understand what would be required to fix that?

Basic issue is the math for format-entry assumes one template too, so that field widths (?) don't work entirely correctly if more than one.

I'm trying to figure out how to implement your suggestion on the linked PR, but stuck at merging the templates, particularly as I contemplate ones where there are multiple entry types. Seems like fixing the above may be easier.

Or I may just give it up entirely, and do away with the distinction between main and suffix.

BTW, what is the purpose of the beginning "t" in this?

((t "${author:36} ${title:*} ${year:4} ${=has-pdf=:1}${=has-note=:1} ${=type=:7}" . 53))

@tmalsburg
Copy link
Owner

... whereas before the change, it was closer to a minute for the run of 100.

Formatting 100 entries should under no circumstances take a minute. On my system and with the current code I can format thousands of entries in a fraction of a second. Am I missing something?

@tmalsburg
Copy link
Owner

tmalsburg commented Apr 14, 2021

Obviously wouldn't happen if I followed your suggested approach, @tmalsburg, but can you help me understand what would be required to fix that?

The lecture period is starting next week and I'm moving to a new university and city (with my family). No time currently, sorry. Things will become a bit more manageable in May I hope.

BTW, what is the purpose of the beginning "t" in this?

You can have separate templates for different types of entries. The template with t is the default template. Here's my own config as an example:

(setq bibtex-completion-display-formats
    '((article       . "${author:30} ${title:*} ${journal:20} ${year:4} ${=has-pdf=:1}${=has-note=:1} ${=type=:3}")
      (inbook        . "${author:30} ${chapter:*} ${title:20} ${year:4} ${=has-pdf=:1}${=has-note=:1} ${=type=:3}")
      (incollection  . "${author:30} ${title:*} ${booktitle:20} ${year:4} ${=has-pdf=:1}${=has-note=:1} ${=type=:3}")
      (inproceedings . "${author:30} ${title:*} ${booktitle:20} ${year:4} ${=has-pdf=:1}${=has-note=:1} ${=type=:3}")
      (phdthesis     . "${author:30} ${title:*} ${school:20} ${year:4} ${=has-pdf=:1}${=has-note=:1} ${=type=:3}")
      (t             . "${author:30} ${title:*} ${year:4} ${=has-pdf=:1}${=has-note=:1} ${=type=:3}")))

@tmalsburg tmalsburg closed this Apr 14, 2021
@bdarcus
Copy link
Author

bdarcus commented Apr 14, 2021 via email

@tmalsburg
Copy link
Owner

But my point was your observation above about the recomputation of the template each format-entry run was definitely on the right track.

Ah, I see. I misinterpreted "the change". Sorry. As I said, I'm a bit overwhelmed currently.

@bdarcus
Copy link
Author

bdarcus commented Apr 24, 2021

In any case, if you were to merge this at some point, it would just require:

  1. change a single line in both helm-bibtex and ivy-bibtex to include the processed template as an argument
  2. remove the processing of that template on the format-entry function
  3. bump the versions of all three packages, and update the bibtex-completion dependency on the front-ends

Bibtex-actions already implements this, so you can always just borrow from that, if you wanted.

@tmalsburg
Copy link
Owner

@bdarcus thank you!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants