From acf8a4287350f16b4ec495f7293627f6998195e4 Mon Sep 17 00:00:00 2001 From: okamsn Date: Mon, 2 Oct 2023 22:27:51 -0400 Subject: [PATCH] Fix use of `macroexpand-all` in `loopy-iter` and `loopy`. We should always be passing an environment to the function, otherwise the environment is reset to nil, which breaks macros like `cl-flet`. - Stop doing top-level and sub-level expansion with different functions. - Instead, add `loopy-iter--level`. - Remove `loopy-iter--sub-level-expanders`, `loopy-iter--top-level-expanders`, `loopy-iter--macroexpand-sub`, `loopy-iter--macroexpand-top`, `loopy-iter--keyword-expander-sub`, and `loopy-iter--keyword-expander-top`. - Pass the macro temporary environment in all places where we didn't before: - `loopy-iter` - `loopy` - `loopy-iter--parse-at-command` - `loopy-iter--opt-accum-expand-val` - Add a `loopy--with-protected-stack` and `loopy--bind-main-body`. - Add tests. --- CHANGELOG.md | 16 + README.org | 2 + doc/loopy.texi | 1127 ++++++++++++++++++++++++------------------- loopy-commands.el | 8 + loopy-iter.el | 330 +++++++------ loopy-misc.el | 1 + loopy.el | 9 +- tests/iter-tests.el | 2 - tests/tests.el | 90 ++++ 9 files changed, 916 insertions(+), 669 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 39de0b30..93659c55 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -48,6 +48,22 @@ This document describes the user-facing changes to Loopy. - Better signal an error with conflicting arguments in `numbers`. See [#172]. +- Fix macro expansion in some cases by not resetting the macro environment. + For example, we failed to pass the current/modified environment to + `macroexpand-all` in some cases. + + ```emacs-lisp + ;; Previously failed to use `cl-flet''s internally defined function + ;; for `10+': + ;; => (11 12 13 14 15) + (loopy (named outer) + (list i '((1 2) (3 4) (5))) + (loopy-iter (listing j i) + (cl-flet ((10+ (y) (+ 10 y))) + (at outer + (collecting (10+ j)))))) + ``` + ### Breaking Changes - Make it an error to re-use iteration variables with multiple iteration diff --git a/README.org b/README.org index 1c230271..950bd981 100644 --- a/README.org +++ b/README.org @@ -56,6 +56,8 @@ please let me know. - The non-keyword arguments of =numbers= are deprecated. Use the keyword arguments instead. A =:test= keyword argument was added, which is more flexible and explicit than the non-keyword arguments. + - Fixed a problem with macro expansion in some cases for sub-macros + that created a new macro environment (e.g., ~cl-flet~). - Versions 0.11.1 and 0.11.2: None. Bug fixes. - Version 0.11.0: - More incorrect destructured bindings now correctly signal an error. diff --git a/doc/loopy.texi b/doc/loopy.texi index cb9916a6..697961f4 100644 --- a/doc/loopy.texi +++ b/doc/loopy.texi @@ -118,7 +118,7 @@ described thoroughly later in this document. @table @asis @item @code{loopy} A macro similar to @code{cl-loop}. Unlike @code{cl-loop}, @code{loopy} uses -parenthesized expressions instead of ``clauses''. +parenthetical forms instead of keyword ``clauses''. @lisp ;; A simple usage of `cl-loop': @@ -161,7 +161,7 @@ by @samp{cl-lib}. @lisp ;; => (1 2 3 (:k1 4) 4) -(loopy-let* (((a b c &rest d &key k1) (list 1 2 3 :k1 4))) +(loopy-let* (((a b c &rest d &key k1) '(1 2 3 :k1 4))) (list a b c d k1)) ;; => ((7 2 8) [9 2 10]) @@ -169,16 +169,20 @@ by @samp{cl-lib}. (my-vector (vector 1 2 3))) (loopy-ref (((a _ c) my-list) ([d _ e] my-vector)) - (setf a 7 c 8 d 9 e 10) + (setf a 7 + c 8 + d 9 + e 10) (list my-list my-vector))) @end lisp @end table Some other things to note are: + @itemize @item -@code{loopy} (and so @code{loopy-iter}) support destructuring for both iteration and +@code{loopy} (and so @code{loopy-iter}) supports destructuring for both iteration and accumulation commands. @lisp @@ -199,7 +203,7 @@ accumulation commands. @end lisp @item -The looping macros are configurable and extensible. One can create their own +The looping macros are configurable and extensible. One can create one's own loop commands and command aliases. ``Flags'' can be used to configure the looping macros' behavior, such as by using an alternative destructuring system like @samp{pcase} or the Dash library. @@ -216,18 +220,15 @@ like @samp{pcase} or the Dash library. (collect cdrs cdr)) (finally-return digits cars cdrs)) @end lisp -@end itemize - -@itemize @item Compared to @code{cl-loop}, more constructs are provided for how loops are -completed and values returned. For example, the @samp{leave} command will exit the -loop without changing what would be returned. This is a more generic form of -@samp{while} and @samp{until}, though those are also provided. @samp{after-do} -(a.k.a. @samp{else-do}) is a construct that runs Lisp code only when the loop -completes successfully, similar to Python's @code{else} statement after @code{for} and -@code{while} loops. +completed and how values are returned. For example, the @samp{leave} command will +exit the loop without changing what would be returned. This is a more generic +form of the commands @samp{while} and @samp{until}, though those are also provided. +@samp{after-do} (a.k.a. @samp{else-do}) is a construct that runs Lisp code only when the +loop completes successfully, similar to Python's @code{else} statement after @code{for} +and @code{while} loops. @lisp ;; => (1 3 5) @@ -247,22 +248,34 @@ completes successfully, similar to Python's @code{else} statement after @code{fo @end lisp @item -One can declare which accumulations should be optimized (and so treated as -implied variables). This can make accumulation into multiple named variables -much faster. +One can declare which accumulations variables should be optimized (and so +treated as user-inaccessible during the loop). This can make accumulation +into multiple named variables much faster. As needed, can make Loopy optimize +for at-start accumulations or at-end accumulations. @lisp ;; Expands into the efficient `push'-`nreverse' idiom, not ;; the `nonc'-`list' idiom that would be used by `cl-loop'. +;; => ((1 3) (2 4)) (loopy (accum-opt cars cdrs) (array elem [(1 . 2) (3 . 4)]) (collect (cars . cdrs) elem) (finally-return cars cdrs)) + +;; Prioritizes collecting at the start of `my-var'. +;; => (5 3 1 4 6) +(loopy (accum-opt (my-var start)) + (array (car . cdr) [(1 . 2) (3 . 4) (5 . 6)]) + (collect my-var car :at start) + (when (> cdr 2) + (collect my-var cdr :at end)) + (finally-return my-var)) @end lisp @end itemize -That being said, Loopy is not yet feature complete. Please request features or -report problems in this project’s @uref{https://github.com/okamsn/loopy/issues, issues tracker}. + +All that being said, Loopy is not yet feature complete. Please request features +or report problems in this project’s @uref{https://github.com/okamsn/loopy/issues, issues tracker}. @node Basic Concepts @chapter Basic Concepts @@ -281,9 +294,9 @@ opposed to ``loop commands'' (@ref{Special Macro Arguments}). command @samp{list} in the expression @samp{(list i '(1 2 3))}. A command inserts code into the loop body, but can also perform additional setup like initializing variables. Many commands set a condition for ending the loop. In the case of -@samp{list}, the command iterates through the elements of a list, binding the -variable @code{i} to each element. After iterating through all elements, the loop is -forced to end. +@samp{list} in the above expression, the command iterates through the elements of a +list, binding the variable @code{i} to each element. After iterating through all +elements, the loop is forced to end. In general, a loop ends when any looping condition required by a loop command evaluates to @code{nil}. If no conditions are needed, then the loop runs infinitely @@ -294,9 +307,9 @@ stated explicitly, as in one of the early-exit commands or part of the @samp{finally-return} macro argument, or come from accumulating loop commands using an implied accumulation variable (@ref{Accumulation, , Accumulation Commands}). -The @code{loopy} macro is configurable. One can add custom commands -(@ref{Custom Commands}), add custom command aliases (@ref{Custom Aliases}), and specify -macro options for a particular loop (@ref{Using Flags}). Each of these ideas is +The @code{loopy} macro is configurable. One can add custom commands (@ref{Custom Commands, , Custom +Commands}), add custom command aliases (@ref{Custom Aliases}), and specify macro +options for a particular loop (@ref{Using Flags}). Each of these features is explained in detail later in this document. @node Special Macro Arguments @@ -305,7 +318,7 @@ explained in detail later in this document. @cindex special macro argument There are only a few special macro arguments. If a macro argument does not match one of these special few, @code{loopy} will attempt to interpret it as a loop -command, and signal an error if that fails. +command, and will signal an error if that fails. These special macro arguments are always processed before loop commands, regardless of the order of the arguments passed to @code{loopy}. @@ -373,12 +386,14 @@ in a @code{let}-like form, but that isn’t always desired. ;; Without `without', `loopy' would try to initialize `a' to nil, which would ;; overwrite the value of 5 above. +;; => (5 4 3 2 1) (let ((a 5)) (loopy (without a) ; Don't initialize `a'. (until (zerop a)) ; Leave loop when `a' equals 0. (collect a) ; Collect the value of `a' into a list. (set a (1- a)))) ; Set `a' to the value of `(1- a)'. +;; => (5 4 3 2 1) (let ((a 5)) (loopy (no-init a) (while (not (zerop a))) @@ -397,7 +412,7 @@ Run Lisp expressions before the loop starts, after variables are initialized. @lisp -;; = > (6 7 8) +;; => (6 7 8) (loopy (with (a 1) (b 2)) ; Set `a' to 1 and `b' to 2. (before-do (cl-incf a) ; Add 1 to `a'. (cl-incf b)) ; Add 1 to `b'. @@ -419,9 +434,10 @@ before the loop starts, after variables are initialized. @findex else @table @asis @item @samp{after-do}, @samp{after}, @samp{else-do}, @samp{else} -Run Lisp expressions after the -loop successfully completes. This is similar to Python’s @code{else} statement -following a @code{for} or @code{while} loop. +Run Lisp expressions after the loop +successfully completes. This is similar to Python’s @code{else} statement +following a @code{for} or @code{while} loop. Unlike @code{progn}, the return values of the +expressions do not affect the return value of the macro. @lisp ;; Messages that no odd number was found: @@ -451,7 +467,8 @@ following a @code{for} or @code{while} loop. @table @asis @item @samp{finally-do}, @samp{finally} Run Lisp expressions after the loop exits, always. -These expressions do not affect the final return value of the loop. +Unlike @code{progn}, the return values of the expressions do not affect the +return value of the macro. @lisp ;; => (nil finally) @@ -682,13 +699,13 @@ and this is not: @end lisp Trying to use loop commands in places where they don't belong will result in -errors when the code is evaluated. +errors while the macro is expanding and when the code is evaluated. You should keep in mind that commands are evaluated in order. This means that -attempting to do something like the below example might not do what you expect, -as @samp{i} is assigned a value from the list after collecting @samp{i} into @samp{coll}. +attempting something like the below example might not do what you expect, as @samp{i} +is assigned a value from the list after collecting @samp{i} into @samp{coll}. -@float Listing,org2573569 +@float Listing,orga2719da @lisp ;; => (nil 1 2) (loopy (collect coll i) @@ -706,8 +723,9 @@ command can be used to iterate through the elements of an array or string@footno for more.}. You can define custom aliases using the macro @code{loopy-defalias} (@ref{Custom Aliases}). -Similar to other libraries, many commands have an alias of the present -participle form (the ``-ing'' form). A few examples are seen in the table below. +Similar to other libraries, many commands have an alias of the +present-participle form (the ``-ing'' form). A few examples are seen in the table +below. @multitable {aaaaaaaaa} {aaaaaaaaaaaa} @headitem Command @@ -799,7 +817,7 @@ for using Loopy's destructuring outside of @code{loopy} loops (@ref{Destructurin This section describes the basic built-in destructuring used by most loop commands, such as @samp{set} and @samp{list}. Destructuring in accumulation commands and -sequence reference commands works slightly differently, and is described more in +sequence-reference commands works slightly differently, and is described more in those sections. The last thing to note is that @code{loopy} loops can be made to use alternative @@ -810,7 +828,7 @@ the flag @samp{dash} provided by the package @samp{loopy-dash}. Below are two examples of destructuring in @code{cl-loop} and @code{loopy}. -@float Listing,orgb4c5e86 +@float Listing,org54c0d3a @lisp ;; => (1 2 3 4) (cl-loop for (i . j) in '((1 . 2) (3 . 4)) @@ -825,7 +843,7 @@ Below are two examples of destructuring in @code{cl-loop} and @code{loopy}. @caption{Destructuring values in a list.} @end float -@float Listing,orgbf178dd +@float Listing,orgdcbd07b @lisp ;; => (1 2 3 4) (cl-loop for elem in '((1 . 2) (3 . 4)) @@ -872,17 +890,18 @@ syntax for sequences. @end lisp @item -The symbol @samp{_}: The symbol @samp{_} (an underscore) means to avoid creating a +The symbol @samp{_} (an underscore): The symbol @samp{_} means to avoid creating a variable. This can be more efficient. @lisp -;; Only create the variables `a' and `c'. -;; => ((1 3) (4 6)) -(loopy (list (a _ c) '((1 2 3) (4 5 6))) - (collect (list a c))) +;; Only creates the variables `a' and `d': +;; => ((1 4) (5 8)) +(loopy (list (a _ _ d) '((1 2 3 4) (5 6 7 8))) + (collect (list a d))) ;; These two destructurings do the same thing, ;; and only bind the variable `a': +;; ;; => (1 3) (loopy (array (a) [(1 2) (3 4)]) (collect a)) @@ -902,6 +921,7 @@ This is the same as when used in a CL @code{lambda} list. @lisp ;; See that the variable `both' holds the value of the entire ;; list element: +;; ;; => (((1 2) 1 2) ;; ((3 4) 3 4)) (loopy (list (&whole both i j) '((1 2) (3 4))) @@ -915,9 +935,9 @@ This is the same as when used in a CL @code{lambda} list. @item The symbol @samp{&rest}: A variable named after @samp{&rest} contains the remaining elements of the destructured value. When destructuring lists, one can also -instead use dotted notation. These variables can themselves be sequences. +use dotted notation. These variables can themselves be sequences. -This is the same as when used in @code{seq-let}. +This @samp{&rest} is the same as when used in @code{seq-let}. @lisp ;; => ((1 [2 3]) (4 [5 6])) @@ -929,19 +949,24 @@ This is the same as when used in @code{seq-let}. (collect (list i j k))) ;; => ((1 (2 3)) (4 (5 6))) -(loopy (list (i . j) '((1 2 3) (4 5 6))) - (collect (list i j))) - -;; Works the same as above: (loopy (list (i &rest j) '((1 2 3) (4 5 6))) (collect (list i j))) -;; The above using `seq-let': -(let ((result)) - (dolist (elem '((1 2 3) (4 5 6))) - (seq-let [i &rest j] elem - (push (list i j) result))) - (reverse result)) +;; => ((1 2 3) (4 5 6)) +(loopy (list (i &rest (j k)) '((1 2 3) (4 5 6))) + (collect (list i j k))) + +;; => ((1 2 3) (4 5 6)) +(loopy (list (i . (j k)) '((1 2 3) (4 5 6))) + (collect (list i j k))) + +;; => ((1 2 3) (4 5 6)) +(loopy (list (i &rest [j k]) '((1 . [2 3]) (4 . [5 6]))) + (collect (list i j k))) + +;; => ((1 2 3) (4 5 6)) +(loopy (list (i . [j k]) '((1 . [2 3]) (4 . [5 6]))) + (collect (list i j))) @end lisp @item @@ -964,7 +989,8 @@ provided, then keys are sought using @code{plist-member}. That way, a value of @code{nil} for a key is not the same as a missing key. @lisp -;; Note that `nil' is not the same as a missing value. +;; Note that `nil' is not the same as a missing value: +;; ;; => ((1 2 nil 25) (4 5 24 25)) (loopy (list (&key a b (c 24) (missing 25)) '((:b 2 :c nil :a 1) (:a 4 :b 5))) @@ -977,11 +1003,13 @@ used. @lisp ;; Keys are only sought after positional variables: +;; ;; => ((1 2 :k1 'ignored 3)) (loopy (array (a b c d &key k1) [(1 2 :k1 'ignored :k1 3)]) (collect (list a b c d k1))) ;; If `&rest' is used, keys are sought only in that variable: +;; ;; => ((1 (:k1 3) 3)) (loopy (array (a &rest b &key k1) [(1 :k1 3)]) (collect (list a b k1))) @@ -1006,6 +1034,10 @@ the dot in dotted lists. @node Generic Evaluation @section Generic Evaluation +These generic commands are for settings values and running sub-commands or +sub-expressions during the loop. These commands do not affect macro's return +value and do no affect how the loop iterates. + @findex do @table @asis @item @samp{(do EXPRS)} @@ -1024,49 +1056,6 @@ To use loopy commands in arbitrary code, use the macro @code{loopy-iter} instead @end lisp @end table -@findex set -@findex setting -@findex expr -@findex exprs -@table @asis -@item @samp{(set|expr VAR [EXPRS])} -Bind @samp{VAR} to each @samp{EXPR} in order. -Once the last @samp{EXPR} is reached, it is used repeatedly for the rest of the -loop. With no @samp{EXPR}, @samp{VAR} is repeatedly bound to @code{nil}. - -This command also has the aliases @samp{setting} and @samp{exprs}. - -@quotation Note -@samp{set} does @emph{not} behave the same as @code{setq} in all situations. - -While @samp{set} can take multiple arguments, it only assigns the value of one -expression to one variable during each iteration of the loop (unless using -destructuring). It does not take pairs of variables and values in the same -way that @code{setq} does. - -Furthermore, variables assigned by @samp{set} (and other commands) are by -default @code{let}-bound around the loop and generally initialized to @code{nil}. -This means that doing @samp{(set VAR EXPR)} will not, by default, affect -variables outside of the loop in the same way that using @samp{(do (setq VAR - EXPR))} would. - -@end quotation - -@lisp -;; => '(1 2 3 3 3) -(loopy (cycle 5) - (set i 1 2 3) - (collect coll i) - (finally-return coll)) - -;; => '(0 1 2 3 4) -(loopy (cycle 5) - (set i 0 (1+ i)) - (collect coll i) - (finally-return coll)) -@end lisp -@end table - @findex group @findex command-do @table @asis @@ -1100,6 +1089,39 @@ expressions. Currently, this command is only useful when used within the @end lisp @end table +@findex set +@findex setting +@findex expr +@findex exprs +@table @asis +@item @samp{(set|expr VAR [EXPRS])} +Bind @samp{VAR} to each @samp{EXPR} in order. Once the last +@samp{EXPR} is reached, it is used repeatedly for the rest of the loop. With no +@samp{EXPR}, @samp{VAR} is bound to @code{nil} during each iteration of the loop. + +This command also has the aliases @samp{setting} and @samp{exprs}. + +Unlike the Emacs Lisp function @code{set}, the variable name should not be quoted. +Unlike the Emacs Lisp special form @code{setq}, the command @samp{set} only sets one +variable, and this variable is by default @code{let}-bound around the loop. To +stop @samp{VAR} from being @code{let}-bound around the loop, use the special macro +argument @samp{without} (@ref{Special Macro Arguments}). + +@lisp +;; => '(1 2 3 3 3) +(loopy (cycle 5) + (set i 1 2 3) + (collect coll i) + (finally-return coll)) + +;; => '(0 1 2 3 4) +(loopy (cycle 5) + (set i 0 (1+ i)) + (collect coll i) + (finally-return coll)) +@end lisp +@end table + @findex set-prev @findex setting-prev @findex prev-set @@ -1108,8 +1130,10 @@ expressions. Currently, this command is only useful when used within the @table @asis @item @samp{(set-prev|prev-expr VAR VAL &key back)} Bind @samp{VAR} to a value @samp{VAL} from a -previous cycle in the loop. With @samp{BACK}, use the value from that many cycles -previous. This command @emph{does not} work like a queue. +previous cycle in the loop. With @samp{BACK} (default: 1), use the value from that +many cycles previous. This command @emph{does not} work like a queue; it always +uses the value from the @samp{BACK}-th previous cycle, regardless of when the +command is run. This command also has the aliases @samp{setting-prev}, @samp{prev-set}, and @samp{prev}. @@ -1151,20 +1175,19 @@ can be exited by using early-exit commands (@ref{Early Exit}) or boolean commands (@ref{Checking Conditions}). Iteration commands must occur in the top level of the @code{loopy} form or in a -sub-loop command (@ref{Sub-Loops}). Trying to do something like the below will -signal an error. +sub-loop command (@ref{Sub-Loops}). Using them elsewhere and trying to do something +like the below example will signal an error. @lisp ;; Signals an error: (loopy (list i '(1 2 3 4 5)) (when (cl-evenp i) - ;; Can't use `list' in a `when'. + ;; Can't use `list' inside `when'. ;; Will signal an error. (list j '(6 7 8 9 10)) (collect j))) @end lisp - In @code{loopy}, iteration commands are named after what they iterate through. For example, the @samp{array} and @samp{list} commands iterate through the elements of arrays and lists, respectively. @@ -1177,6 +1200,16 @@ argument, but this can result in less efficient code. @end quotation +Because some iteration commands use their variable to manage state, it is an +error to use the same iteration variable for multiple iteration commands. + +@lisp +;; Signals an error due to the re-use of `i': +(loopy (numbers i :from 1 :to 10) + (list i '(1 2 3)) + (finally-return t)) +@end lisp + @menu * Generic Iteration:: Looping a certain number of times. * Numeric Iteration:: Iterating through numbers. @@ -1249,18 +1282,6 @@ and to update @samp{VAR} during the loop. This command also has the name @samp{iterating}. -@quotation Warning -The loop ends when the iterator finishes, which must be checked before the -code in loop body is run. To check whether a iterator is finished, @code{loopy} -checks whether it signals an error when trying to yield a value. - -Because values are yielded before the next iteration step of the loop, trying -to yield more values from the iterator after the loop ends will result in lost -values. One option for working around this is to use the generic command -@samp{set} with the function @code{iter-next} directly. - -@end quotation - @lisp ;; With var: ;; => ((1 . 4) (2 . 5) (3 . 6)) @@ -1273,15 +1294,54 @@ values. One option for working around this is to use the generic command ;; Without var: ;; => (1 2 3) -(loopy (with (iter-maker (iter-lambda () - ;; These yielded values are all ignored. - (iter-yield 'first-yield) - (iter-yield 'second-yield) - (iter-yield 'third-yield)))) - (iter (funcall iter-maker)) +(loopy (iter (funcall (iter-lambda () + ;; These yielded values are all ignored. + (iter-yield 'first-yield) + (iter-yield 'second-yield) + (iter-yield 'third-yield)))) (set i 1 (1+ i)) (collect i)) @end lisp + +@quotation Warning +Be aware that values are yielded from the iterator before running the loop +body. When the iterator can no longer yield values, it is finished. + +Because values are yielded before the next iteration step of the loop, trying +to yield more values from the iterator after the loop ends will result in lost +values. One option for working around this is to use the generic command +@samp{set} with the function @code{iter-next} directly. + +@end quotation + +@lisp +;; => 5, not 4 as one might expect. +(loopy (with (iter-obj (funcall (iter-lambda () + (let ((i 0)) + (while t + (iter-yield (cl-incf i)))))))) + (iter iter-obj :close nil) + (cycle 3) + (finally-return (prog1 (iter-next iter-obj) + (iter-close iter-obj)))) + +;; Avoiding missed yielded values: +;; +;; => ((1 2 3) 4) +(loopy (with (iter-obj (funcall (iter-lambda () + (let ((i 0)) + (while t + (iter-yield (cl-incf i))))))) + (j nil)) + (cycle 3) + (set j (condition-case nil + (iter-next iter-obj) + (iter-end-of-sequence nil))) + (collect j) + (finally-return (prog1 + (list loopy-result (iter-next iter-obj)) + (iter-close iter-obj)))) +@end lisp @end table @node Numeric Iteration @@ -1296,24 +1356,25 @@ variants @samp{numbers-up} and @samp{numbers-down}. @findex numbering @findex numbers @table @asis -@item @samp{(numbers|nums VAR [START [END [STEP]]] &key KEYS)} -Iterate -through numbers. @samp{KEYS} is one or several of @samp{from}, @samp{upfrom}, @samp{downfrom}, -@samp{to}, @samp{upto}, @samp{downto}, @samp{above}, @samp{below}, and @samp{by}. +@item @samp{(numbers|nums VAR &key KEYS)} +Iterate through numbers. @samp{KEYS} is one or +several of @samp{from}, @samp{upfrom}, @samp{downfrom}, @samp{to}, @samp{upto}, @samp{downto}, @samp{above}, +@samp{below}, @samp{by}, and @samp{test}. This command also has the aliases @samp{num}, @samp{number}, and @samp{numbering}. The command @samp{numbers} is used to iterate through numbers. For example, -@samp{(numbers i :from 1 :to 10)} is similar to @samp{(list i (number-sequence 1 10))}, -and @samp{(numbers i 3)} is similar to @samp{(set i 3 (1+ i))}. +@samp{(numbers i :from 1 :to 10)} is similar to the command +@samp{(list i (number-sequence 1 10))}, and @samp{(numbers i 3)} is similar to +@samp{(set i 3 (1+ i))}. -For efficiency, @samp{VAR} is initialized to the starting numeric value, not @code{nil}, -and is updated at the end of each step of the loop. This can be overridden -using the @samp{with} special macro argument, which can result in slower code. +For efficiency, @samp{VAR} is initialized to the starting numeric value, not +@code{nil}, and is updated at the end of each step of the loop. This can be +overridden using the @samp{with} special macro argument, which can result in slower +code. -To balance convenience and similarity to other commands, @samp{numbers} has a -flexible argument list. In its most basic form, it uses no keywords and takes -a starting value and an ending value. The ending value is inclusive. +In its most basic form, @samp{numbers} iterates from a starting value to an +inclusive ending value using the @samp{:from} and @samp{:to} keywords, respectively. @lisp ;; => (1 2 3 4 5) @@ -1331,11 +1392,10 @@ end. (collect i)) @end lisp -To specify the step size, one can use an optional third argument (like in -Python's @code{range}) or the keyword @samp{:by} (like in @code{cl-loop}). The value of the -optional third argument can be positive or negative. @emph{However}, in keeping -with @code{cl-loop}, the value for @samp{:by} should always be positive; other keyword -arguments then control whether the variable is incremented or decremented. +To specify the step size, one can use the keyword @samp{:by}. Except when @samp{:test} +is given, the value for @samp{:by} must be positive. Other keyword arguments +(@samp{:upfrom}, @samp{:downfrom}, @samp{:upto}, @samp{:downto}, @samp{:above}, and @samp{:below}) control +whether the variable is incremented or decremented. @lisp ;; => (1 3 5) @@ -1358,9 +1418,11 @@ arguments then control whether the variable is incremented or decremented. By default, the variable's value starts at 0 and increases by 1. To specify whether the value should be increasing or decreasing when using the @samp{:by} -keyword, one can use the keywords @samp{:downfrom}, @samp{:downto}, @samp{:upfrom}, and -@samp{:upto}. The keywords @samp{:from} and @samp{:to} don't by themselves specify a -direction. +keyword, one can use the keywords @samp{:downfrom}, @samp{:downto}, @samp{:upfrom}, @samp{:upto}, +@samp{:above}, and @samp{:below}. The keywords @samp{:from} and @samp{:to} don't by themselves +specify a direction, and they can be used with the keyword arguments that do +without conflict. Using arguments that contradict one another will signal an +error. @lisp ;; => (3 2 1) @@ -1380,14 +1442,13 @@ direction. (loopy (numbers i :from 10 :downto 2 :by 2) (collect i)) -;; Produced code is not as efficient as above: -;; => (10 8 6 4 2) -(loopy (numbers i :from 10 :to 2 -2) - (collect i)) - ;; => (1 2 3 4 5 6 7) (loopy (numbers i :from 1 :upto 7) (collect i)) + +;; => Signals an error: +(loopy (numbers i :downfrom 10 :upto 20) + (collect i)) @end lisp To specify an @emph{exclusive} ending value, use the keywords @samp{:below} for @@ -1398,7 +1459,7 @@ increasing values and @samp{:above} for decreasing values. (loopy (numbers i :from 1 :below 10) (collect i)) -;; Same as +;; Same as above: (loopy (set i 1 (1+ i)) (while (< i 10)) (collect i)) @@ -1412,13 +1473,40 @@ increasing values and @samp{:above} for decreasing values. (collect i)) @end lisp -@quotation Note -Because the @code{loopy} macro can't test the value of the step size ahead of time, -being more explicit by using the keyword parameters can produce faster code. +If you do not know whether you will be incrementing or decrementing, you can +use the keyword argument @samp{test}, whose value is a function that should return +a non-nil value if the loop should continue, such as @code{#'<=}. The function +receives @samp{VAR} as the first argument and the final value as the second +argument, as in @code{(funcall TEST VAR FINAL-VAL)}. @samp{test} can only be used with +@samp{from} and @samp{to}; it cannot be used with keywords that already describe a +direction and ending condition. To match the behavior of @code{cl-loop}, the +default testing function is @code{#'<=}. When @samp{test} is given, @samp{by} can be +negative. -@end quotation -@end table +@lisp +;; => (10 9.5 9.0 8.5 8.0 7.5 7.0 6.5 6.0 5.5) +(loopy (with (start 10) + (end 5) + (func #'>) + (step -0.5)) + (numbers i :to end :from start :by step :test func) + (collect i)) +;; Expands to similar code as above. +;; Note that with `:above', step must be positive. +;; +;; => (10 9.5 9.0 8.5 8.0 7.5 7.0 6.5 6.0 5.5) +(loopy (with (start 10) + (end 5) + (step 0.5)) + (numbers i :from start :above end :by step) + (collect i)) + +;; Signals an error because `:upto' implies a testing function already: +(loopy (numbers i :from 1 :upto 10 :test #'<) + (collect i)) +@end lisp +@end table If you prefer using positional arguments to keyword arguments, you can use the commands @samp{numbers-up} and @samp{numbers-down} to specify directions. These commands @@ -1547,7 +1635,8 @@ This command also has the aliases @samp{arraying} and @samp{stringing}. @samp{KEYS} is one or several of @samp{from}, @samp{upfrom}, @samp{downfrom}, @samp{to}, @samp{upto}, @samp{downto}, @samp{above}, @samp{below}, @samp{by}, and @samp{index}. @samp{index} names the variable -used to store the index being accessed. For others, see the @samp{numbers} command. +used to store the index being accessed. For others, see the @samp{numbers} +command. If multiple arrays are given, then the elements of these arrays are distributed into an array of lists. In that case, the above keywords apply to @@ -1611,7 +1700,7 @@ This command also has the alias @samp{consing}. @table @asis @item @samp{(list|each VAR EXPR [EXPRS] &key by)} Loop through each element of the -list @samp{EXPR}. Optionally, update the list by @samp{by} instead of @samp{cdr}. +list @samp{EXPR}. Optionally, update the list using @samp{by} instead of @samp{cdr}. This command also has the alias @samp{listing}. @@ -1700,7 +1789,7 @@ command can have a noticeable start-up cost when working with very large maps. (finally-return keys values))) @end lisp -Depending on how a map is created, a map might repeat a key multiple times. +Depending on how a map is created, a map might contain a key multiple times. Currently, the function @code{map-pairs} returns such keys. By default, the @code{loopy} command @samp{map-pairs} ignores such duplicate keys. This is for two reasons: @@ -1746,16 +1835,24 @@ Again, this can be disabled by setting @samp{unique} to nil. @table @asis @item @samp{(sequence|seq VAR EXPR [EXPRS] &key KEYS)} Loop through the sequence -@samp{EXPR}, binding @samp{VAR} to the elements of the sequence (see @ref{Sequences Arrays Vectors,elisp#Sequences -Arrays Vectors,,elisp,}). This is a more generic form of the commands @samp{list} and -@samp{array}, though it is somewhat less efficient. These sequences should not be -confused with those generic sequences as understood by the library @samp{seq.el}. +@samp{EXPR}, binding @samp{VAR} to the elements of the sequence (a list or an array). +Because it is more generic, @samp{sequence} is somewhat less efficient than the +@samp{list} and @samp{array} commands. + +@quotation Note +For more on sequences, see @ref{Sequences Arrays Vectors,,,elisp,}. This command +works with the basic sequences understood by the Emacs Lisp functions @code{length} +and @code{elt}. It does not work with the generic sequences understood by the +library @samp{seq.el}. + +@end quotation This command also has the aliases @samp{seqing} and @samp{sequencing}. @samp{KEYS} is one or several of @samp{from}, @samp{upfrom}, @samp{downfrom}, @samp{to}, @samp{upto}, @samp{downto}, @samp{above}, @samp{below}, @samp{by}, and @samp{index}. @samp{index} names the variable -used to store the index being accessed. For others, see the @samp{numbers} command. +used to store the index being accessed. For others, see the @samp{numbers} +command. If multiple sequences are given, then these keyword arguments apply to the resulting sequence of distributed elements. @@ -2022,8 +2119,9 @@ map @samp{EXPR}, binding @samp{VAR} as a @code{setf}-able place. Like the comma this command uses the @samp{map.el} library. @samp{key} is a variable in which to store the current key for the @code{setf}-able -place referred to by @samp{VAR}. This is similar to the @samp{index} keyword -parameter of other commands. +place referred to by @samp{VAR}. This is similar to the @samp{index} keyword parameter +of other commands. This is not the same as the @samp{key} keyword parameter of the +accumulation commands. Like in the command @samp{map}, the keys of the map are generated before the loop is run, which can be expensive for large maps. @@ -2113,8 +2211,8 @@ Accumulation commands are used to accumulate or aggregate values into a variable, such as creating a list of values or summing the elements in a sequence. -If needed, you can refer to the same accumulation variable in multiple -accumulation commands. +Unlike iteration commands, you can refer to the same accumulation variable in +multiple accumulation commands if needed. @lisp ;; => (1 6 2 7 3 8) @@ -2130,11 +2228,13 @@ accumulation commands. This restriction allows accumulations to be much faster. @end quotation +@cindex accumulation initial values Like with other loop commands, variables created by accumulation commands (such -as @samp{coll} in the above example) are initialized to @code{nil} unless stated +as @code{coll} in the above example) are initialized to @code{nil} unless stated otherwise. When otherwise, such as for the commands @samp{sum} and @samp{multiply}, the initial value of a variable depends on the first accumulation command using that -variable in the arguments given to the macro. +variable in the arguments given to the macro. Remember that a variable's +initial value can be controlled using the @samp{with} special macro argument. @lisp ;; => 27 @@ -2144,10 +2244,19 @@ variable in the arguments given to the macro. (finally-return my-accum)) ;; => 21 -(loopy (numbers i 1 3) +(loopy (numbers i :from 1 :to 3) (multiply my-accum i) ; Starts at 1. (sum my-accum i) (finally-return my-accum)) + +;; Using `with': +;; +;; => 87 +(loopy (with (my-accum 10)) + (numbers i :from 1 :to 3) + (sum my-accum i) ; Starts at 0. + (multiply my-accum i) + (finally-return my-accum)) @end lisp @cindex accumulation destructuring @@ -2174,7 +2283,6 @@ accumulated, instead of the destructured value. (finally-return sum1 sum2 sum3)) @end lisp - @cindex implied accumulation results Like in @code{cl-loop}, you do not need to supply a variable name to accumulation commands. If no accumulation variable is given, then the accumulated value is @@ -2191,10 +2299,9 @@ be overridden by using the the @samp{return} and @samp{return-from} loop command @end lisp @vindex loopy-result -Unlike in @code{cl-loop}, this implied return value is bound to the variable -@code{loopy-result} after the loop completes, even when the loop is left early. This -variable can be used in the @samp{after-do}, @samp{finally-do}, and @samp{finally-return} -special macro arguments. +Unlike @code{cl-loop}, Loopy uses a default accumulation variable, which is named +@code{loop-result}. This variable can be used in the @samp{after-do}, @samp{finally-do}, and +@samp{finally-return} special macro arguments. @lisp ;; => (0 1 2 3 4 5) @@ -2220,6 +2327,7 @@ future. @lisp ;; See how the variable `my-explicit-variable' is ignored when ;; returning a final value: +;; ;; => (1 2 3) (loopy (list i '(1 2 3)) (collect i) @@ -2227,32 +2335,33 @@ future. @end lisp Therefore, when mixing implicit and explicit accumulation variables, you must -use the @samp{finally-return} special macro argument to return all of the +always use the @samp{finally-return} special macro argument to return all of the accumulation results. @lisp -;; => ((1 2 3) ; loopy-result -;; (2 4 6) ; my-other-collection -;; (1 2 3) ; car-coll -;; (2 4 6)) ; cdr-coll +;; => ((1 2 3) ; `loopy-result' +;; (2 4 6) ; `my-other-collection' +;; (1 2 3) ; `car-coll' +;; (2 4 6)) ; `cdr-coll' (loopy (list i '(1 2 3)) - (collect i) + (collect i) ; Uses `loopy-result' (set j (* 2 i)) (collect my-other-collection j) (collect (car-coll . cdr-coll) (cons i j)) (finally-return loopy-result my-other-collection - car-coll cdr-coll)) + car-coll + cdr-coll)) @end lisp @cindex accumulation compatibility Like in @code{cl-loop}, when using implied variables, multiple accumulation commands will use the same variable (@code{loopy-result}). For all accumulation variables used by multiple accumulation commands, you should make sure that the commands -are actually compatible. If not, then @code{loopy} will raise an error. +are actually compatible. If not, then @code{loopy} will signal an error. For example, you should not try to accumulate @samp{collect} results and @samp{sum} -results into the same variable, as one cannot use a list as a number. On the +results into the same variable, as you cannot use a list as a number. On the other hand, @samp{sum} and @samp{multiply} are compatible, since they both act on numbers. @lisp @@ -2269,7 +2378,6 @@ other hand, @samp{sum} and @samp{multiply} are compatible, since they both act o (multiply i)) @end lisp - By default, one must specify separate accumulation variables to be able to accumulate into separate values. This can make accumulation slower, because @code{loopy} ensures that named accumulation variables (excluding the previously @@ -2285,6 +2393,7 @@ is more complex and uses a slower way of building the accumulated list. @lisp ;; Optimized accumulation: +;; ;; => (1 3 2 6 3 9) (loopy (accum-opt coll) (numbers i :from 1 :to 3) @@ -2293,6 +2402,8 @@ is more complex and uses a slower way of building the accumulated list. (finally-return coll)) ;; Optimized example expansion: +;; +;; => (1 3 2 6 3 9) (let* ((coll nil) (i 1) (nums-end192 3) @@ -2306,9 +2417,9 @@ is more complex and uses a slower way of building the accumulated list. coll) @end lisp - @lisp ;; Unoptimized accumulation: +;; ;; => (1 3 2 6 3 9) (loopy (numbers i :from 1 :to 3) (collect coll i) @@ -2316,6 +2427,8 @@ is more complex and uses a slower way of building the accumulated list. (finally-return coll)) ;; Unoptimized example expansion: +;; +;; => (1 3 2 6 3 9) (let* ((coll nil) (coll-last-link-190 coll) (i 1) @@ -2411,7 +2524,7 @@ any command definition. @item @samp{test} A function of two arguments, usually used to test for equality. Most tests default to @code{equal}, like in other Emacs Lisp libraries. This is -different from @code{cl-lib}, which mimics Common Lisp and prefers using @code{eql}. +different from @samp{cl-lib}, which mimics Common Lisp and prefers using @code{eql}. @end table @cindex accumulation keyword key @@ -2435,41 +2548,80 @@ knows how to expand efficiently for either case. @node Generic Accumulation @subsection Generic Accumulation -Generic accumulation commands accumulate the output of functions that receive -the accumulation variable. They are very similar to updating a variable's value -using the @samp{set} command. +Generic accumulation commands are more explicit uses of the accumulation +variable. They are very similar to updating a variable's value +using the @samp{set} command and exist for situation not covered by the other +accumulation commands. -@findex set-accum -@findex setting-accum +@itemize +@item +@samp{reduce} is like @code{cl-reduce}, calling a function that receives (1) the +accumulation variable and (2) the value to accumulate, in that order. +@item +@samp{accumulate} works by calling a function that receives (1) the value to +accumulate and (2) the accumulation variable, in that order. +@item +@samp{set-accum} is the most generic, and works like @samp{set} for only one value. +@end itemize + + +The commands are described in more detail below. + +@findex reduce +@findex reducing @table @asis -@item @samp{(set-accum VAR EXPR)} -Set the accumulation variable @samp{VAR} to the -value of @samp{EXPR}. +@item @samp{(reduce VAR EXPR FUNC)} +Reduce @samp{EXPR} into @samp{VAR} by @samp{FUNC}, like in +@code{cl-reduce} and @code{(funcall FUNC VAR EXPR)}. @samp{FUNC} is called with @samp{VAR} as the +first argument and @samp{EXPR} as the second argument. This is unlike +@samp{accumulate}, which gives @samp{VAR} and @samp{EXPR} to @samp{FUNC} in the opposite order +(that is, @samp{EXPR} first, then @samp{VAR}). -This command also has the alias @samp{setting-accum}. +This command also has the alias @samp{reducing}. -This command is a basic wrapper around @samp{set} for only one value. Because this -command cannot be optimized (as it does not construct a sequence), it is safe -to access the implicit variable @code{loopy-result} in @samp{EXPR}, so long as the -variable is not being modified by another command for which that would be -unsafe. +When @samp{VAR} does not have an explicit starting value (given with the special +macro argument @samp{with}), the first accumulated value is @samp{EXPR}. The first +accumulated value is not the result of passing @samp{VAR} and @samp{EXPR} to @samp{FUNC}. +Using the @samp{with} special macro argument is similar to using @code{cl-reduce}'s +@samp{:initial-value} keyword argument. @lisp -;; => 16 -(loopy (array i [1 2 3]) - (set-accum (+ loopy-result i))) - -;; These are equivalent to the above example: +;; => 6 +(loopy (list i '(1 2 3)) + (reduce i #'*)) -;; => 16 -(loopy (array i [1 2 3]) - (set loopy-result (+ loopy-result i)) +;; Similar to the above: +(loopy (list i '(1 2 3)) + (set loopy-result i (* i loopy-result)) (finally-return loopy-result)) -;; => 16 -(loopy (array i [1 2 3]) - (set-accum loopy-result (+ loopy-result i)) - (finally-return loopy-result)) +;; = > 6 +(loopy (with (my-reduction 0)) + (list i '(1 2 3)) + (reduce my-reduction i #'+) + (finally-return my-reduction)) + +;; Similar to the above: +(cl-reduce #'+ (list 1 2 3) :initial-value 0) +(seq-reduce #'+ [1 2 3] 0) +@end lisp + +This command also has the alias @samp{callf}. It is similar to using the +function @code{cl-callf}, except that the function argument is given last and +must be quoted. This alias is intended to help users remember argument +order. + +@lisp +(loopy (with (my-reduction 0)) + (list i '(1 2 3)) + (callf my-reduction i #'+) + (finally-return my-reduction)) + +;; Is similar to the above: +(loopy (with (my-reduction 0)) + (list i '(1 2 3)) + (do (cl-callf + my-reduction i)) + (finally-return my-reduction)) @end lisp @end table @@ -2477,12 +2629,9 @@ unsafe. @findex accumulating @table @asis @item @samp{(accumulate|accumulating VAR EXPR FUNC)} -Accumulate the result -of applying function @samp{FUNC} to @samp{EXPR} and @samp{VAR}. @samp{EXPR} and @samp{VAR} are used as -the first and second arguments to @samp{FUNC}, respectively. - -This is a generic accumulation command in case the others don't meet your -needs. It is similar in effect to using the command @samp{set}. +Accumulate the result of applying +function @samp{FUNC} to @samp{EXPR} and @samp{VAR} like in @code{(funcall FUNC EXPR VAR)}. @samp{EXPR} +and @samp{VAR} are used as the first and second arguments to @samp{FUNC}, respectively. @lisp ;; Call `(cons i my-accum)' @@ -2522,54 +2671,40 @@ quoted. This alias is intended to help users remember argument order. @end lisp @end table -@findex reduce -@findex reducing +@findex set-accum +@findex setting-accum @table @asis -@item @samp{(reduce VAR EXPR FUNC)} -Reduce @samp{EXPR} into @samp{VAR} by @samp{FUNC}. @samp{FUNC} is -called with @samp{VAR} as the first argument and @samp{EXPR} as the second argument. -This is unlike @samp{accumulate}, which gives @samp{VAR} and @samp{EXPR} to @samp{FUNC} in the -opposite order (that is, @samp{EXPR} first, then @samp{VAR}). +@item @samp{(set-accum VAR EXPR)} +Set the accumulation variable @samp{VAR} to the +value of @samp{EXPR}. -This command also has the alias @samp{reducing}. +This command also has the alias @samp{setting-accum}. -This command is similar in effect to the @samp{set} command. +This command is a basic wrapper around @samp{set} for only one value. Because this +command cannot be optimized (as it does not construct a sequence), it is safe +to access the implicit variable @code{loopy-result} in @samp{EXPR}, so long as the +variable is not being modified by another command for which that would be +unsafe. @lisp -;; = > 6 -(loopy (with (my-reduction 0)) - (list i '(1 2 3)) - (reduce my-reduction i #'+) - (finally-return my-reduction)) - -;; Works similarly to above: -(loopy (with (my-reduction 0)) - (list i '(1 2 3)) - (set my-reduction (+ i my-reduction)) - (finally-return my-reduction)) - -;; => 24 -(loopy (with (loopy-result 1)) - (list i '(1 2 3 4)) - (reduce i #'*)) -@end lisp +;; => 6 +(loopy (with (loopy-result 0)) + (array i [1 2 3]) + (set-accum (+ loopy-result i))) -This command also has the alias @samp{callf}. It is similar to using the -function @code{cl-callf}, except that the function argument is given last and -must be quoted. This alias is intended to help users remember argument -order. +;; These are equivalent to the above example: -@lisp -(loopy (with (my-reduction 0)) - (list i '(1 2 3)) - (callf my-reduction i #'+) - (finally-return my-reduction)) +;; => 6 +(loopy (with (loopy-result 0)) + (array i [1 2 3]) + (set loopy-result (+ loopy-result i)) + (finally-return loopy-result)) -;; Is similar to the above: -(loopy (with (my-reduction 0)) - (list i '(1 2 3)) - (do (cl-callf + my-reduction i)) - (finally-return my-reduction)) +;; => 6 +(loopy (with (loopy-result 0)) + (array i [1 2 3]) + (set-accum loopy-result (+ loopy-result i)) + (finally-return loopy-result)) @end lisp @end table @@ -2679,30 +2814,19 @@ Sequence accumulation commands are used to join lists (such as @samp{union} and @findex adjoin @findex adjoining @table @asis -@item @samp{(adjoin VAR EXPR &key at test key)} +@item @samp{(adjoin VAR EXPR &key at test)} Repeatedly add @samp{EXPR} to @samp{VAR} if it is not already present in the list. This command also has the alias @samp{adjoining}. -@lisp -;; Without a test, defaults to `eql' as in `cl-adjoin'. -;; => ((1 . 1) (1 . 2) (1 . 2) (2 . 3)) -(loopy (list i '((1 . 1) (1 . 2) (1 . 2) (2 . 3))) - (adjoin i)) +Unlike @code{cl-adjoin} and like the other accumulation commands, this command +defaults to adjoining @samp{EXPR} to the end of @samp{VAR}, not the beginning. -;; Using `equal' for the test. +@lisp ;; => ((1 . 1) (1 . 2) (2 . 3)) (loopy (list i '((1 . 1) (1 . 2) (1 . 2) (2 . 3))) - (adjoin i :test #'equal)) - -;; Using `=' for the test and `car' for the key. This -;; treats '(1 . 2) as equivalent to '(1 . 1), so it -;; won't be added. -;; -;; => ((1 . 1) (2 . 3)) -(loopy (list i '((1 . 1) (1 . 2) (1 . 2) (2 . 3))) - (adjoin i :test #'= :key #'car)) + (adjoin i)) ;; Coerced to a vector /after/ the loop ends. ;; => [1 2 3 4] @@ -2764,7 +2888,8 @@ This command also has the alias @samp{collecting}. ;; => [1 2 3] (loopy (list j '(1 2 3)) - (collect j :result-type 'vector)) + (collect j) + (finally-return (cl-coerce loopy-result 'vector))) ;; => (3 2 1) (loopy (list j '(1 2 3)) @@ -2781,13 +2906,11 @@ This command also has the alias @samp{collecting}. @table @asis @item @samp{(concat VAR EXPR &key at)} Repeatedly @code{concat} the value of @samp{EXPR} onto -@samp{VAR}, as a string. For concatenating values onto a vector, see the command +@samp{VAR}, as a string. For concatenating values into a vector, see the command @samp{vconcat}. This command also has the alias @samp{concating}. -@samp{VAR} is a string throughout the loop. - @lisp ;; => "abc" (loopy (list i '("a" "b" "c")) @@ -2835,7 +2958,7 @@ apply wherever that value is used (@ref{Self-Evaluating Forms,,,elisp,}). @findex nunion @findex nunioning @table @asis -@item @samp{(nunion VAR EXPR &key test key at)} +@item @samp{(nunion VAR EXPR &key test at)} Repeatedly and @emph{destructively} insert into @samp{VAR} the elements of @samp{EXPR} which are not already present in @samp{VAR}. @@ -2847,11 +2970,6 @@ This command also has the alias @samp{nunioning}. (nunion var i) (finally-return var)) -;; => ((a . 2)) -(loopy (array i [((a . 1)) ((a . 2))]) - (nunioning var i :key #'car) - (finally-return var)) - ;; => (4 2 (1 1) 3) (loopy (list i '(((1 1) 2) ((1 1) 3) (3 4))) (nunioning var i :test #'equal) @@ -2862,10 +2980,6 @@ This command also has the alias @samp{nunioning}. ((1 2 3) (3 4))]) (nunion (var1 var2) i :test #'equal) (finally-return var1 var2)) - -;; => ((4 2) (1 2) (3 2)) -(loopy (list i '(((1 2) (3 2)) ((1 1) (4 2)))) - (nunion i :at start :key #'car)) @end lisp @end table @@ -2922,7 +3036,7 @@ convenience. @findex union @findex unioning @table @asis -@item @samp{(union VAR EXPR &key test key at)} +@item @samp{(union VAR EXPR &key test at)} Repeatedly insert into @samp{VAR} the elements of the list @samp{EXPR} that are not already present in @samp{VAR}. @@ -2934,11 +3048,6 @@ This command also has the alias @samp{unioning}. (union var i) (finally-return var)) -;; => ((a . 2)) -(loopy (array i [((a . 1)) ((a . 2))]) - (unioning var i :key #'car) - (finally-return var)) - ;; => (4 2 (1 1) 3) (loopy (list i '(((1 1) 2) ((1 1) 3) (3 4))) (unioning var i :test #'equal) @@ -2949,26 +3058,19 @@ This command also has the alias @samp{unioning}. ((1 2 3) (3 4))]) (union (var1 var2) i :test #'=) (finally-return var1 var2)) - -;; => ((4 2) (1 2) (3 2)) -(loopy (list i '(((1 2) (3 2)) ((1 1) (4 2)))) - (union var i :at 'start :key #'car) - (finally-return var)) @end lisp @end table @findex vconcat @findex vconcating @table @asis -@item @samp{(vconcat VAR EXPR)} -Repeatedly concatenate the value of @samp{EXPR} onto @samp{VAR} -via the function @code{vconcat}. For concatenating values onto a string, see the -command @samp{concat}. +@item @samp{(vconcat VAR EXPR &key at)} +Repeatedly concatenate the value of @samp{EXPR} +onto @samp{VAR} via the function @code{vconcat}. For concatenating values into a +string, see the command @samp{concat}. This command also has the alias @samp{vconcating}. -@samp{VAR} is a vector throughout the loop. - @lisp ;; => [1 2 3 4 5 6] (loopy (list i '([1 2 3] [4 5 6])) @@ -2988,86 +3090,39 @@ This command also has the alias @samp{vconcating}. @findex finding @table @asis @item @samp{(find VAR EXPR TEST &key ON-FAILURE)} -If @samp{TEST} is non-nil, the loop stops -and @samp{EXPR} is used as a returned value. If @samp{TEST} is never non-nil, then -@samp{ON-FAILURE} is used as a returned value, if provided. +If the expression @samp{TEST} is non-nil, +then the loop stops and @samp{VAR} is set to the value of @samp{EXPR}. If @samp{TEST} is +never non-nil, then @samp{VAR} is set to the value of @samp{ON-FAILURE}, if provided. This command also has the alias @samp{finding}. -@samp{VAR} takes the value of @samp{EXPR} if @samp{TEST} is non-nil or @samp{ON-FAILURE} if the -loop completes successfully. It is bound to @code{nil} during the loop. As with -other accumulation commands, if @samp{VAR} is provided, then @samp{EXPR} is not used as -a return value. Instead, it is assigned to @samp{VAR}, which must be returned -explicitly. +If the loop is left early and @samp{TEST} was never non-nil, this is the same as a +normal failure and @samp{VAR} will be set to the value of @samp{ON-FAILURE}, if +provided. @lisp -;; => 3 -(loopy (list i '(1 2 3)) - (finding i (> i 2))) - -;; Equivalent to above. -(loopy (list i '(1 2 3)) - (when (> i 2) (return i))) +;; => (13 (1 2)) +(loopy (list i '(1 2 3 4 5 6 7 8)) + (find (+ i 10) (> i 2)) + (collect coll i) + (finally-return loopy-result coll)) ;; => nil -(loopy (list i '(1 2 3)) - (finding i (> i 4))) - -;; Equivalent to above. -(loopy (list i '(1 2 3)) - (when (> i 4) (return i))) - -;; => "not found" -(loopy (list i '(1 2 3)) - (finding i (> i 4) :on-failure "not found")) - -;; Equivalent to above. -(loopy (list i '(1 2 3)) - (when (> i 4) (return i)) - (else-do (cl-return "not found"))) - -;; Does not display message. -;; => 2 -(loopy (list i '(1 2 3)) - (finding i (= i 2) :into found) - (after-do (message "found: %s" found)) - (finally-return found)) - -;; Equivalent to above. -(loopy (list i '(1 2 3)) - (when (= i 2) - (set found i) - (leave)) - (after-do (message "found: %s" found)) - (finally-return found)) - -;; Messages "found: 2" in echo area. -;; => 2 -(loopy (list i '(1 2 3)) - (finding found i (= i 2)) - (finally-do (message "found: %s" found)) - (finally-return found)) +(loopy (list i '(1 2 3 4 5 6)) + (find i (> i 12))) -;; Equivalent to above. -(loopy (list i '(1 2 3)) - (when (= i 2) - (set found i) - (leave)) - (finally-do (message "found: %s" found)) - (finally-return found)) +;; => 27 +(loopy (list i '(1 2 3 4 5 6)) + (find i (> i 12) :on-failure 27)) -;; => "not found" -(loopy (list i '(1 2 3)) - (finding whether-found i (> i 4) :on-failure "not found") - (finally-return whether-found)) +;; => 27 +(loopy (list i '(1 2 3 4 5 6)) + (while (< i 3)) + (find i (> i 12) :on-failure 27)) -;; Equivalent to above. -(loopy (list i '(1 2 3)) - (when (> i 4) - (set whether-found i) - (leave)) - (else-do (setq whether-found "not found")) - (finally-return whether-found)) +;; => nil +(loopy (list i '(1 2 3 4 5 6)) + (find nil (> i 3) :on-failure 27)) @end lisp @end table @@ -3108,8 +3163,8 @@ order, so commands using such variables can produce more efficient code. @end lisp The situation becomes more complex when commands place values at both sides of a -sequence. In that case, @code{loopy} keeps track of the beginning @emph{and} the end -of the sequence. @code{loopy} does @emph{not} merely append to the end of the resulting +sequence. In that case, @code{loopy} keeps track of the beginning @emph{and} the end of +the sequence. @code{loopy} @emph{does not} merely append to the end of the accumulating list, since that would be much slower for large lists. @lisp @@ -3167,6 +3222,7 @@ Because accumulation into @code{coll} has been optimized, the order of values in @lisp ;; This code optimizes for insertions at the end of `coll': +;; ;; => ((23 13 22 12 21 11 1 2 3) ;; ((1 11 21) (2 1 11 21 12 22) (3 2 1 11 21 12 22 13 23))) (loopy (accum-opt (coll end)) @@ -3180,8 +3236,8 @@ Because accumulation into @code{coll} has been optimized, the order of values in The @samp{accump-opt} special macro argument can also be used with destructuring. Because destructuring requires using named variables, such variables are by -default required to be ordered correctly during the loop. If you do not require -that, you are recommended to use @samp{accum-opt} on those variables. +default required to be ordered correctly during the loop. If you do not need +them to be so, you are recommended to use @samp{accum-opt} on those variables. @lisp ;; => ((1 3) @@ -3201,51 +3257,36 @@ that, you are recommended to use @samp{accum-opt} on those variables. @dfn{Boolean commands} are used to test whether a condition holds true during the loop. They work like a combination of iteration and accumulation -commands, in that values are stored in @code{loopy-result} and that can terminate the -loop. +commands, in that values are stored in @code{loopy-result} and that they can +terminate the loop. -The behavior and use of the boolean commands is a compromise between consistency -with other commands, similarity to how similar features are used in other -libraries, and convenience for how they are commonly used. This gives us the -following: +Be aware of the following: @itemize @item -@code{loopy-result} is used as the implicit return value of the loop. +Currently, unlike accumulation commands, there is no non-keyword way to +specify a variable. The first argument (the only required argument) of each +boolean command is a condition to check. @item Like accumulation commands, the keyword @samp{:into} can be used the specify a variable other than @code{loopy-result}. -@itemize -@item -Unlike accumulation commands, there is no non-keyword way to specify a -variable. The first argument (the only required argument) of each boolean -command is a condition to check. - @item The @samp{always} and @samp{never} commands must use the same variable to work together correctly. By default, the both use @code{loopy-result}. -@end itemize - -@item -These commands exit the loop without forcing a value (@ref{Early Exit}). - -@itemize -@item -Therefore, optimized accumulation variables can be finalized even when the -loop ends, as happens with the @samp{leave} command. @item +These commands exit the loop without forcing a return value +(@ref{Early Exit}). Therefore, optimized accumulation variables can be +finalized even when the loop ends, as happens with the @samp{leave} command. However, because the boolean commands already use @code{loopy-result}, such optimized accumulation variables must be created with the special macro argument @samp{accum-opt} and must be used explicitly, as in the below example. @end itemize -@end itemize @lisp -;; A maybe unidiomatic example: ;; => (nil (1 3 5)) (loopy (accum-opt coll) (list i '(1 3 5 6 9)) @@ -3253,19 +3294,10 @@ argument @samp{accum-opt} and must be used explicitly, as in the below example. (collect coll i) (finally-return loopy-result coll)) -;; Same as above, but maybe more idiomatic: -;; => (nil (1 3 5)) -(loopy (with (succes t)) - (list i '(1 3 5 6 9)) - (if (cl-oddp i) - (collect i) - (set success nil) - (leave)) - (finally-return success loopy-result)) - ;; Works similarly, but forces the `nil' return value. ;; Returns the collection if `always' doesn't trigger an exit. ;; Attempting similar with CL's `iterate' will signal an error. +;; ;; => nil (cl-loop for i in '(1 3 5 6 9) always (cl-oddp i) @@ -3273,12 +3305,14 @@ argument @samp{accum-opt} and must be used explicitly, as in the below example. @end lisp @quotation Warn -Using the command @samp{thereis} is incompatible with using the commands @samp{always} and -@samp{never}, as this would create conflicting initial values for the implicit return -value (both using @code{loopy-result}). +Using the command @samp{thereis} for a variable is incompatible with using the +commands @samp{always} and @samp{never} on that same variable, as this would create +conflicting initial values for the implicit return value (both using +@code{loopy-result}). @end quotation + @findex always @table @asis @item @samp{(always EXPR &key into)} @@ -3345,7 +3379,7 @@ the loop's implicit return value. The value of the condition is checked. @item If the condition is non-nil, then the variable is set to @code{nil} -and the loop is ended via the @samp{leave} command. +and the loop is exited. @end enumerate @end table @@ -3381,9 +3415,9 @@ the same variable. @findex thereis @table @asis @item @samp{(thereis EXPR &key into)} -Check the result of the condition -@samp{EXPR}. If the condition evaluates to a non-@code{nil} value, the loop returns -that value. Otherwise, the loop returns nil. +Check the result of the condition @samp{EXPR}. If +the condition evaluates to a non-@code{nil} value, the loop returns that value. +Otherwise, the loop returns nil. The steps are thus: @enumerate @@ -3473,7 +3507,8 @@ Run @samp{CMDS} only if @samp{EXPR} is non-nil. This is the @code{loopy} version of the @code{when} macro from normal Emacs Lisp. @lisp -;; Get only the inner lists with all even numbers. +;; Get only the inner lists with all even numbers: +;; ;; => '((2 4 6) (8 10 12) (16 18 20)) (loopy (list i '((2 4 6) (8 10 12) (13 14 15) (16 18 20))) (when (loopy (list j i) @@ -3490,7 +3525,8 @@ Run @samp{CMDS} only if @samp{EXPR} is nil. This is the @code{loopy} version of the @code{unless} macro from normal Emacs Lisp. @lisp -;; Get only the inner lists with all even numbers. +;; Get only the inner lists with all even numbers: +;; ;; => '((2 4 6) (8 10 12) (16 18 20)) (loopy (list i '((2 4 6) (8 10 12) (13 14 15) (16 18 20))) (unless (loopy (list j i) @@ -3508,18 +3544,17 @@ version of the @code{unless} macro from normal Emacs Lisp. @findex skipping @findex continuing @table @asis -@item @samp{(skip|skipping|continue|continuing)} -Skip the remaining commands and -continue to the next loop iteration. +@item @samp{(skip|continue)} +Skip the remaining commands and continue to the next loop +iteration. + +This command also has the aliases @samp{skipping} and @samp{continuing}. @lisp -;; => (2 4 6 8 12 14 16 18) +;; => (2 4 6 8 10 12 14 16 18 20) (loopy (sequence i (number-sequence 1 20)) - (when (zerop (mod i 10)) - (skip)) - (when (cl-evenp i) - (push-into my-collection i)) - (finally-return (nreverse my-collection))) + (when (cl-oddp i) (skip)) + (collect i)) @end lisp @end table @@ -3528,17 +3563,19 @@ continue to the next loop iteration. @findex skipping-from @findex continuing-from @table @asis -@item @samp{(skip-from|skipping-from|continue-from|continuing-from NAME)} -Skip the -remaining commands and continue to the next loop iteration of the loop @samp{NAME}. +@item @samp{(skip-from|continue-from NAME)} +Skip the remaining commands and continue +to the next loop iteration of the loop @samp{NAME}. + +This command also has the aliases @samp{skipping-from} and @samp{continuing-from}. @lisp ;; => ((1 2 3) (7 8 9)) (loopy outer (array i [(1 2 3) (4 5 6) (7 8 9)]) - (loop (list j i) - (if (= 5 j) - (skip-from outer))) + (loopy (list j i) + (when (= 5 j) + (skip-from outer))) (collect i)) @end lisp @end table @@ -3547,24 +3584,92 @@ remaining commands and continue to the next loop iteration of the loop @samp{NAM @subsection Early Exit The loop is contained in a @code{cl-block}, which can be exited by the function -@code{cl-return-from}. Indeed, the @samp{return} and @samp{return-from} commands are just -wrappers around that function. +@code{cl-return-from}. Indeed, the @samp{return} and @samp{return-from} commands described +below are just wrappers around that function. As with the @samp{finally-return} special macro argument, passing multiple return values to those commands will return a list of those values. If no value is given, @code{nil} is returned. -The commands @samp{leave}, @samp{leave-from}, @samp{while}, and @samp{until} leave the current loop -without forcing a returned value. Unlike the @samp{return} commands, they do not -stop the loop from returning any implied return values, such as the collections -in their respective examples. +In Loopy, implied accumulation variables can be modified a final time after the +loop exits in order to finalize their values. For example, if an accumulated +list is built in reverse, then it will be reversed into the correct order after +the loop completes. Loopy has ``return'' commands for (1) immediately returning a +value from the loop without finalizing values and ``leave'' commands for (2) +leaving the loop without forcing a return value, allowing values to be +finalized. + +@lisp +;; An example of not finalizing the accumulated value: +;; +;; => (4 3 2 1 0) +(loopy (numbers i :to 10) + (if (< i 5) + (collect i) + ;; Not finalized: + (return loopy-result))) + +;; An example of finalizing the accumulated value: +;; +;; => (0 1 2 3 4) +(loopy (numbers i :to 10) + (if (< i 5) + (collect i) + (leave)) + (finally-return loopy-result)) +@end lisp + +As noted in @ref{Special Macro Arguments}, the special macro argument @samp{finally-do} does not +affect the return value of the loop. + +@lisp +;; => (0 1 2 3 4) +(loopy (numbers i :to 10) + (if (< i 5) + (collect i) + (leave)) + (finally-do 7)) + +;; => (4 3 2 1 0) +(loopy (numbers i :to 10) + (if (< i 5) + (collect i) + ;; Not finalized: + (return loopy-result)) + (finally-do + (cl-callf2 cons 22 loopy-result))) +@end lisp + +As noted in @ref{Special Macro Arguments}, the special macro argument @samp{finally-return} +overrides the return value of the loop, including values that would have been +returned by any @samp{return} commands. + +@lisp +;; => 22 +(loopy (numbers i :to 10) + (if (< i 5) + (collect i) + ;; Not finalized: + (return loopy-result)) + (finally-return 22)) + +;; => 22 +(loopy (numbers i :to 10) + (if (< i 5) + (collect i) + (leave)) + (finally-return 22)) +@end lisp + @findex leave @findex leaving @table @asis -@item @samp{(leave|leaving)} +@item @samp{(leave)} Leave the current loop without forcing a return value. +This command also has the alias @samp{leaving}. + @lisp ;; => (1 2 3 4) (loopy (list i '(1 2 3 4 5 6 7)) @@ -3581,6 +3686,8 @@ Leave the current loop without forcing a return value. Leave the loop @samp{NAME} without forcing a return value. This command is equivalent to @samp{(at NAME (leave))} (@ref{Sub-Loops}). +This command also has the alias @samp{leaving-from}. + @lisp ;; => ([2 4] [6 8]) (loopy outer @@ -3596,8 +3703,11 @@ return value. This command is equivalent to @samp{(at NAME (leave))} (@ref{Sub- @findex return @findex returning @table @asis -@item @samp{(return|returning [EXPRS])} -Leave the current loop, returning @samp{[EXPRS]}. +@item @samp{(return [EXPRS])} +Return from the current loop without finalizing values, +returning @samp{[EXPRS]}. + +This command also has the alias @samp{returning}. @lisp ;; => 6 @@ -3611,9 +3721,11 @@ Leave the current loop, returning @samp{[EXPRS]}. @findex return-from @findex returning-from @table @asis -@item @samp{(return-from|returning-from NAME [EXPRS])} -Leave the loop @samp{NAME}, -returning @samp{[EXPRS]}. +@item @samp{(return-from NAME [EXPRS])} +Return from the loop @samp{NAME}, returning +@samp{[EXPRS]}. This command is equivalent to @samp{(at NAME (return))} (@ref{Sub-Loops}). + +This command also has the alias @samp{returning-from}. @lisp ;; => 'bad-val? @@ -3634,7 +3746,14 @@ return value. @samp{(while COND)} is the same as @samp{(until (not COND))}. @lisp ;; => (1 2 3 4) (loopy (list i '(1 2 3 4 5 6 7)) - (while (not (= i 5))) + (while (/= i 5)) + (collect i)) + +;; Same as the above: +;; +;; => (1 2 3 4) +(loopy (list i '(1 2 3 4 5 6 7)) + (unless (/= i 5) (leave)) (collect i)) @end lisp @end table @@ -3650,31 +3769,52 @@ value. @samp{(until COND)} is the same as @samp{(while (not COND))}. (loopy (list i '(1 2 3 4 5 6 7)) (until (= i 5)) (collect i)) + +;; Same as the above: +;; +;; => (1 2 3 4) +(loopy (list i '(1 2 3 4 5 6 7)) + (when (= i 5) (leave)) + (collect i)) @end lisp @end table @node Sub-Loops @section Sub-Loops -The following commands exist to properly handle sub-loops. Depending on -one's needs, trying to use a @code{loopy} macro call inside the @samp{do} command could -signal an error. This is because arguments of the @samp{do} command are inserted -literally (that is, without interpretation or modification) into the expanded -code. This means that using @code{loopy} inside the @samp{do} command cannot affect -any of the super-loop's generated code, such as variable declarations. The -following commands do not have that problem, because their expansions are -processed during the expansion of the top-level macro. +Loopy provides two sets of commands for working with sub-loops: +@enumerate +@item +The @samp{loopy} and @samp{loopy-iter} commands, which correctly expand inner loops +during the expansion of outer loops. +@item +The @samp{at} command, which controls the named outer loop that commands interact +with. For example, it can control to which loop an implied accumulation +variable is scoped and so for which loop the variable is used as an implied +return value. +@end enumerate + +In the example below, the arguments of the @samp{do} command are inserted into the +loop body literally, so by the time the inner loop expands, the outer loop has +already been expanded into normal Emacs Lisp code, and so the inner macro cannot +find any outer Loopy loop named ``outer''. @lisp -;; Raises an error: +;; Can signal an error or not work as expected: (loopy (named outer) (list i '((1 2) (3 4) (5 6))) (do (loopy (list j i) (when (= j 5) (leave-from outer)))) (collect i)) +@end lisp +In general, correct code requires that inner loops be expanded during the +expansion of the outer loop, as done with other commands. + +@lisp ;; Works as expected: +;; ;; => ((1 2) (3 4)) (loopy outer (list i '((1 2) (3 4) (5 6))) @@ -3684,27 +3824,42 @@ processed during the expansion of the top-level macro. (collect i)) @end lisp +The commands are described in more detail below. + @findex loopy command @table @asis @item @samp{(loopy [SPECIAL-MACRO-ARGUMENTS or CMDS])} -Specifically wrap a call to -the @code{loopy} macro. +Use the @code{loopy} macro as a +command. +@quotation Warning Don't confuse using this command with using calls to the macro @code{loopy}. For example, the @samp{EXPR} parameter to loop commands is used literally, and -is not able to affect macro expansion. Therefore, the warning at the start -of this section applies to @samp{EXPR} parameters as well. This is demonstrated -in the example below. +is not guaranteed to be able to affect macro expansion. + +@end quotation +@end table + + +@findex loopy-iter command +@table @asis +@item @samp{(loopy-iter [SPECIAL-MACRO-ARGUMENTS or CMDS or LISP-EXPRS])} +Use the +@code{loopy-iter} macro as a command (@ref{The @code{loopy-iter} Macro}). + +This feature can only be used after first loading the library @samp{loopy-iter}. @lisp +(require 'loopy-iter) + +;; => '(11 12 13 14 15 16 17 18) (loopy outer - (list i '(1 2 3)) - ;; This is the macro `loopy', not the loop command. - ;; Therefore, the warning about macro expansion applies, - ;; and this signals an error. - (when (loopy (at outer (collect i)) - (leave)) - (do nil))) + (list i '([(1 2) (3 4)] + [(5 6) (7 8)])) + (loopy-iter (arraying j i) + (cl-flet ((10+ (x) (+ x 10))) + (at outer + (appending (mapcar #'10+ j)))))) @end lisp @end table @@ -3745,28 +3900,6 @@ which they are used, even when using the @samp{at} command. @end lisp @end table -@findex loopy-iter command -@table @asis -@item @samp{(loopy-iter [SPECIAL-MACRO-ARGUMENTS or CMDS or LISP-EXPRS])} -Specifically -wrap a call to the @code{loopy-iter} macro (@ref{The @code{loopy-iter} Macro}). - -This feature can only be used after first loading the library @samp{loopy-iter}. - -@lisp -(require 'loopy-iter) - -;; => '(11 12 13 14 15 16 17 18) -(loopy outer - (list i '([(1 2) (3 4)] - [(5 6) (7 8)])) - (loopy-iter (arraying j i) - (cl-flet ((10+ (x) (+ x 10))) - (at outer - (appending (mapcar #'10+ j)))))) -@end lisp -@end table - @node Destructuring Macros @chapter Destructuring Macros @@ -3968,7 +4101,7 @@ using the @code{let*} special form. This method recognizes all commands and their aliases in the user option @code{loopy-aliases}. -@float Listing,org6ad6355 +@float Listing,org70be5e9 @lisp ;; => ((1 2 3) (-3 -2 -1) (0)) (loopy-iter (arg accum-opt positives negatives other) @@ -4828,7 +4961,7 @@ accumulation itself must still occur within the loop @samp{inner}. ;; => (1 2 3 4) (loopy (named outer) (array i [(1 2) (3 4)]) - (sub-loop inner + (loopy inner (list j i) (at outer (collect coll j))) (finally-return coll)) diff --git a/loopy-commands.el b/loopy-commands.el index abd35e62..686cc75c 100644 --- a/loopy-commands.el +++ b/loopy-commands.el @@ -114,6 +114,14 @@ The lists will be in the order parsed (correct for insertion)." ;; Return the sub-lists. (list (nreverse wrapped-main-body) (nreverse other-instructions)))) +;; We find ourselves doing this pattern a lot. +(cl-defmacro loopy--bind-main-body ((main-expr other-instrs) value &rest body) + "Bind MAIN-EXPR and OTHER-INSTRS for those items in VALUE for BODY." + (declare (indent 2)) + `(cl-destructuring-bind (,main-expr ,other-instrs) + (loopy--extract-main-body ,value) + ,@body)) + (defun loopy--convert-iteration-vars-to-other-vars (instructions) "Convert instructions for `loopy--iteration-vars' to `loopy--other-vars'. diff --git a/loopy-iter.el b/loopy-iter.el index ce23d4b2..ce6a698e 100644 --- a/loopy-iter.el +++ b/loopy-iter.el @@ -47,6 +47,11 @@ ;; Previously, `loopy-iter' used its tree-walking functions, as done by CL's ;; Iterate, but now it just defers to what Emacs already does when expanding ;; macros, such as in `macroexpand-all'. +;; +;; NOTE: When using `macroexpand-all', always pass an environment like +;; `macroexpand-all-environment'. Otherwise, the environment is +;; set to nil, which breaks any macros that build their own environment +;; while expanding commands. For example, `cl-flet'. ;;;; Custom User Options (defgroup loopy-iter nil @@ -169,7 +174,9 @@ This variable is bound while `loopy-iter' is running, combining `loopy-iter-overwritten-command-parsers'.") (defun loopy-iter--parse-command (command) - "An Iter version of `loopy--parse-loop-command'." + "Parse COMMAND using parsers in`loopy-iter--command-parsers'. + +See also `loopy--parse-loop-command'." (let* ((cmd-name (cl-first command)) (parser (loopy--get-command-parser cmd-name @@ -182,61 +189,47 @@ This variable is bound while `loopy-iter' is running, combining (defvar loopy-iter--non-main-body-instructions nil "Used to capture other instructions while expanding. -Expanding functions `push' lists of instructions into this variable.") +Expanding functions `push' lists of instructions into this +variable. The contents of main-body instructions are inserted +into the expanded body in the command's place during macro +expansion.") ;;;;; Expanders -(defvar loopy-iter--sub-level-expanders nil - "Macro expanders for sub-level expressions.") - -(defvar loopy-iter--top-level-expanders nil - "Macro expanders for top-level expressions.") - -(defun loopy-iter--macroexpand-top (expr) - "Expand a top-level expression using `loopy-iter--top-level-expanders'" - (macroexpand-1 expr loopy-iter--top-level-expanders)) - -(defun loopy-iter--macroexpand-sub (expr) - "Expand a top-level expression using `loopy-iter--sub-level-expanders'" - (macroexpand-all expr loopy-iter--sub-level-expanders)) - -(defun loopy-iter--keyword-expander-top (&rest args) - "Expand top-level commands preceded by keywords in `loopy-iter-keywords'." - (cl-destructuring-bind (main other) - (loopy--extract-main-body - (loopy-iter--parse-command args)) - (push other loopy-iter--non-main-body-instructions) - (macroexp-progn main))) - -(defun loopy-iter--keyword-expander-sub (&rest args) - "Expand sub-level commands preceded by keywords in `loopy-iter-keywords'." - (cl-destructuring-bind (main other) - (loopy--extract-main-body - (let ((loopy--in-sub-level t)) - (loopy-iter--parse-command args))) - (push other loopy-iter--non-main-body-instructions) - (macroexp-progn main))) +(defvar loopy-iter--level nil + "The level of the expression `loopy-iter' is currently processing. + +For example, iteration commands should only be processed during +level 1, which is the top level. The next level of nesting is +level 2, and so on. If `loopy-iter--level' is greater than 1, +then `loopy--in-sub-level' is set to `t'. + +The macro initially `let'-binds this variable to 0, and it is +incremented upon parsing a new function.") (defun loopy-iter--opt-accum-expand-val (arg) - "Macro expand only the value of an optimized accumulation. + "Macro expand only the value of the optimized accumulation expression ARG. Optimized accumulations are expanded into a special form, after which this function will recursively expand the expression of the accumulated value. To avoid an infinite loop, this function replaces the `loopy--optimized-accum' -in the expression with `loopy--optimized-accum-2'." +in the expression with `loopy--optimized-accum-2', which is then processed +during a second pass on the expanded code." (loopy (with (plist (cadr arg))) (cons (k v) plist :by #'cddr) (collect k) ;; By this point, command expansion are already defined, so we don't ;; need to try to handle instructions. (collect (if (eq k :val) - (macroexpand-all v loopy-iter--sub-level-expanders) + (let ((loopy-iter--level (1+ loopy-iter--level)) + (loopy--in-sub-level t)) + (macroexpand-all v macroexpand-all-environment)) v)) (finally-return `(loopy--optimized-accum-2 (quote ,loopy-result))))) -;;;;; Overwritten definitions +;;;;; Overwritten parser definitions (defcustom loopy-iter-overwritten-command-parsers '((at . loopy-iter--parse-at-command)) @@ -258,15 +251,17 @@ These commands affect other loops higher up in the call list." (loopy--check-target-loop-name target-loop) ;; We need to capture all non-main-body instructions into a new `at' ;; instruction, so we just temporarily `let'-bind - ;; `loopy-iter--non-main-body-instructions' while expanders push to it, - ;; we which then wrap back in a new instruction and pass up to the calling - ;; function, which consumes instructions. + ;; `loopy-iter--non-main-body-instructions' while the expanding functions push + ;; to it, which we then wrap back in a new instruction and pass up to the + ;; calling function, which consumes instructions. (loopy (with (loopy-iter--non-main-body-instructions nil) (loopy--loop-name target-loop) - (loopy--in-sub-level t)) + (loopy--in-sub-level t) + (loopy-iter--level (1+ loopy-iter--level))) (list cmd commands) - (collect (list 'loopy--main-body - (loopy-iter--macroexpand-sub cmd))) + (collect (list 'loopy--main-body (macroexpand-all + cmd + macroexpand-all-environment))) (finally-return ;; Return list of instructions to comply with expectations of calling ;; function, which thinks that this is a normal loop-command parser. @@ -444,7 +439,6 @@ for other reasons. The macros `cl-block', `cl-return-from', and `cl-return' are known to fall into the first group.") ;;;; The macro itself - (defmacro loopy-iter (&rest body) "Allows embedding loop commands in arbitrary code within this macro's body. @@ -462,137 +456,135 @@ on how to use `loopy-iter'. See the Info node `(loopy)' for how to use `loopy' in general. \(fn CODE-or-COMMAND...)" + ;; We expand the code in BODY in two passes. The macro works like this: + ;; + ;; 1) Like normal, process the special macro arguments. + ;; + ;; 2) Set up the environments used for macro expansion. These are alists of + ;; macro names and functions that process the macro arguments. The + ;; functions only receive the arguments, not the name. + ;; + ;; 3) Parse commands and process the initial resulting instructions. The + ;; result is a new body with the commands expanded, but the optimized + ;; accumulations incomplete and `at' instructions are not finished. + ;; + ;; 4) Parse the optimized accumulations and process the `at' instructions the + ;; resulted from processing the commands and instructions in Step 3. + ;; + ;; 5) Set `loopy--main-body' to the now expanded expressions (as a list, no + ;; `macroexpand-progn'). + ;; + ;; 6) Then we manipulate the variables and build the loop like normal, as we + ;; do in `loopy'. + (loopy--wrap-variables-around-body (mapc #'loopy--apply-flag loopy-default-flags) - (setq body (loopy-iter--process-special-arg-loop-name body)) - (setq body (loopy-iter--process-special-arg-flag body)) - (setq body (loopy-iter--process-special-arg-with body)) - (setq body (loopy-iter--process-special-arg-without body)) - (setq body (loopy-iter--process-special-arg-accum-opt body)) - (setq body (loopy-iter--process-special-arg-wrap body)) - (setq body (loopy-iter--process-special-arg-before-do body)) - (setq body (loopy-iter--process-special-arg-after-do body)) - (setq body (loopy-iter--process-special-arg-finally-do body)) - (setq body (loopy-iter--process-special-arg-finally-return body)) - (setq body (loopy-iter--process-special-arg-finally-protect body)) - - ;; Process the main body. - (unwind-protect - (progn - (let ((suppressed-alist (loopy (list i loopy-iter-suppressed-macros) + (setq body (thread-first body + loopy-iter--process-special-arg-loop-name + loopy-iter--process-special-arg-flag + loopy-iter--process-special-arg-with + loopy-iter--process-special-arg-without + loopy-iter--process-special-arg-accum-opt + loopy-iter--process-special-arg-wrap + loopy-iter--process-special-arg-before-do + loopy-iter--process-special-arg-after-do + loopy-iter--process-special-arg-finally-do + loopy-iter--process-special-arg-finally-return + loopy-iter--process-special-arg-finally-protect)) + + (loopy--with-protected-stack + (let* ((suppressed-expanders (loopy (list i loopy-iter-suppressed-macros) (collect (cons i nil)))) - (loopy-iter--command-parsers - (or loopy-iter--command-parsers - (append loopy-iter-overwritten-command-parsers - loopy-command-parsers)))) - - ;; During the initial top-level expansion and the subsequent - ;; all-level expansion, we make an effort to keep instructions in the - ;; same order that they are received. This might help to avoid - ;; unexpected behavior regarding variable declarations. For example, - ;; if the top level of a following expression refers back to a - ;; variable initialized in a preceeding sub-expression. - (let ((loopy-iter--non-main-body-instructions) - (cmd-alist-1) - (cmd-alist-sub) - (keyword-alist-1) - (keyword-alist-sub)) - - ;; Entries for command names. - (dolist (cmd loopy-iter-bare-commands) - (let ((cmd cmd)) - (push (cons cmd (lambda (&rest args) - (cl-destructuring-bind (main other) - (loopy--extract-main-body - (loopy-iter--parse-command - (cons cmd args))) - (push other - loopy-iter--non-main-body-instructions) - (macroexp-progn main)))) - cmd-alist-1) - (push (cons cmd (lambda (&rest args) - (cl-destructuring-bind (main other) - (loopy--extract-main-body - (let ((loopy--in-sub-level t)) - (loopy-iter--parse-command - (cons cmd args)))) - (push other - loopy-iter--non-main-body-instructions) - (macroexp-progn main)))) - cmd-alist-sub))) - - ;; Entries for keyword commands - (dolist (keyword loopy-iter-keywords) - (push (cons keyword #'loopy-iter--keyword-expander-top) - keyword-alist-1) - (push (cons keyword #'loopy-iter--keyword-expander-sub) - keyword-alist-sub)) - - (let* ((loopy-iter--top-level-expanders - `(,@suppressed-alist - (loopy--optimized-accum . loopy-iter--opt-accum-expand-val) - (loopy--optimized-accum-2 . nil) - ,@cmd-alist-1 - ,@keyword-alist-1)) - (loopy-iter--sub-level-expanders - `(,@suppressed-alist - (loopy--optimized-accum . loopy-iter--opt-accum-expand-val) - (loopy--optimized-accum-2 . nil) - ,@cmd-alist-sub - ,@keyword-alist-sub - ,@macroexpand-all-environment))) - - ;; Now process the main body. - (loopy (accum-opt new-body) - (list expr body) - (collect new-body - (thread-last expr - loopy-iter--macroexpand-top - loopy-iter--macroexpand-sub)) - (finally-do - (setq loopy--main-body new-body) - (loopy--process-instructions - (thread-last loopy-iter--non-main-body-instructions - nreverse - (apply #'append))))))) - - ;; Expand any uses of `loopy--optimized-accum' as if it were a macro, - ;; using the function `loopy--expand-optimized-accum'. - (loopy - ;; TODO: - ;; - Is there a way to only expand `loopy--optimized-accum'? - (with (macro-funcs `(,@suppressed-alist - ;; Identify second version of optimized accumulation. - (loopy--optimized-accum-2 . loopy--expand-optimized-accum) - ,@macroexpand-all-environment))) - (list i loopy--main-body) - (collect (macroexpand-all i macro-funcs)) - (finally-do (setq loopy--main-body loopy-result)))) - - - (loopy--process-instructions (map-elt loopy--at-instructions - loopy--loop-name) - :erroring-instructions - '(loopy--main-body))) - (loopy--clean-up-stack-vars)) - - ;; Make sure the order-dependent lists are in the correct order. - (loopy--correct-var-structure :exclude-main-body t) - ;; (setq loopy--iteration-vars (nreverse loopy--iteration-vars) - ;; loopy--accumulation-vars (nreverse loopy--accumulation-vars) - ;; loopy--implicit-return (when (consp loopy--implicit-return) - ;; (if (= 1 (length loopy--implicit-return)) - ;; ;; If implicit return is just a single thing, - ;; ;; don't use a list. - ;; (car loopy--implicit-return) - ;; ;; If multiple items, be sure to use a list - ;; ;; in the correct order. - ;; `(list ,@(nreverse loopy--implicit-return))))) - - ;; Produce the expanded code, based on the `let'-bound variables. - (loopy--expand-to-loop))) + (loopy-iter--command-parsers (or loopy-iter--command-parsers + (append loopy-iter-overwritten-command-parsers + loopy-command-parsers))) + (loopy-iter--non-main-body-instructions) + (loopy-iter--level 0) + (command-env + (append (loopy (list keyword loopy-iter-keywords) + (collect + (cons keyword + (lambda (&rest args) + (loopy--bind-main-body (main other) + ;; Bind here in case a command required to be + ;; in the top level is found in an expression + ;; while parsing an actual top-level command. + (let* ((loopy-iter--level (1+ loopy-iter--level)) + (loopy--in-sub-level (> loopy-iter--level 1))) + (loopy-iter--parse-command args)) + (push other loopy-iter--non-main-body-instructions) + (macroexp-progn main)))))) + (loopy (list command loopy-iter-bare-commands) + (collect + (cons command + ;; Expanding functions do not receive the head + ;; of the expression, only the arguments, so + ;; we use a lexical lambda to include that + ;; information. + (let ((cmd command)) + (lambda (&rest args) + (loopy--bind-main-body (main other) + ;; Bind here in case a command required to + ;; be in the top level is found in an + ;; expression while parsing an actual + ;; top-level command. + (let* ((loopy-iter--level (1+ loopy-iter--level)) + (loopy--in-sub-level (> loopy-iter--level 1))) + (loopy-iter--parse-command (cons cmd args))) + (push other loopy-iter--non-main-body-instructions) + (macroexp-progn main))))))))) + (common-env `(,@suppressed-expanders + ,@command-env + ,@macroexpand-all-environment)) + (first-pass-env `((loopy--optimized-accum . loopy-iter--opt-accum-expand-val) + (loopy--optimized-accum-2 . nil) + ,@common-env)) + (second-pass-env `(;; Identify second version of optimized accumulation. + (loopy--optimized-accum-2 . loopy--expand-optimized-accum) + ,@common-env))) + + (cl-labels (;; A wrapper to set `loopy--in-sub-level' correctly: + ;; If this is a known command, expand as normal. The command + ;; parser will handle sub-level-ness. Otherwise, while EXPR + ;; isn't a command itself, bind `loopy--in-sub-level' in case + ;; of any commands further down. + (iter-macroexpand-all (expr) + (if (map-elt command-env (car expr)) + (macroexpand-all expr first-pass-env) + (let ((loopy-iter--level (1+ loopy-iter--level)) + (loopy--in-sub-level t)) + (macroexpand-all expr first-pass-env)))) + ;; Process body, insert data for optimized accumulations, + ;; then process the other instructions: + (first-pass (body) + (prog1 + (mapcar #'iter-macroexpand-all body) + (loopy--process-instructions + (thread-last loopy-iter--non-main-body-instructions + nreverse + (apply #'append))))) + ;; Expand the optimized accumulation variables, + ;; then process the `at' instructions for this loop: + (second-pass (body) + (prog1 + (mapcar (lambda (expr) (macroexpand-all expr second-pass-env)) + body) + (loopy--process-instructions (map-elt loopy--at-instructions + loopy--loop-name) + :erroring-instructions + '(loopy--main-body))))) + (setq loopy--main-body + (thread-first body + first-pass + second-pass))) + + ;; Make sure the order-dependent lists are in the correct order. + (loopy--correct-var-structure :exclude-main-body t) + + ;; Produce the expanded code, based on the `let'-bound variables. + (loopy--expand-to-loop))))) ;;;; Add `loopy-iter' to `loopy' (cl-defun loopy-iter--parse-loopy-iter-command ((_ &rest body)) diff --git a/loopy-misc.el b/loopy-misc.el index 255d7662..45cdbf8e 100644 --- a/loopy-misc.el +++ b/loopy-misc.el @@ -309,6 +309,7 @@ splitting (1 2 3) or (1 2 . 3) returns ((1 2) 3)." ;;;; Destructuring + ;; This better allows for things to change in the future. (defun loopy--var-ignored-p (var) "Return whether VAR should be ignored." diff --git a/loopy.el b/loopy.el index 09d7a06e..1b172232 100644 --- a/loopy.el +++ b/loopy.el @@ -654,6 +654,12 @@ code and must instead be cleaned up manually." (cl-callf2 seq-drop-while (lambda (x) (eq loopy--loop-name (caar x))) loopy--accumulation-variable-info)) +(defmacro loopy--with-protected-stack (&rest body) + "Protect the stack variables from BODY during unwind and cleanup." + `(unwind-protect + ,(macroexp-progn body) + (loopy--clean-up-stack-vars))) + ;;;;; Process Instructions (cl-defun loopy--process-instruction (instruction &key erroring-instructions) "Process INSTRUCTION, assigning values to the variables in `loopy--variables'. @@ -1003,7 +1009,8 @@ see the Info node `(loopy)' distributed with this package." with macro-funcs = `(,@(cl-loop for i in loopy--suppressed-macros collect (cons i nil)) (loopy--optimized-accum - . loopy--expand-optimized-accum)) + . loopy--expand-optimized-accum) + ,@macroexpand-all-environment) for i in loopy--main-body collect (macroexpand-all i macro-funcs))) diff --git a/tests/iter-tests.el b/tests/iter-tests.el index 5f0cd2d1..d3c93bfd 100644 --- a/tests/iter-tests.el +++ b/tests/iter-tests.el @@ -762,8 +762,6 @@ E.g., \"(let ((for list)) ...)\" should not try to operate on the (push j target))) target)))) - - (ert-deftest loopy-iter-clean-stack-variables () (let ((loopy--known-loop-names) (loopy--accumulation-places) diff --git a/tests/tests.el b/tests/tests.el index 9bf18086..f1376971 100644 --- a/tests/tests.el +++ b/tests/tests.el @@ -5549,6 +5549,96 @@ This assumes that you're on guix." :iter-bare ((list . listing) (cycle . cycling))) +(loopy-deftest accum-flet-outside + :doc "Make sure that macro expansion doesn't mess with `cl-flet' environment. +We don't want to rebind the environment to nil by failing to pass +the existing environment (`macroexpand-all-environment') to +`macroexpand-all'." + :result '(11 12 13 14 15) + :wrap ((x . `(cl-flet ((10+ (y) (+ 10 y))) ,x))) + :body ((list i '(1 2 3 4 5)) + (collect (10+ i))) + :loopy t + :iter-keyword (list collect) + :iter-bare ((list . listing) + (collect . collecting))) + +(loopy-deftest accum-flet-outside-wrap-sma + :doc "Make sure that macro expansion doesn't mess with `cl-flet' environment. +We don't want to rebind the environment to nil by failing to pass +the existing environment (`macroexpand-all-environment') to +`macroexpand-all'." + :result '(11 12 13 14 15) + :body ((wrap (cl-flet ((10+ (y) (+ 10 y))))) + (list i '(1 2 3 4 5)) + (collect (10+ i))) + :loopy t + :iter-keyword (list collect) + :iter-bare ((list . listing) + (collect . collecting))) + +(loopy-deftest flet-iter-subloop + :doc "Make sure that macro expansion doesn't mess with `cl-flet' environment. +We don't want to rebind the environment to nil by failing to pass +the existing environment (`macroexpand-all-environment') to +`macroexpand-all'." + :result '(11 12 13 14 15) + :multi-body t + :body [((named outer) + (list i '((1 2) (3 4) (5))) + (loopy-test-escape + (loopy-iter (listing j i) + (at outer + (cl-flet ((10+ (y) (+ 10 y))) + (collecting (10+ j))))))) + + ((named outer) + (list i '((1 2) (3 4) (5))) + (loopy-test-escape + (loopy-iter (listing j i) + (at outer + (cl-flet ((10+ (y) (+ 10 y))) + (collecting (funcall #'10+ j))))))) + + ((named outer) + (list i '((1 2) (3 4) (5))) + (loopy-test-escape + (loopy-iter (listing j i) + (cl-flet ((10+ (y) (+ 10 y))) + (at outer + (collecting (10+ j))))))) + + ((named outer) + (list i '((1 2) (3 4) (5))) + (loopy-test-escape + (loopy-iter (listing j i) + (cl-flet ((10+ (y) (+ 10 y))) + (at outer + (collecting (10+ j))))))) + + ((named outer) + (list i '((1 2) (3 4) (5))) + (loopy-test-escape + (loopy-iter (listing j i) + (at outer + (cl-flet ((10+ (y) (+ 10 y))) + (collecting (funcall #'10+ j)))))))] + :loopy t + :iter-keyword (list collect) + :iter-bare ((list . listing) + (collect . collecting))) + +(loopy-deftest iter-list-in-top-level-expr + :doc "Macros that are required to be at the top level should not consider +a sub-expression as the top level." + :error loopy-iteration-in-sub-level + :macroexpand t + :body ((let ((var 1)) + (listing i '(1 2 3 4 5))) + (collecting i)) + :iter-keyword (listing collecting) + :iter-bare t) + ;; Local Variables: ;; End: ;; LocalWords: destructurings backquote