About POSTPONE semantics in edge cases

[ruv]

Preceding part: How postpone should work.

Introduction

In the section "2.1 Definitions of terms" of the standard (both Forth-94 and Forth-2012 versions) we can find the following definition for the compilation semantics term:

compilation semantics: The behavior of a Forth definition when its name is encountered by the text interpreter in compilation state.

An obvious premise is that any behavior can be observed by some detectable side effects (including changes in the stacks). If, in some conditions, no effects are detected, then the behavior under those conditions is to do nothing (take no actions).

Since a definition of a term implies equivalence of the left and right parts (the definiendum and the definiens), we can state:

• (behavior ⟹ compilation semantics) If some effect can be detected when a name is encountered by the text interpreter in compilation state, then this effect is a part of the compilation semantics for the corresponding word.
• (compilation semantics ⟹ behavior) If some effect is a part of the compilation semantics for a word, then this effect can be detected when the name of this word is encountered by the text interpreter in compilation state.

Exploration #

Now we can explore one interesting thing.

Let's consider a word x] that is defined as follows:

: x] ( -- )  ] ; immediate

The word "]" (right-bracket) is an ordinary word that enters compilation state.

So what are the compilation semantics for x]?

This is probably quite unexpected, but no side effect can be detected by a standard program when x] is encountered (and processed) by the Forth text interpreter in compilation state. Hence, the compilation semantics for x] are to do nothing!

Now let's consider the ordinary word y], which is defined as follows:

: y] ( -- )  postpone x] ;

The compilation semantics of postpone are to parse a lexeme (that is a word name), find the corresponding word, and append the compilation semantics of the found word to the current definition.

Hence, the phrase postpone x] above during compilation appends the compilation semantics of x] to the execution semantics of the word y]. Since the compilation semantics for x] are to do nothing, semantically nothing is appended to y].

So, what are the execution semantics for y]? They are to do nothing! And hence, the interpretation semantics for y] are to do nothing too.

Excusing #

Now let's test^[1] your Forth system:

: x] ( -- )  ] ; immediate
: y] ( -- )  postpone x] ;
y] 123 . \ should print 123

The last line should print 123. But on most systems it prints nothing or throws an exception. Do they have an excuse for that? The only possible excuse is the official TC answer to the RFI Q99-027 (raw copy). This answer means that a standard program is not allowed to perform compilation semantics in interpretation state.

So, according to A99-027, an ambiguous condition exists when y] is performed in interpretation state. And then, a Forth system is allowed to do anything in this case.

But if your standard Forth system doesn't rely on A99-027, then this test should print 123 according to the normative text of the standard.

^[1] This test (in a slightly different form) was first published in comp.lang.forth by Anton Ertl. He and me both supposed that y] should enter compilation state. But later I realized that this was incorrect.

Solution #

A standard compliant polyfill for postpone can be defined via find-name and name>compile, or via a full-fledged find as follows:

[undefined] lit, [if]
  : lit, ( x -- ) postpone literal ;
[then]
synonym parse-lexeme parse-name
: compilation  ( -- flag ) state @ 0<> ;
: enter-compilation  ( -- )           ] ;
: leave-compilation  ( -- )  postpone [ ;
: execute-compiling ( i*x xt -- j*x ) \ or "execute-compilatingly"
  compilation    if  execute  exit  then
  enter-compilation  execute  leave-compilation
;
[defined] find-name [defined] name>compile and [if]
: postpone ( "name" -- )
  parse-lexeme find-name dup 0= -13 and throw name>compile ( x xt-compiler )
  compilation if swap lit, lit, ['] execute-compiling compile, exit then execute-compiling
; immediate
[else]
: postpone ( "name" -- )
  bl word ['] find execute-compiling dup 0= -13 and throw 1 = ( xt flag-special )
  if ['] execute-compiling else ['] compile, then ( xt xt-compiler )
  compilation if swap lit, compile, exit then execute
; immediate
[then]

This implementation takes into account that the behavior of the phrase name>compile ( x xt-compiler ) execute may be STATE-dependent (since it can be STATE-dependent for some user-defined words), therefore execute-compiling is used in this case too.

This implementation also defines the interpretation semantics for postpone, that are to just perform the compilation semantics of the parsed argument.

See also an implementation for postpone that doesn't rely on the previous postpone in my gist.

NB: such implementation for postpone will correctly work for any immediate state-dependent word, including definitions (or redefinitions) for standard dual-semantics words. But some non standard programs can work incorrectly, if they rely on A99-027 and on some particular behavior in the case of a corresponding ambiguous condition.

Questions or objections? :)

[mattsteeldue]

I think there is a false premise in one of your first sentences, when you say:

Any behavior can be observed via some detectable side effects (including changes in the stacks). If, in some conditions, nothing effects are detected, then this behavior is to do nothing in these conditions.

But, this is incorrect: If no effect is detected, it does not imply that the behavior is "to do nothing". The opposite implication is obviously true, though.

We all agree that the word ] does "Enter compilation state" but, based on how STATE is defined, this means "it [also] sets a suitable non-zero value to STATE variable".

Making x] to be IMMEDIATE , that is forcing its compilation semantics to perform its execution semantics does not change its behavior and it will continue to eventually invoke ] .

In the example

 : z] ( -- ) x] ;

the word x] is encountered during compilation state (i.e. while STATE already has such a non-zero value), it sets STATE again to the very same non-zero value, so that - as you said and I agree with - no apparent effect can be detected and it seems to "do nothing".

In your example, giving at prompt

 y] 123 . 

when y] is executed, it invokes x] which enters compilation state in the middle of nothing that in turn fools the Forth interpreter which perhaps tries to compile 123 and . nobody knows where. This is an ambiguous condition IMO.

Regards.

[ruv]

If no effect is detected, it does not imply that the behavior is "to do nothing".

What make you think so?
If you mean that some effect can be undetected by a mistake, let's take on a more strict wording:

• If, in some conditions, no effect can be detected, it does imply that the behavior is observationally equivalent to doing nothing in these conditions.

If you still claim it's incorrect, you probably can provide a counterexample when some undetectable behavior is not observationally equivalent to doing nothing.

---

it sets STATE again to the very same non-zero value

How do you know?

Setting the same value is not distinguishable from doing nothing. Moreover, the word ] may be actually implemented to not set STATE in this case, e.g.:

: ] ( -- ) state @ 0= if 1 state ! then ; 

---

when y] is executed, it invokes x]

It's incorrect. y] does not invoke x], it performs the compilation semantics of x], which are to do nothing.

If you think otherwise, please provide a test case that shows some side effect when the compilation semantics of x] are performed.

NB: since this test case shows what should be appended by postpone, you cannot use postpone itself to show that.

[ruv]

Just for reference.

The word ] can be also implemented in such a way that it doesn't check state even under the hood and literally does nothing in compilation state.

defer (leave-compilation)    defer leave-compilation
defer (enter-compilation)    defer enter-compilation

:noname \ (leave-compilation)
  state off   ['] noop is leave-compilation  ['] (enter-compilation) is enter-compilation
;   is (leave-compilation)

:noname \ (enter-compilation)
  state on    ['] noop is enter-compilation  ['] (leave-compilation) is leave-compilation
;   is (enter-compilation)

(leave-compilation) \ init

: [   leave-compilation ; immediate
: ]   enter-compilation ;

[ruv]

The reason for the different understanding of how postpone shall treat immediate words lies in the following definition from the section 2.1 Definitions of terms:

immediate word: A Forth word whose compilation semantics are to perform its execution semantics.

The reader could think that this definition for the term immediate word also defines another meaning for the term compilation semantics, which applies to immediate words only. But this is not true, because every term is defined separately and explicitly. In this definition the term compilation semantics is used rather than defined.

The term compilation semantics is still the same:

compilation semantics: The behavior of a Forth definition when its name is encountered by the text interpreter in compilation state.

I have prepared a proposal to make the definition for the term immediate word more clear.