15.6.2.1909.20 NAME>INTERPRET name-to-interpret TOOLS EXT

For a standard program, this is usually a restriction rather than an entitlement: It has to cope with the possibility of NAME>INTERPRET returning 0. Anyway, here is an example of how to deal with that: A user-defined text interpreter:

... parse-name 2dup find-name if nip nip
  state @ if
    name>compile
  else
    name>interpret dup 0= -14 and throw
  then
  execute
else
  ... deal with numbers
then

My point is that this would only be useful in a system that has some way to explicitly mark definitions as having no interpretation semantics.

The standard defines words with undefined interpretation semantics, and some systems implement this in some way; that's the reason for allowing NAME>INTERPRET to return 0. If you know your system never returns 0 from NAME>INTERPRET, you do not need to write the program in that way, but for portable programs, you have to take this possibility into account.

I agree, the possibility of the zero result in name>interpret is only a restriction for programs (see also my rationale in comment #1111). But what does this possibility provide for systems? Nothing! Because in any case a system can have an internal word that returns xt|0 if it needs that.

It seems, a better choice was to make name>interpret always return some xt.

Some systems (e.g., Gforth-0.7) implement words (e.g., if) without interpretation semantics; what should these systems do when the nt of if is passed to name>interpret? The standard says that it returns 0. If it did not say that, what else should it say for that case?

If a system implements interpretation semantics for if (e.g., Gforth-1.0, which makes them the same as compilation semantics), name>interpret of course returns an xt for those interpretation semantics.

So, as a program, you cannot use name>interpret to detect that the standard does not define the interpretation semantics, but you can use it to detect that the system at hand does not define interpretation semantics. What it does with that information is up to the program; in the text interpreter example above a -14 throw was performed, but, e.g., an implementation of find will probably not do that.

Some systems (e.g., Gforth-0.7) implement words (e.g., if) without interpretation semantics; what should these systems do when the nt of if is passed to name>interpret? The standard says that it returns 0.

The standard says "If nt has no interpretation semantics, NAME>INTERPRET returns 0". This is an unclear formulation, since it's impossible for a Forth system not to have interpretation semantics for a word (see below); and if one means the standard, one should say "interpretation semantics are not defined by this standard".

Anyway, the standard allows to return 0 in this case, but it also allows to return an xt of a system-specific behavior.

If it did not say that, what else should it say for that case?

I think the standard could say that in this case name>interpret shall return an xt for a system-defined behavior (the system anyway shows some behavior).

An example of wording (based on my proposal 1110):

• xt identifies the execution semantics of the word identified by nt. When this xt is executed in interpretation state, the interpretation semantics for the word are performed. If interpretation semantics for the word are not defined by this standard, the system-defined interpretation semantics are performed.
  An ambiguous condition exists in any of the following conditions:
  • interpretation semantics for the word are not defined by this standard and xt is executed;
  • execution semantics of the word are not defined by this standard and xt is executed in compilation state;

What does Gforth-0.7 do when its text interpreter encounters if in interpretation state? This behavior is the implemented interpretation semantics for if. And name>interpret may return an xt that shows this behavior in interpretation state, also it may return [: -14 throw ;] regardless what behavior is shown.

And no standard program can have any problem due to this behavior of name>interpret. Right?

but you can use it to detect that the system at hand does not define interpretation semantics.

I would say it does not define an execution token for the word (and for that semantics). Because some interpretation semantics are actually defined by the system anyway (for example, it shows an error message and does abort).

What it does with that information is up to the program; in the text interpreter example above a -14 throw was performed, but, e.g., an implementation of find will probably not do that.

find in interpretation state may return the same xt that name>interpret returns.

If you want to propose words that are not standardized, the way to do it is to write a proposal. But note that words for which there is little common practice are very unlikely to become standard eventually.

So the first thing you should do is to look at common practice. Gforth (development version) has words that work on the most recent definition:

• SET-EXECUTE ( ca -- ), but it takes a machine-code address and is unlikely to be standardizable.

• SET-DOES> ( xt -- ), which changes the most recent definition like DOES> does.

• DEFINER! ( definer xt -- ), but it is obsolete (will be removed in a future version of Gforth); it goes with >DEFINER ( xt -- definer ) for getting a definer.

• SET-TO ( xt -- ), but it takes the xt of a to-table that you have to define with TO-TABLE: (for also supporting +TO etc.). These words are still under discussion among Gforth developers.

You can make a definition "most recent" with MAKE-LATEST, but note that if you change a definition after it has already been used, the behaviour coming out of earlier uses may or may not change. For the thinking behind these words, you may want to read The new Gforth Header and/or watch the video.

VFX has a word for defining the interpretation semantics, but AFAIK not for TO.

For other Forth systems I am not aware that they have words for this kind of stuff, and it probably requires significant changes to add something like SET-TO (probably in VFX, too).

Thank you Anton. I've thought proposals, but as you said I'd need to check common practice, plus I want to make sure I'm not talking nonsense.

I've implemented this in my (Scheme-based) Forth, and it appears to ... just work. I kind of like it, as it allows the users to define TO-handlers in pure FORTH:

\ NAME>TO ( nt -- to-xt )  \ implementation-specific
\ NAME<TO ( to-xt nt -- )  \ implementation-specific, see do_to_value below for to-xt stack signature
: TO
  PARSE-NAME DUP 0= IF -16 THROW THEN
  FIND-NAME  DUP 0= IF -13 THROW THEN
  DUP NAME>TO  \ ( nt to-xt -- )
  STATE @ IF SWAP POSTPONE LITERAL COMPILE, ELSE EXECUTE THEN ;
IMMEDIATE

: do_to_value ( x nt -- ) NAME>INTERPRET >BODY ! ;
: VALUE >IN @ PARSE-NAME 2>R
              >IN ! CREATE , 
              ['] do_to_value 2R> FIND-NAME NAME<TO
              DOES> @ ;

(sorry for spamming an unrelated word... this should have been in the comment section for TO, but I don't know how to move it)

Classic single-xt approach

In the classic single-xt approach, a Forth system defines execution semantics for every word, and these semantics are identified by an execution token (xt) that is associated with the word.

So, in the classic single-xt approach, to perform the observable interpretation semantics of a word we should perform its execution semantics in interpretation state.

To perform the observable interpretation semantics of a word regardless of the current state, we should:

• if the system is in interpretation state: perform the execution semantics;
• if the system is in compilation state: leave compilation state (i.e., enter compilation state), perform the execution semantics, enter compilation state.

Similar, to perform the observable compilation semantics of a word regardless of the current state, we should:

• if the system is in compilation state: perform the execution semantics;
• if the system is in interpretation state: enter compilation state, perform the execution semantics, leave compilation state (i.e., enter interpretation state).

To perform for a word the observable interpretation semantics if the Forth system in interpretation state, and the observable compilation semantics if the Forth system in compilation state we should simply perform the execution semantics of the word.

Any standard system

At first glance, one might think that to perform the observable interpretation semantics for a word regardless of the state it is sufficient to simply execute the xt returned by name>interpret for the word's nt. But this is not true on most Forth systems.

Some examples:

: test1 ( -- flag ) state @ 0<> ;
: test2 ( --  -1 | ) s" true" evaluate ;

These words are ordinary words, which are defined by specifying their execution semantics. Note that in both cases the execution semantics depends on the system's state.

The observable interpretation semantics of test1 are to place 0 on the data stack.

The observable interpretation semantics of test2 are to find the word "true" and perform its interpretation semantics, that are (in a standard system) to place -1 on the data stack, and if the word is not found, throw exception -13.

On most Forth systems, executing of the xt returned by name>interpret for these words produces different results depending on the state. Thus, to perform the observable interpretation semantics for these words, we should execute an xt returned by name>interpret in interpretation state only.

What name>interpret returns

If we admit that the above behavior of name>interpret is desired, then there are no difference with the single-xt approach: in the general case, we should set interpretation state to perform the observable interpretation semantics for a word (see the steps in the section "Classic single-xt approach" above).

Consequently, we can admit that name>interpret always returns an xt that identifies the execution semantics of the word, because:

1. it is true for any user-defined word (except synonyms for standard words that are considered the same as the corresponding standard words);
2. it is true for ordinary standard words;
3. it is true for immediate standard words;
4. it is true in the cases when special interpretation semantics are defined for a standard word, because the execution semantics are not defined in these cases, thus, a returned xt can be considered as an identifier for the system-dependent execution semantics of the word, but executing of this xt in interpretation state shall exhibit the observable interpretation semantics for the word, which are specified by the standard;
5. it is true in the cases when interpretation semantics are undefined for a standard word, because obtaining the execution token of the word is ambiguous (at the moment), and so a returned xt can be considered as an identifier for the system-dependent execution semantics of the word, but executing of this xt in interpretation state shall exhibit the observable interpretation semantics for the word, which are system-defined.

There are no other cases. This applies to single-xt systems as well as to dual-xt systems.

Standard language

In all cases when special interpretation semantics or special compilation semantics are defined for a standard word, they are observable interpretation semantics or observable compilation semantics for the word correspondingly — and this is as it must be, unless we want to impose a particular implementation.

Also, by the definitions, the Forth text interpreter performs the observable interpretation semantics for the word when in interpretation state, and the observable compilation semantics for the word when in compilation state.

There are only few places in the normative parts of the standard where a reader can be confused whether observable semantics are meant or something else.

For example, in 3.4.3.2 Interpretation semantics: "unless otherwise specified [...], the interpretation semantics of a Forth definition are its execution semantics". Obviously, in this case it is not about observable interpretation semantics. The observable interpretation semantics in this case are to perform the execution semantics in interpretation state. I think, that is an omission — that item should describe observable interpretation semantics.

So, in few places the term interpretation semantics does not mean observable interpretation semantics, whereas in most cases this term means observable interpretation semantics. This is an inconsistency that produces many problems.

I think, the wording should be adjusted so that everywhere the term interpretation semantics means observable interpretation semantics. Similar for "compilation semantics". This will eliminate some confusing, many questions and disputes.

There is a typo in my above post, in the section "Classic single-xt approach":

if the system is in compilation state: leave compilation state (i.e., enter compilationinterpretation state), perform the execution semantics, enter compilation state.

I plan to refactor and/or update some my proposals once we reach consensus on this topic (if any). Among them:

• [129] NAME>INTERPRET wording
• [122] Clarify FIND, more classic approach
• [163] Tick and undefined execution semantics
• [251] Clarification for execution token
• [249] Revert rewording the term "execution token"
• [292] Update rationale for SLITERAL

The compilation semantics is more complicated: If the word is immediate, compilation semantics in the single-xt system is to execute the word in compilation state; if it is non-immediate, it is COMPILE, the word. The information whether the word is immediate is not part of the xt. Therefore, the more interesting part is NAME>COMPILE, which converts a name token to a tuple on the stack which can be executed to get the compilation semantics. There's still the need on single-xt systems to be in compilation state, as the immediate word could be state-smart.

As current pre-1.0-Gforth shows, you can have an xt as single identifier for a word and executing that xt always gives you the interpretation semantics; in the case that there is some other compilation semantics, NAME>COMPILE replaces that xt with another one.

What we might need instead of NAME>INTERPRET EXECUTE and NAME>COMPILE EXEUCTE might be a word like COMPILE-XT, which does whatever is needed to get the compilation semantics of a word. It's not the same as COMPILE,.

The compilation semantics is more complicated: If the word is immediate, compilation semantics in the single-xt system is to execute the word in compilation state; if it is non-immediate, it is COMPILE, the word.

Sure. It's an oversight that I wrote the part about only the immediate words, and did not make any note about htat (I thought only about the case of unusual words at that moment). But you get the idea.

What we might need instead of NAME>INTERPRET EXECUTE and NAME>COMPILE EXEUCTE might be a word like COMPILE-XT, which does whatever is needed to get the compilation semantics of a word. It's not the same as COMPILE,.

Do you mean something like this?

: compile-nt ( i*x nt -- j*x )
  name>compile execute
   \ or
   \ name>compile execute-compiling \ it's ensures that xt executed in compilation state
;

I think, the wording should be adjusted so that everywhere the term interpretation semantics means observable interpretation semantics. Similar for "compilation semantics". This will eliminate some confusing, many questions and disputes.

I have prepared a proposal to fix Incorrect use of semantics terms.

@BerndPaysan wrote:

As current pre-1.0-Gforth shows, you can have an xt as single identifier for a word and executing that xt always gives you the interpretation semantics;

How to perform the observable interpretation semantics from this xt in Gforth? In general case, should I set interpretation state or not?

This is obvious that for some words I can execute this xt in compilation state and that performs the observable interpretation semantics. But not for words test1 and test2 defined above, as I can see.

If I should set the interpretation state in the general case, what does this follow from? Shouldn't this be clear from the section 15.6.2.1909.20 (along with other sections)?