Digest #346 2026-09-12
Contributions
requestClarification - What should happen when calling ALSO when the search order is empty?
It's mentioned that there is only an ambiguous condition if the search order is full. But I don't understand what would happen if the search order is empty.
The proposed implementation doesn't seem to handle that case gracefully.
Replies
Test test 123
My vote did not work when I wrote my comment. Here it is again. At the moment I will implement what is needed for my system!
It would be possible to add a "Categories:" entry to each word that lists the set of categories the word belongs to, with the categories being included in the sorted index (forth.wds). I would suggest a standard set of categories and a new annex witch lists the words in alphabetic order with a set of categories for each word. My preferred option would be to provide a list of words that belong to each category. I think this would be more useful that than alphabetic list.
Yes I am My.
I have ussed I will not implement!
proposal - New words: latest-name and latest-name-in
How about
latest-name-int ( wid -- nt | 0 )
latest-name ( -- nt | 0 )
and include the following the in the rationale for latest-name:
: recent-name ( -- nt ) latest-name ?dup not if -80 throw then ;
proposal - Exclude zero from the data types that are identifiers
I find in the 2007 Notes:
0 as NIL: [Federico de Ceballos'] proto-RfD changes HERE and ALLOCATE. Some alternative suggestions were made, in particular changing the definition of contiguous region.
I did not find anything substantive about the topic in other notes, minutes, and other places where I looked.
Author
Ruv
Change Log
(the latest at the top)
- 2026-06-04
Narrow down ambiguous conditions.
Clarify meaning of terms in the typical usage examples.
Improve wording in some places.
Remove ambiguous conditions for
postponeand[compile]. Add more test cases. - 2025-10-03 Correct some normative statements; add ambiguous conditions; make better wording. Add more rationale. Correct some typos.
- 2025-09-30 Add some rationale; correct some typos and grammar mistakes; minor rewording; add consequences.
- 2025-09-29 Better wording in some places; corrections; update in ambiguous conditions.
- 2025-09-28 Huge update; incorporate proposals [249], [212] (partially), [163], [122] (partially); add tests. According to my comment on 2024-09-24, this was planned.
- 2022-09-19 explicitly allow a short formula, describe what it means, better wording, fix some typos
- 2022-08-13 Initial version
Preceding history
(the latest at the top)
2022-08-12 [249] Revert rewording the term "execution token" (proposal, retracted on 2025-09-28).
2021-09-08 [157] Reword the term "execution token" (proposal, accepted on 2021-09-13).
2021-09-08 [212] Tick and undefined execution semantics - 2 (proposal, considered on 2024-09-26).
2020-10-29 [163] Tick and undefined execution semantics (proposal, retracted on 2025-09-12)
2020-09-03 [157] Reword the term "execution token" (proposal, replaced on 2021-09-08; an attempt to solve the problem by changing the term)
2020-02-20 [129]
NAME>INTERPRETwording (proposal, in progress; indication of a problem)2019-10-08 [122] Clarify FIND, more classic approach (proposal, in progress)
Problem
By the definition of the term "execution token" in Forth-94 and Forth-2012, it's a value that identifies execution semantics. Can such a value identify other behavior, e.g., some interpretation semantics or compilation semantics? It's unclear at the first glance.
Another problem is that, following unfortunate change to the term "execution token" in the very quickly accepted proposal [157], the standard does not formally state that an execution token identifies anything at all.
Yet another problem is that it is unclear what behavior are identified by the execution token of a word whose execution semantics or interpretation semantics are not specified by the standard.
There are five words for wich the standard defines interpretation semantics
but not execution semantics:
s", s\", to, is, action-of.
Obtaining execution token (and the "tick" operation) for the last three of them is prohibited,
while the behavior identified by the execution tokens of the first two
is unclear in edge cases (it is neither explicitly defined nor undefined in compilation state).
Solution
Actually, an execution token can identify (and does identify) other semantics too, but only if they are equivalent to the execution semantics that this token also identifies.
Example 1
:noname postpone if ; ( xt )
- xt, which is left on the stack, identifies the execution semantics of this unnamed Forth definition.
- The execution semantics of this definition
are equivalent
to the compilation semantics for
if(this follows from the standard). - Then, this xt also identifies the compilation semantics for
if(identifies in terms of behavior, not object code).
Example 2
: foo postpone if ;
- xt of
fooidentifies the execution semantics offoo. - The execution semantics of
fooare equivalent to the compilation semantics forif(this follows from the standard). - Then, this xt also identifies the compilation semantics for
if.
Note that the Forth system may provide
system-dependent execution semantics for if
that are not equivalent to the execution semantics of foo.
Reasoning
Thus, for any execution token there exists at least one Forth definition (named or unnamed) the execution semantics of which are identified by this execution token. So, in any case, an execution token always identifies some execution semantics, but accidentally (or intentionally) these semantics can be equivalent to some interpretation semantics, or some compilation semantics, and then it identifies them too. It's unnecessary that they connected to the same Forth definition. Also, consequently, it's impossible that an execution token identifies some compilation semantics, or some interpretation semantics, but doesn't identify the equivalent execution semantics.
Note that there are cases where the semantics cannot be identified by an execution token in a Forth system, because the implementation of the system does not have or cannot have a Forth definition with equivalent execution semantics.
Examples of semantics that cannot be identified by any execution token:
- in some systems
(in which the phrase
postpone literal postpone executeis equivalent tocompile,), the formally specified execution semantics of>r; - in some systems
(where FVM does not have access to the underlying return stack,
e.g. WAForth),
the initiation semantics of
:noname; - typically, the run-time semantics of
if(which are appended to the current definition by the compilation semantics ofif);
Of course, the standard allows such implementations and disallows programs to obtain an execution token of the corresponding semantics, or even does not provide a way to obtain it.
Roadmap
To solve the initial problem, we can:
- formally and explicitly state the basics described above,
- specify what particular semantics are identified by the execution token of a word (in which cases they are defined by the standard and in which by the implementation, and to what extent),
- update the definition of the "execution token" term to say what these tokens identify.
Clearly defining execution tokens for edge cases
eliminates ambiguity
when obtaining them for the words to, is, and action-of.
Furthermore, the ambiguous conditions
on applying postpone and [compile] to these words
can be removed.
These conditions were inherited from the word to in Forth-94,
where they were introduced to provide a room
for both "parsing" and "non-parsing" implementations.
In reality "non-parsing" implementations were impossible
because a child of value is allowed to be immediate.
In Forth-2012, there is also no room for implementation variance of postpone
due to name>compile.
A close topic was also discussed in comp.lang.forth in 2024,
Subject: "single-xt approach in the standard".
- Message-ID:
<2024Sep23.185202@mips.complang.tuwien.ac.at>- Date: 2024-09-23 16:52:02 +00:00
- Message-ID:
<vd0e11$3itr4$1@dont-email.me>- Date: 2024-09-25 07:27:28 +00:00
Typical use
"The execution semantics identified by xt are equivalent to the interpretation semantics of
BAZ"- This seems pretty clear.
- Note that this means that the execution semantics identified by xt are independent of STATE.
"xt identifies the interpretation semantics for the word
BAZ"- This means that the execution token xt identifies the execution semantics which are equivalent to the interpretation semantics for the word
BAZ. - Also, this means that the execution semantics identified by xt are independent of STATE.
- At the same time, the execution semantics of the word
BAZmay differ from the execution semantics identified by this xt.
- This means that the execution token xt identifies the execution semantics which are equivalent to the interpretation semantics for the word
"xt identifies the compilation semantics for the word
FOO"- This means that the execution token xt identifies the execution semantics which are equivalent to the compilation semantics for the word
FOO. - Also, this means that the execution semantics identified by xt are independent of STATE.
- At the same time, the execution semantics of the word
FOOmay differ from the execution semantics identified by this xt.
- This means that the execution token xt identifies the execution semantics which are equivalent to the compilation semantics for the word
"xt of the word
BAR"- This means that xt identifies the execution semantics of the word
BAR. - Whether this xt also identifies
the compilation semantics,
or the interpretation semantics,
or both of them, or neither,
for the word
BARdepends on the wordBAR(on how it is defined or specified). - Regardless of how
BARis defined, executing xt in interpretation state performs the interpretation semantics forBAR.
- This means that xt identifies the execution semantics of the word
Incorrect use
Actually, the standard contains only one place
where the "execution token" term is used ambiguously
in a normative part —
the glossary entry for FIND.
The problem is that it says that FIND returns
the execution token of the found Forth definition
("its execution token", so it should identify
the execution semantics of that definition),
but:
FINDmay return two different execution tokens (one while compiling and another while not compiling) for the same input string (and, formally, the same Forth definition), which may identify different semantics; then, formally, at least one of them does not identify the execution semantics of the found definition (contrary to the semantic description).
In the glossary entry for
NAME>INTERPRET,
the language is just slightly non normative,
since it uses the form "xt represents" instead of the form "xt identifies".
These glossary entries also have some other problems, so they should be corrected anyway; my other proposals on this matter are in progress.
Proposal
Update "execution token" term
In the section 2.1 Definitions of terms, change (as of 2021-09-13):
execution token: A value that can be passed to
EXECUTE(6.1.1370)
into
execution token: A value that identifies the execution semantics of a definition.
Rationale
- This is necessary to correctly describe the "execution token" data type.
Update "execution token" data type description
In the section 3.1.3.5 Execution tokens, make the following changes.
1. Add the following paragraphs to the beginning
For any valid execution token in the system, there is at least one Forth definition (named or unnamed) whose execution semantics are identified by that execution token.
The execution semantics identified by an execution token may be equivalent to the interpretation semantics, compilation semantics, or other semantics for some named Forth definition (or definitions). In such cases, the execution token also identifies those interpretation, compilation, or other semantics.
The system does not need to identify every specified semantics by any execution token.
The execution token of a Forth definition, if available, identifies the execution semantics that are either specified by this standard for that definition or are implementation dependent (if allowed).
If the interpretation semantics for a Forth definition are defined by this standard, the execution token of that definition shall be available.
Rationale
- We use the clause "valid" in "any valid execution token"
because an execution token may become invalid
after using words like
forgetand children ofmarker. - The "if available" clause is used since
find(in interpretation state) andname>interpretmay return zero for some existing (but not user-defined) words; this effectively means that the execution token is not available for those words, andsearch-wordlistshould also return zero for them. - The last paragraph guarantees that any word that is allowed to be Ticked has an execution token.
2. Add the following paragraph at the end
See also: A.3.1.3.5 Execution tokens.
Update "Execution semantics" notion
In the section 3.4.3.1 Execution semantics, make the following changes.
1. Change the first paragraph
The execution semantics of
eachForth definition are specified in an "Execution:" section of its glossary entry. When a definition has only one specified behavior, the label is omitted.
into
The execution semantics of a Forth definition are specified in an "Execution:" section of its glossary entry. When a definition has only one specified behavior, the label is omitted.
Rationale
- For some words, the execution semantics are not specified by the standard.
2. Add the following paragraphs after the first one
If the execution semantics for a Forth definition are specified by this standard and the glossary entry of that definition does not have an "Interpretation:" section, the execution token of that definition identifies the specified execution semantics. Otherwise, the execution token of that definition, if available, identifies the implementation dependent execution semantics.
Rationale
- We have to rely on the absence of an "Interpretation:" section
to refer to ordinary and immediate words
until words like
>rwill be defined using a "Run-Time:" section instead of an "Execution:" section in their glossary entries (commented on 2019-06-21, 2020-08-30). - We use the term "implementation dependent" rather than "implementation defined" because this allows implementors to avoid documenting the provided execution semantics of standard words whose execution semantics are not specified by the standard.
The execution semantics identified by the execution token of a Forth definition, when they are performed in interpretation state, shall perform the interpretation semantics of that definition.
Rationale
- This guarantees that, while interpreting,
' foo executeis equivalent tofoo, even when the execution token (xt) offooidentifies implementation dependent execution semantics.
An ambiguous condition exists if the execution semantics of a Forth definition are not specified by this standard and its execution token is executed in compilation state under conditions other than those specified in this standard.
Rationale
- That behavior in compilation state might depend on the implementation.
Namely, if such a word (e.g.
s") is implemented using the single-xt approach (all classic Forth systems), this performs the compilation semantics for the word, but if such a word (e.g.s") is implemented using the dual-xt approach, this typically performs the interpretation semantics for the word. - An example of conditions under which
execution in compilation state
of the execution token of such a word (e.g.
s") is not ambiguous: iffindin compilation state returns( xt1 1 )for a word, and, coincidentally, xt1 is the execution token of the word.
An ambiguous condition exists if the interpretation semantics of a Forth definition are undefined by this standard and its execution token is executed under conditions other than those specified in this standard.
Rationale
- The execution semantics of the definition
:noname s" if" forth-worldlist search-wordlist if execute then ;is ambiguous, regardless of whether they are performed while interpreting or compiling, because they execute (at least on some systems) the execution token of the wordif, whose interpretation semantics are undefined by this standard. - In the same time, the execution semantics of the definition
:noname c" if" find 1 = if execute then ;
in compilation state is not ambiguous even if
find in this case returns the same execution token as ['] if
(which identifies the implementation dependent execution semantics of if).
Update ambiguous conditions
In the section 4.1.2 Ambiguous conditions,
replace the phrase:
attempting to obtain the execution token, (e.g., with 6.1.0070
', 6.1.1550FIND, etc. of a definition with undefined interpretation semantics;
with the phrase:
attempting to obtain, using the words 6.1.0070
'or 6.1.2510['], the execution token of a definition with undefined interpretation semantics;
Rationale
- "
find" and "etc." are excluded from the list, becausefind(in interpretation state),search-wordlist, andname>interpretreturn either the execution token of the word or zero, and there is no ambiguous condition in this regard.
Update glossary entries
In the glossary entries
6.2.2295 TO,
6.2.1725 IS,
6.2.0698 ACTION-OF,
remove the sentence that starts with:
An ambiguous condition exists if any of
POSTPONE,[COMPILE],'or[']are applied
Editorial note: in the first case the form “are applied” is used, in the second and third cases the form “is applied” is used.
Rationale
- Historically, there were
two approaches
to implement
to: "parsing" and "non-parsing". Forth-94 formally specified the "parsing" approach. It also declared an ambiguous condition, perhaps, as a concession to implementations that adhered to the "non-parsing" approach. Note that no similar exception was made for the words", since it always do parsing of the parse area. In Forth-2012, more exceptions for the wordtowere added without obvious ground, and also for the wordsisandaction-of. - It is now specified that the execution tokens of these words identify implementation dependent execution semantics that, when performed in interpretation state, shall perform the specified interpretation semantics for these words.
- Also, since these words shall parse the parse area for their immediate argument,
there is no any reason
for an ambiguous condition
when applying
postponeor[compile]to these words (just as there is no such reason when applying them tos"). - Note that
[compile]is obsolescent.
Consequences
This change specifies all the conditions under wich
an xt returned by search-wordlist,
' (Tick), and ['] (Bracket Tick)
can (or cannot) be used by a standard program.
This change removes prohibition on Ticking the words to, is, action-of,
and specifies that executing of the returned xt in interpretation state
shall perform the interpretation semantics for the word.
Note that executing can be performed directly by execute,
or indirectly by executing of the definition
in which this xt is compiled using compile,.
Also, this change removes prohibition on applying postpone and [compile]
to the words to, is, action-of.
If the system throws an error on postponing these words, or on ticking these words, or does not provide a correct xt for them, it should be updated to be compliant.
Most Forth systems in which these words are parsing words (as the standard requires) already comply with this change.
According to Anton Ertl's testing of five Forth systems
(comment [r885])
in 2022,
iForth 5.0.27 and VfxForth 5.11 did not comply with this change
with respect to the execution tokens of the word to.
In VfxForth 5.43,
the words is and action-of do parsing
(as specified in the standard), but the word to does not.
Another issue is that the execution tokens of these three words,
as well as the word s" (ticking of which is allowed),
are not compiled correctly by compile,.
This change does not affect the existing standard programs. However, it allows the use of certain new techniques (previously disallowed) in standard programs.
This change answers to the following requests for clarification:
- Ambiguous condition in case of undefined execution semantics
- Eliminating ambiguous conditions for Tick (partially)
Testing
See:
The committee asks the author to add test cases, then this could go to call for votes.
On behalf of the committee: This is an interesting idea. However, there are the following problems with it:
- It's not specified how the markup is visible, and the usual editors will not display it in a sensible way.
- No rendering is specified.
- It would need infrastructure changes in other software, such as editors.
- There is no concrete proposal, just a rough idea.
- The testing section should contain tests.
- No changes to the document are proposed (but that only becomes relevant once the proposal should go to formal).
- And of course, even if all of that is fixed, there is still the question of existing practice in standard systems.
On behalf of the committee: We are intrigued and confused by your proposal. We thought a nicer way of doing this would be using create like so:create name_of_word $hexcode , $hexcode , $hexcode ,
And then use something likename_of_word perform
Some of us think that this would be an interesting tool for trying out shellcodes.
If you could please explain in greater detail, what you are trying to solve and provide some more example, we will consider this proposal again. Please move it into informal again, once you are done addressing these issues.
proposal - Forth Standards, Backward Compatibility and modern `state of the art` words with their `historical traditional` counterparts.
On behalf of the committee:
The kind of material you are looking tends to be in Appendix C. We need to update Appendix C for the upcoming version of the standard, and would welcome you to contribute to that.
In particular, C.7 should be summarized in the new standard. A new C.8 (differences between Forth-2012 and Forth-202x) needs to be written along the lines of your suggestions.
Please contact the editorial team at anton@mips.complang.tuwien.ac.at
If you obtain the nt of one of these words, NAME>INTERPRET and NAME>COMPILE will always give you an xt/a pair of xts, which can be used to get the interpretation or compilation semantics (possibly only if in interpretation or compilation state).
What you can't have is a single xt that gives you both interpretation and compilation semantics. And you can't know if you get the compilation semantics through an xt+EXECUTE or xt+COMPILE,. But every possible action is available either through one or a pair of xts.
Thank you. This is exactly the sort of feedback I was hoping for.
I agree that the proposal needs to progress from an idea to a concrete implementation, with actual tests, before it could be considered formal. I intend to experiment with it in pForth.
One clarification: the lack of a prescribed visual rendering was intentional. I regard the semantic tag as part of the source and color, bold, underline, font changes, audible cues, etc. as alternative presentations of that semantic information. However, I agree that a proposal needs to specify how the semantics are made apparent to the programmer, at least sufficiently to demonstrate a usable implementation.
It also appears that extensible existing editors such as Emacs or Vim could provide an experimental semantic-source mode/plugin, rather than requiring a new editor. They can handle the source representation and provide fonts, styles, rendering, color, etc. according to a programmer's preference. I will investigate that along with the pForth implementation.
I appreciate the checklist. It is encouraging. It gives me concrete things to work on before revising the proposal.
Alternative stack comment:
( addr1 n -- addr2 addr1 )
Character strings are represented on the stack by values of the data type ( c-addr u ). Therefore, to allow both
s" foo bar" bounds
here -3 bounds
, the stack diagram should be ( addr1 u|n -- addr2 addr1 ).
However, since implementations where u ⇒ +n are allowed (provided the minimum range requirements are met), it should probably be added that an ambiguous conditions exists if overflow or underflow occurs.
proposal - Tighten the specification of SYNONYM (version 1)
We discussed this issue at the meeting in 2026-09 and decided to promote this to CfV status, status, in order to hear more feedback from the wider community. Please tell us about your usage, intended usage, or implementation status or plans for this proposal.
requestClarification - Why do we use +n and not u in the stack diagram for n>r and nr>
This request for clarification is closed by another comment on 2025-09-29.
I wrote on 2024-06-30
We should use the smallest appropriate data type to describe a stack parameter.
Because arrow type are contravariant in their input and covariant in their output, the most specific arrow type has the most general input type and the most specific output type.
@bernd wrote:
If you obtain the nt of one of these words,
NAME>INTERPRETandNAME>COMPILEwill always give you an xt/a pair of xts, which can be used to get the interpretation or compilation semantics (possibly only if in interpretation or compilation state).
I assume, by "these words" you mean the words s", s\", to, is, and action-of.
However, while my proposal concerns name>interpret, it does not concern name>compile.
If name>interpret returns an xt for a word, then that xt must be equal to one returned by search-wordlist for the same word, and that xt is the execution token of the word. Whether name>compile (or find in compilation state) returns under the top the word's execution token or another execution token depends on the implementation.
NAME>COMPILE allows to obtain execution tokens to describe the compilation action of a word (a pair of xts is needed).
If we talk about what can be identified by execution tokens, we still have the actions of words responding to TO, IS, ACTION-OF and friends. In that case, the only way to get an xt that describes such an action is something like
123 value foo
:noname to foo ;
That :noname definition returns an xt that can be used to assign a new value to foo.
NAME>COMPILEallows to obtain execution tokens to describe the compilation action of a word (a pair of xts is needed).
Yes, name>compile returns a data object of type ( xt1 xt2 ) (or maybe ( x1 xt2 )), which allows us to perform the compilation semantics for the word. However, it does not follow that execution semantics identified by xt1 are equivalent to the compilation semantics of the word. It is possible that they are not equivalent even for words like to or s".
If we talk about what can be identified by execution tokens, we still have the actions of words responding to
TO,IS,ACTION-OFand friends.execute-interpreting
It is possible to create definitions whose execution semantics are equivalent to those actions.
For example, the action of «to» has two formally specified counterparts: interpretation semantics and compilation semantics. The system is not required to provide xts specifically for those actions, but the user can create corresponding definitions:
:noname ['] to execute-interpreting ; ( xt.interpretation-semantics-for-to )
:noname [ s" to" find-name ?found name>compile ] 2literal execute-compiling ; ( xt.compilation-semantics-for-to )
:noname postpone to ; ( xt.compilation-semantics-for-to )
Where execute-compiling and execute-interpreting are defined as follows:
: compilation ( -- flag ) state @ 0<> ;
: enter-compilation ( -- ) ] ;
: leave-compilation ( -- ) postpone [ ;
: execute-compiling ( i*x xt --j*x )
compilation if execute exit then
enter-compilation execute leave-compilation
;
: execute-interpreting ( i*x xt --j*x )
compilation 0= if execute exit then
leave-compilation execute enter-compilation
;
I found the recognizer proposal handy while working on implementing HenceForth, my own flavor of Forth aimed for a niche target environment of the Intcode VM loosely specified in https://adventofcode.com/2019/day/9