Proposal: Clarification for execution token

(the latest at the top)

• 2022-09-19 explicitly allow a short formula, describe what it means, better wording, fix some typos

(the latest at the top)

Actually, an execution token can identify other semantics too, but only if they are equivalent to the execution semantics that this token also identifies.

It is so because for any execution token there exists at least one named or unnamed Forth definition the execution semantics of which are identified by this execution token. So, in any case, an execution token always identifies some execution semantics, but accidentally these semantics can be equivalent to some interpretation semantics, or some compilation semantics, and then it identifies them too. It's unnecessary that they connected to the same Forth definition. Also, consequently, it's impossible that an execution token identifies some compilation semantics, or some interpretation semantics, but doesn't identify the equivalent execution semantics.

The execution semantics of foo are equivalent to the compilation semantics for if.

In the same time, a Forth system may provide system-dependent execution semantics of if that are not equivalent to the execution semantics of foo.

xt, which is left on the stack, identifies the execution semantics of an anonymous Forth definition, and these execution semantics are equivalent to the compilation semantics for if.

Typical use

• "xt identifies the compilation semantics for the word FOO"
  • It means that the execution token xt identifies the execution semantics which are equivalent to the compilation semantics for the word FOO.
  • In the same time the execution semantics of the word FOO can be different form the execution semantics identified by this xt.

• "the execution token for the word BAR"
  • It means that this execution token identifies the execution semantics of the word BAR.

• The execution semantics identified by xt are equivalent to the interpretation semantics for the word BAZ.
  • This seems pretty clear.

Wrong use

Actually, the standard contains only one place where the "execution token" notion is used ambiguously in a normative part — the glossary entry for FIND. Since it says that FIND returns the execution token for a word, but actually this token cannot identify the execution semantics of this word in some cases in dual-xy systems.

In another glossary entry — for NAME>INTERPRET — the language is just slightly non normative, since it uses the form "xt represents" instead of the form "xt identifies".

These entries have not only this but also some other problems, so they should be corrected anyway, and my proposals for them are in progress.

Add into beginning of the section 3.1.3.5 Execution tokens the following paragraphs:

The execution semantics identified by an execution token can be equivalent to the interpretation semantics or compilation semantics for some word, or to some run-time semantics. In such a case this execution token also identifies the corresponding interpretation semantics, compilation semantics, or run-time semantics.

Unless otherwise indicated, the execution token for a named Forth definition identifies the execution semantics for this definition.

• 2025-09-28 Huge update; incorporate proposals [249], [212] (partially), [163], [122] (partially); add tests.

• 2022-08-12 [249] Revert rewording the term "execution token" (proposal, retracted on 2025-09-28).

• 2021-09-08 [157] Reword the term "execution token" (proposal, accepted on 2021-09-13).

• 2021-09-08 [212] Tick and undefined execution semantics - 2 (proposal, considered on 2024-09-26).

• 2020-10-29 [163] Tick and undefined execution semantics (proposal, retracted on 2025-09-12)

• 2020-09-03 An attempt to solve the problem in NAME>INTERPRET by change meaning of "execution token" — [157] Reword the term "execution token"

• 2019-10-08 [122] Clarify FIND, more classic approach (proposal, in progress)

Another problem is that, following unfortunate change to the term "execution token" in the very quickly accepted proposal [157], the standard does not formally state that an execution token identifies anything at all.

Yet another problem is that it is unclear what behavior are identified by the execution token of a word whose execution semantics are not specified by the standard.

Actually, an execution token can identify (and does identify) other semantics too, but only if they are equivalent to the execution semantics that this token also identifies.

Note that there are cases where the semantics cannot be identified by an execution token in a Forth system, because the implementation of the system does not have or cannot have an unnamed (anonymous) definition with equivalent execution semantics.

Some examples of semantics that cannot be identified by an execution token:

• typically, the run-time semantics of if (an instance of);
• in some systems (in which compile, is equivalent to postpone literal postpone execute), the execution semantics of >r;
• in some system (where FVM does not have access to the underlying return stack, e.g. WAForth), the initiation semantics of :noname.

Of course, the standard allows such implementations and disallows programs to obtain an execution token of the corresponding semantics, or even does not provide a way to obtain it.

To solve the initial problem we can formally state these basics explicitly in a normative part, and specify what semantics are identified by the execution token of a word.

Also, we should update the definition of "execution token" term to say what it identifies.

In the same time, a Forth system may provide system-dependent execution semantics for if that are not equivalent to the execution semantics of foo.

xt, which is left on the stack in the second line, identifies the execution semantics of an anonymous Forth definition, and these execution semantics are equivalent to the compilation semantics for if.

- At the same time, the execution semantics of the word `FOO` may differ form the execution semantics identified by this _xt_.

- Note: if the standard does not define interpretation semantics for `BAR`, the execution token of `BAR` could identify some system-specific execution semantics, because an ambiguous condition [could occur (4.1.2)](https://forth-standard.org/standard/doc#doc:ambiguous) when the program obtains the execution token of `BAR`.

• "xt identifies the interpretation semantics for the word BAZ"
  • It means that the execution token xt identifies the execution semantics which are equivalent to the interpretation semantics for the word BAZ.
  • At the same time, the execution semantics of the word BAZ may differ form the execution semantics identified by this xt.

Incorrect use

Actually, the standard contains only one place where the "execution token" term is used ambiguously in a normative part — the glossary entry for FIND. Since it says that FIND returns the execution token for the word ("its execution token", so it should identify the execution semantics of the word), but:

• FIND may return two different tokens (one while compiling and another while not compiling) for the same word, which may identify different semantics (then at least one of them does not identifies the execution semantics of the word);
• in some cases, none of the returned execution tokens identifies the specified execution semantics of the word (for example, for the word >r in some systems).

These glossary entries also have some other problems, so they should be corrected anyway; my other proposals for that are in progress.

Update "execution token" term

In the section 2.1 Definitions of terms, change:

execution token: A value that can be passed to EXECUTE (6.1.1370)

into

execution token: A value that identifies the execution semantics of a definition.

Update "execution token" data type description

In the section 3.1.3.5 Execution tokens, add the following paragraphs to the beginning:

For any valid execution token in the system, there is at least one Forth definition (named or unnamed) whose execution semantics are identified by that execution token.

It it is not required that every specified semantics be identified by some execution token in the system.

The execution token of a Forth definition, if available, identifies the execution semantics of that definition, which are either specified by this standard or implementation dependent (if permitted).

Update "Execution semantics" notion

In the section 3.4.3.1 Execution semantics,

Change the paragraph:

The execution semantics of each Forth definition are specified in an "Execution:" section of its glossary entry. When a definition has only one specified behavior, the label is omitted.

into

The execution semantics of a Forth definition are specified in an "Execution:" section of its glossary entry. When a definition has only one specified behavior, the label is omitted.

Rationale: for some Forth definitions execution semantics are not specified.

After that, add the the following two paragraphs:

If the execution semantics for a Forth definition are specified by this standard and the glossary entry of that definition does not have an "Interpretation:" section, the execution token of that definition identifies the specified execution semantics. Otherwise the execution token of that definition, if available, identifies the implementation dependent execution semantics.

The implementation dependent execution semantics of a Forth definition, when they are performed in interpretation state, shall perform the interpretation semantics of that definition. An ambiguous condition exists if they are performed in compilation state.

Rationale: until such words like >r are defined using an "Execution" section, we have to rely on absence of an "Interpretation:" section to refer to ordinary words (commented on 2019-06-21, 2020-08-30).

Update glossary entries

In the glossary entries 6.2.2295 TO, 6.2.1725 IS, 6.2.0698 ACTION-OF,

replace the phrase:

An ambiguous condition exists if any of POSTPONE, [COMPILE], ' or ['] are applied to TO.

with the phrase:

An ambiguous condition exists if POSTPONE or [COMPILE] are applied to TO.

Testing

t{ ' s"  execute abc"  s" abc"  compare -> 0 }t
t{ 1 value x  2  ' to  execute x  x -> 2 }t
t{ : (to) [ ' to compile, ] ;  3 (to) x  x -> 3 }t

• 2025-09-29 Better wording in some places; corrections; update in ambiguous conditions.

Note that there are cases where the semantics cannot be identified by an execution token in a Forth system, because the implementation of the system does not have or cannot have a Forth definition with equivalent execution semantics.

xt, which is left on the stack in the second line, identifies the execution semantics of an unnamed Forth definition, and these execution semantics are equivalent to the compilation semantics for if.

• The execution semantics identified by xt are equivalent to the interpretation semantics of BAZ.

Since it says that FIND returns the execution token of the Forth definition ("its execution token", so it should identify the execution semantics of that definition), but:

• FIND may return two different tokens (one while compiling and another while not compiling) for the same string (and, formally, the same Forth definition), which may identify different semantics; then at least one of them does not identifies the execution semantics of the definition;

The execution semantics identified by an execution token may be equivalent to the interpretation semantics or compilation semantics for some word, or to some run-time semantics. In such a case this execution token also identifies that interpretation semantics, compilation semantics, or run-time semantics.

The execution token of a Forth definition, if available, identifies the execution semantics that are either specified by this standard for that definition or are implementation dependent (if permitted).

If the interpretation semantics for a Forth definition are defined by this standard, the execution token of that definition shall be available.

Update ambiguous conditions

In the section 4.1.2 Ambiguous conditions,

replace the phrase:

attempting to obtain the execution token, (e.g., with 6.1.0070 ', 6.1.1550 FIND, etc. of a definition with undefined interpretation semantics;

with the phrase:

attempting to obtain the execution token with 6.1.0070 ' or 6.1.2510 ['] of a definition with undefined interpretation semantics;

Rationale: find (in interpretation state) and search-wordlist return either execution token of the word or zero, and there is no ambiguous condition in this regard.

An ambiguous condition exists if any of POSTPONE, [COMPILE], ' or ['] are applied.

An ambiguous condition exists if POSTPONE or [COMPILE] are applied.

• 2025-09-30 Add some rationale; correct some typos and grammar mistakes; minor rewording; add consequences.

According to my comment on 2024-09-24, this was planned.

• 2020-09-03 [157] Reword the term "execution token" (proposal, replaced on 2021-09-08; an attempt to solve the problem by changing the term)

• 2020-02-20 [129] NAME>INTERPRET wording (proposal, in progress; indication of a problem)

Example 1

:noname postpone if ; ( xt )

1. xt, which is left on the stack, identifies the execution semantics of this unnamed Forth definition.
2. The execution semantics of this definition are equivalent to the compilation semantics for if.
3. Then, this xt also identifies the compilation semantics for if.

Example 2

: foo postpone if ;

1. xt of foo identifies the execution semantics of foo.
2. The execution semantics of foo are equivalent to the compilation semantics for if (this follows from the standard).
3. Then, this xt also identifies the compilation semantics for if.

Note that the Forth system may provide system-dependent execution semantics for if that are not equivalent to the execution semantics of foo.

Reasoning

Thus, for any execution token there exists at least one Forth definition (named or unnamed) the execution semantics of which are identified by this execution token. So, in any case, an execution token always identifies some execution semantics, but accidentally (or intentionally) these semantics can be equivalent to some interpretation semantics, or some compilation semantics, and then it identifies them too. It's unnecessary that they connected to the same Forth definition. Also, consequently, it's impossible that an execution token identifies some compilation semantics, or some interpretation semantics, but doesn't identify the equivalent execution semantics.

Examples of semantics that cannot be identified by an execution token:

• typically, the run-time semantics of if (an instance of);
• in some systems (in which the phrase postpone literal postpone execute is equivalent to compile,), the formally specified execution semantics of >r;
• in some systems (where FVM does not have access to the underlying return stack, e.g. WAForth), the initiation semantics of :noname.

Roadmap

To solve the initial problem we can

• formally and explicitly state the basics described above,
• specify what particular semantics are identified by the execution token of a word (in what cases are they defined by the standard and when by the implementation, and to what extent).

• "The execution semantics identified by xt are equivalent to the interpretation semantics of BAZ"
  • This seems pretty clear.

- This means that the execution token _xt_ identifies the execution semantics which are equivalent to the interpretation semantics for the word `BAZ`.

• "xt identifies the compilation semantics for the word FOO"
  • This means that the execution token xt identifies the execution semantics which are equivalent to the compilation semantics for the word FOO.
  • At the same time, the execution semantics of the word FOO may differ from the execution semantics identified by this xt.

• "xt of the word BAR"
  • This means that xt identifies the execution semantics of the word BAR.
  • Whether this xt also identifies the compilation semantics, or the interpretation semantics, or both of them, or neither, for the word BAR depends on the word BAR (on how it is defined or specified).
  • Regardless of how BAR is defined, executing xt in interpretation state performs the interpretation semantics for BAR.

The problem is that it says that FIND returns the execution token of the Forth definition ("its execution token", so it should identify the execution semantics of that definition), but:

• FIND may return two different tokens (one while compiling and another while not compiling) for the same string (and, formally, the same Forth definition), which may identify different semantics; then at least one of them does not identify the execution semantics of the definition (despite the statement).

In the glossary entry for NAME>INTERPRET, the language is just slightly non normative, since it uses the form "xt represents" instead of the form "xt identifies".

These glossary entries also have some other problems, so they should be corrected anyway; my other proposals on this matter are in progress.

change (as of 2021-09-13):

execution token: A value that identifies the execution semantics of a definition.

The execution semantics identified by an execution token may be equivalent to the interpretation semantics, compilation semantics, or other semantics for some named Forth definition. In such cases, the execution token also identifies those interpretation, compilation, or other semantics.

The system does not need to identify every specified semantics by any execution token.

The execution token of a Forth definition, if available, identifies the execution semantics

Rationale

• We use the clause "valid" in "any valid execution token" because an execution token may become invalid after using words like forget and children of marker.
• The "if available" clause is used since find (in interpretation state) and name>interpret may return zero for some exisitng (but not user-defined) words; this effectively means that the execution token is not available for those words, and search-wordlist should also return zero for them.
• The last paragraph guarantees that any word that is allowed to be Ticked has an execution token.

The execution semantics of each Forth definition are specified in an "Execution:" section of its glossary entry. When a definition has only one specified behavior, the label is omitted.

Rationale: for some words, the execution semantics are not specified by the standard.

After that, add the following two paragraphs:

Rationale

• We have to rely on the absence of an "Interpretation:" section to refer to ordinary and immediate words until words like >r will be defined using a "Run-Time:" section instead of an "Execution:" section in their glossary entries (commented on 2019-06-21, 2020-08-30).
• In this section, we use the term "implementation dependent" rather than "implementation defined" because this allows implementors to avoid documenting some system-specific words altogether.
• When the standard allows to implement system-specific interpretation semantics for a standard word (by specifing that the interpretation semantics are undefined), and the system does so, executing xt of the word (if this xt is available) in interpretation state shall perform the system-specific interpretation semantics. The behavior of this xt in compilation state is not restricted by the standard. In classic Forth systems, in compilation state, it performs the specified compilation semantics for the word.

Rationale: find (in interpretation state) and search-wordlist return either the execution token of the word or zero, and there is no ambiguous condition in this regard.

6.2.0698 ACTION-OF,

An ambiguous condition exists if any of POSTPONE, [COMPILE], ' or ['] are applied

An ambiguous condition exists if POSTPONE or [COMPILE] are applied

Consequences

This change specifies all the cases in which an xt returned by search-wordlist, ' (Tick), ['] (Bracket Tick), can (or cannot) be used by a standard program in the general case.

This change removes prohibition on Ticking the words to, is, action-of, and specifies that executing of the returned xt in interpretation state shall perform the interpretation semantics for the word. Note that executing can be performed directly by execute, or indirectly by executing of the definition in which this xt is compiled using compile,.

If the system throws an error on ticking these words, or does not provide a correct xt for them, it should be updated to be compliant.

All classic Forth systems comply with this change.

This change does not affect the existing standard programs.

• 2025-10-03 Correct some normative statements; add ambiguous conditions; make better wording. Add more rationale. Correct some typos.

By the definition of the term "execution token" in Forth-94 and Forth-2012, it's a value that identifies execution semantics. Can such value identify other behavior, e.g., some interpretation semantics or compilation semantics? It's unclear at the first glance.

Actually, an execution token can identify (and does identify) other semantics too, but only if they are equivalent to the execution semantics that this token also identifies.

to the compilation semantics for if (this follows from the standard).

Note that there are cases where the semantics cannot be identified by an execution token in a Forth system, because the implementation of the system does not have or cannot have a Forth definition with equivalent execution semantics.

Examples of semantics that cannot be identified by any execution token:

:noname;

• typically, the run-time semantics of if (which are appended to the current definition by the compilation semantics of if);

Of course, the standard allows such implementations and disallows programs to obtain an execution token of the corresponding semantics, or even does not provide a way to obtain it.

To solve the initial problem, we can:

and when by the implementation, and to what extent),

• update the definition of the "execution token" term to say what these tokens identify.

- At the same time, the execution semantics of the word `BAZ` may differ from
the execution semantics identified by this _xt_.

Actually, the standard contains only one place where the "execution token" term is used ambiguously in a normative part — the glossary entry for FIND. The problem is that it says that FIND returns the execution token of the Forth definition ("its execution token", so it should identify the execution semantics of that definition), but:

Rationale: this is necessary to correctly describe the "execution token" data type.

For any valid execution token in the system, there is at least one Forth definition (named or unnamed) whose execution semantics are identified by that execution token.

If the interpretation semantics for a Forth definition are defined by this standard, the execution token of that definition shall be available.

may return zero for some existing (but not user-defined) words;

Add the following paragraph at the end:

See also: A.3.1.3.5 Execution tokens.

Change the first paragraph:

After that, insert the following paragraphs:

If the execution semantics for a Forth definition are specified by this standard and the glossary entry of that definition does not have an "Interpretation:" section, the execution token of that definition identifies the specified execution semantics. Otherwise, the execution token of that definition, if available, identifies the implementation dependent execution semantics.

• We use the term "implementation dependent"

to avoid documenting the provided execution semantics of standard words whose execution semantics are not specified by the standard.

The execution semantics identified by the execution token of a Forth definition, when they are performed in interpretation state, shall perform the interpretation semantics of that definition.

Rationale: this guarantees that, while interpreting, ' foo execute is equivalent to foo, even when the execution token (xt) of foo identifies implementation dependent execution semantics.

An ambiguous condition exists if the execution semantics of a Forth definition are not specified by this standard and its execution token is executed in compilation state.

Rationale: In all classic Forth systems, this performs the compilation semantics for the word, but in most dual-xt Forth systems, this performs the interpretation semantics for the word.

An ambiguous condition exists if the interpretation semantics of a Forth definition are undefined by this standard and its execution token is executed.

Rationale

• The execution semantics of the phrase s" if" forth-worldlist search-wordlist if execute then is always ambiguous, regardless of whether they are performed while interpreting or compiling, because they execute (at least on some systems) the xt of the word if, whose interpretation semantics are undefined by this standard.

attempting to obtain, using the words 6.1.0070 ' or 6.1.2510 ['], the execution token

Rationale: find and "etc." are excluded from the list, because find (in interpretation state), search-wordlist, and name>interpret return either

replace the fragment:

with the fragment:

Note: in the first case the form “are applied” is used, in the second and third cases the form “is applied” is used. They should be harmonized/corrected.

Rationale: it is now specified what semantics the execution token identifies for these words.

According to Anton Ertel's testing of five Forth systems (comment [r885]) in 2022, iForth 5.0.27 and VfxForth 5.11 did not comply with this change with respect to the execution tokens of the word to (and, probably, is and action-of). However, they were already noncompliant without this change, as they returned an incorrect execution token for s" (ticking of which is allowed).

Note that all classic Forth systems natively comply with this change.