Proposal: Forth Standards, Backward Compatibility and modern `state of the art` words with their `historical traditional` counterparts.

Creating backward compatibility word sets and move the historical traditional words to there, if modern state of the art word pendants exist.

If you are considering creating a new section (a word set) in the document, please note that proposals to move a word between sections usually do not find sufficient support. Regarding obsolescent words, they are usually excluded from the document on the next iteration. I don't think we should create a new normative section (a word set) containing specifications for obsolescent word.

Dictionary Manipulation: LATEST vs. GET-CURRENT
Previously, LATEST was a variable that could be easily read using @.

Probably, you mean the word current.

In FIG-Forth, the word latest returns an NFA (Name Field Area, now name token), not an address of the NFA (as I recently mentioned). The word last traditionally returns an address that contains NFA, see Forth-83, the section "UNCONTROLLED REFERENCE WORDS" (link).

What I'd like to propose for discussion is a more future-oriented approach to the standard (what I mean by "state of the art"), incorporating all the positive experiences and progressive insights gained, without clinging too tightly to the past.

However, to ensure that the past is also adequately addressed, the standard should provide a framework for integrating it when needed.

So, I envision the standard having two parts: the current standardizations and, secondly, the option to bridge the gap to the past.

And sometimes I sense a kind of fear, between the lines, that it will be seen as "bad" if a system doesn't conform to Standard 999 but only to Standard 666. But Standard 666 is also perfectly fine if it fulfills its purpose, even if sometimes with somewhat more "complex" methods, isn't it?

And programs that conform to standard 666 might be considered inferior to those that conform to standard 999. But isn't the primary goal of programming a program that fulfills its tasks, not necessarily adhering to the latest standard at all costs?

The last two paragraphs were more philosophical in nature.

I can only suggest something like this. Whether such a committee then considers such modern approaches is beyond my control. And I realize that this will likely be a very long-term undertaking that can't be implemented in the very next standard.

The last two paragraphs were more philosophical in nature.

Yes, and the ideas are quite general. Could you please identify and describe the first specific change in the document of the Standard? The smaller the change, the better.

And sometimes I sense a kind of fear, between the lines,

This feeling resonates with me in discussions when it's argued that system A does things this way and system B reacts in a certain way, or when a decision is attributed to how things are usually done. The focus on the past always carries so much weight.

Could you please identify and describe the first specific change

My first idea was to add an Pargraph with a list as overview and than later down describe all the resources and token the standard handles with on an absract implementation independent level.

Than from there it could be possible to isolate the resources and token an abstract Forth kernel shall have to be a Forth system in the manner of the forth standard. Of course there may be more other systems that deal in other ways. But this shows "only" that the abstraction isn't abstarct enough or that this is considered not as a topic the standard wants to handle.

This may be considered as an implementation independent description what a forth system is at an abstract level.

A first collection at a draft level could look like follows. The version is only for my personal knowledge of my thoughts state. The '+' sign tells that this token are part of a structur token

I'm shure that this is not complete due to my not always sufficiently in-depth knowledge. And some things are viewed from the perspective of someone interested in Forth, who misses a kind of system-thematic overview presentation as a "research anchor".

A draft of a Forth system abstraction (Version 0.4)

Forth System Components

• Forth Stack system
  • Data-Stack token (former called 'parameter stack' too)
    • Data-Stack-Entry token
  • Return-Stack token
    • Return-Stack-Entry token
  • Control-Stack token
    • Control-Stack-Entry token
  • Float-Stack token
    • Float-Stack-Entry token
  • Exeption-Stack token
    • Exeption-Stack-Entry token
  • Input-Source-Stack token
    • Input-Source-Stack-Entry token
• Forth Word system
  • Word token
    • Word-Header token
      • +Word-Name token
      • +Word-Link token
      • +Word-State token (former called 'flags' too)
        • +Word-Execution-State
        • +Word-Visibility-State
      • +Word-Behavior token (former called 'codefield')
    • Word-Data token
• Forth Dictonary system
  • Dictionary token
    • Dictionary-Entry token
• Forth Input-Source system
  • String-Literal Input-Source token
  • User-Input-Device Input-Source token
  • File-Input Input-Source token
    • +File-Descriptor token
    • +File-Position token
    • +File-Buffer token
      • +Buffer-Address token
      • +Buffer-Length token
      • +Buffer-Position token
  • Stream-Input Input-Source token
• Forth String-Token-Type system
  • Dictionary-Word-String token
  • Numerical-String system
    • Integer-String system
      • Half-Integer-String token
      • Single-Integer-String token
      • Double-Integer-String token
      • Quad-Integer-String token
    • Float-String system
      • Half-Float-String token
      • Single-Float-String token
      • Double-Float-String token
      • Quad-Float-String token
• Forth Numerical-Value system
  • Integer-Value system
    • Half-Integer token
    • Single-Integer token
    • Double-Integer token
    • Quad-Integer token
  • Float-Value system
    • Half-Float token
    • Single-Float token
    • Double-Float token
    • Qaud-Float token

Forth system components are considered to have at least one member component.

A Forth system shall define all the needed components by mapping them to the underlying system any way direct or indierct by using other components.

Forth Token

A token is considered as a collection of at least one a token.

A Forth system shall define all the needed token.

Forth Basic Components

Forth basic components are considered as token itself.

• Forth character token (CHAR)
  • Forth character-string token
    • Forth counted-string token
    • Forth zero-terminated-string token
    • Forth descriptor-string token
• Forth datacell token (CELL)
• Forth pointer token
• Forth boolean token
  • Forth true token
  • Forth false token

A base consideration is that a Forth datacell token is a multiple of a Forth character token.

So a datacell token may considered to represent a fixed count (the multiple) of charakter token.

A Forth pointer token holds the information from where to get a token and its served informations.

Forth Basic Components may be considered as bricks to build the building of an Abstract Forth Kernel and the levels of Forth Systems built upon it.

All the needed Forth Basic Components shall be mapped to the underlying system any way.

Forth Resources

To organize components and token they may considered as logical ressource spaces. This spaces are not considered to be contigous but may of course.

• Dictionary-System Space: Dictionary Space collection
  • Dictionary Space: Word-Header Token Collection
• Data-System Space: Data Space Collection
  • Word-Data Space: Word-Data Token Collection
• Character-String Space: String-Space Collection
  • Name-String Space: Word-Name Token Collection
  • Data-String Space: Character-String Collection
  • Transient-String Space: Character-String Collection
• Behavior Space: Word-Behavior Token Collection
• Stack Space: Stack Space Collection
  • Data Stack Space: Data-Stack-Entry Token Collection
  • Return Stack Space: Return-Stack-Entry Token Collection
  • Control Stack Space: Control-Stack-Entry Token Collection
  • Float Stack Space: Float-Stack-Entry Token Collection
  • Input-Source Space: Input-Source-Stack-Entry Token Collection
• Input Space: Input-Source Collection
  • Text_String Space: Character-String Token Collection
  • User-Input Space: Terminal-input-Buffer-String Token Collection
  • Text-File Space: Text-File-Buffer-String Token Collection
  • Text-Stream Space: Stream-Input-Buffer-String Token Collection

The second chapter I would introduce is an optional wordset KERNEL that provides missing words to describe a functioning Forth-Kernel in Forth words inside a dictionary.

These are for instance words to handle the stacks (push-xx, pop-xx, constants to reset or something else), the missing interpret word and the words associated with it, including the group of recognizer words to define the interpre word and others.

One goal at all from possible multiple more is to be able to describe a standard Forth system based on a Forth kernel, not as the optimal way to do but as an showcase, that fits all the needs the standard prescribes and requires.

The third step could be to reorder the word groups by making modern more state of the art words mandatory and elder words optional. But this is a big step into the future to make history backpacks smaler.

One unsystematic aspect, in my opinion, is that there are words with three, two, or one behavior (interpreting, compiling, run-time). On the other hand, there are groups of words that represent this behavior in separate, distinct words. The standard should address one version here.

However, I do think it makes sense to separate the interpreting behavior from the compiling and run-time behavior. This would certainly simplify the work of the text interpreter in its three planned states (interpreting, compiling as immediate conditional compilation, and postponing as unconditional compilation).

Addendum:

According to the last thought, the Word Behavior token should have three sub-tokens:

• +Word Behavior token (formerly called 'codefield')
  • +Word Interpreting Behavior token
  • +Word Conditional Compiling Behavior token (compiling)
  • +Word Unconditional Compiling Behavior token (postponing)

This may be considered as an implementation independent description what a forth system is at an abstract level.

Take a look at the section 1.2 Scope: "This standard specifies an interface between a Forth System and a Forth Program by defining the words provided by a Standard System". Thus, the normative part should not include anything that does not serve the purpose of defining standard words, a standard system, and a standard program.

It seems to me, your "description what a forth system is" (as presented in your draft) is not normative and is outside the scope of the standard. Perhaps you could put this description into a separate independent document.

I do think it makes sense to separate the interpreting behavior from the compiling and run-time behavior.

The standard terms are "interpretation semantics", "compilation semantics", "run-time semantics", etc. They are specified separately in glossary entries. How they are separated in implementations is beyond the scope of the standard.

According to the last thought, the Word Behavior token should have three sub-tokens

It should be possible to perform any of the mentioned behaviors from nt (name token).

On the other hand, the requirement to provide three additional tokens per each word seems too high for existing systems and does not bring obvious benefits to programs.

Take a look at the section 1.2 Scope: "This standard specifies an interface between a Forth System and a Forth Program by defining the words provided by a Standard System".

So it should define too what are the two sourrounded things (Forth system and Forth programm) as well. And that's what I want the standard should do.

Thus, the normative part should not include anything that does not serve the purpose of defining standard words, a standard system, and a standard program.

I think I never told that. I only told to do so. Of course the system should be described before Chapter 6 and/or inside the apendixes. And all needed words should be added as standard (optional or mandatory) words.

The standard terms are "interpretation semantics", "compilation semantics", "run-time semantics", etc. They are specified separately in glossary entries. How they are separated in implementations is beyond the scope of the standard.

But a standard should be in handling things consistent. But in this sense it isn't. For som words there are different Words for compiling and run-time/inzterpretig semenatics, if I understand the things rigth: All word pairs xxx and [XXX] for instance.

It should be possible to perform any of the mentioned behaviors from nt (name token).

But name is part of the word header whats part of the word.

On the other hand, the requirement to provide three additional tokens per each word seems too high for existing systems and does not bring obvious benefits to programs.

And there is what I mention 'the backpack thinking'. Any standard shall state what should be from here and should not be a saver for "priviledges" and expense from the past although it could.

And believe me, I know this can and will influence the acceptance of such standards. But a standard shouldn't act as a lobbyist for historical ideas.

And yes, I want to advocate for a "Rethinking Forth" and for an "Out of Itself Forth" standard.

I want to emphasize that I consider the work of the old guard, who kept Charles Moore's original forms alive and in practice, to be comprehensive and magnificent. And I, too, belong to an old guard at 72, and I haven't been very actively involved in the further development of Force. However, I maintain that it's now time to hand over the keys to a new generation of thinking, tasked with abstracting the original fundamental ideas of a forestry system and encapsulating them in a new, state-of-the-art, knowledge-based system.

And in addition, I'd like to share a piece of wisdom from my grandmother that has often helped me at crucial decision points:

Junge, lass dir nicht das Vergangene in das Zukünftige schreiben, aber nutze es weise bei Entscheidungen in der Gegenwart

Son, don't let the past shape your future, but use it wisely when making decisions in the present.

Could you please identify and describe the first specific change

There are several words that can be parsed. And there are some discussions about transient areas, among other things. What bothers me is that there are so many different output behaviors, certainly due to historical reasons.

One could derive all parsed words from a SOURCE API similar to the RECOGNIZER API to standardize parsing. And in principle, we can distinguish between a "technical" input buffer, which is filled by the hardware-connecting primitives, and a virtual source buffer, which only processes one line and uses the technical buffer by pointing.

Furthermore, one could also introduce a "multi-parse" or "whitespace parse" that iterates through a string of delimiters. This delimiter list then corresponds somewhat to what the recognizer chain is for recognizers. This would always return the first found position and the found string.

I'm also always surprised why three different string types are allowed. Couldn't the standard simply remain neutral and say there's only one string token, with the system determining the structure of that token? And then an extended word group could provide the conversion functions? And all corresponding words would then return this standard string token?

Could you please identify and describe the first specific change

FORTH Word Set System

For the future, I can easily imagine redesigning the Word Set System. For each topic, I propose three word groups.

An optional core word group containing all those words that are likely to be used, or are most likely to be implemented in existing systems in a very hard-coded way, and should therefore be considered primitive.

Then the topic word group with those words that can be derived from all existing core word groups and that should be present for the functionality that this word group represents.

And, for the sake of user-friendliness, the optional third group, the extended word group, containing words that make it more convenient or easier for the user to apply frequently occurring tasks in the form of macros or functions.

The first word group is optional because I don't want to prevent anyone from describing the words in the core group as primitive.

Could you please identify and describe the first specific change?

Forth QUIT

The word "Quit" essentially has the first task of setting the FORTH system to a defined initial state.

One more task is to continue doing what every FORTH system should be able to do from the very beginning after startup.

And in the third part, it has the task of resolving the ABORT situation and starting over.

The defined initial state is the activation of the interpretation of an input line, such as the terminal system. This state is precisely the requirement of the initial system from this standard.

However, what I consider a historically conditioned constraint here, in terms of the abstraction of a FORTH system, is the specification of the input source as terminal input. Because even if I start a FORTH system differently, for example via the parameter line, I run into a conflict with this standard.

So I asked myself, why shouldn't it be a different input source, such as a stream or any other conceivable input source?

A follow-up question is: what instruction must quit receive for its first task, namely resetting the FORTH system?

A second follow-up question is: how should the startup behavior than be described in the standard?

My initial thoughts here suggest that it might be useful to define an input source chain, similar to the recognizer chain. This would allow each system to define its own system-dependent startup behavior.

Setting up this chain would then be the task of the initialization phase for the FORTH system.

So it should define too what are the two sourrounded things (Forth system and Forth programm) as well.

Yes, see:

• 5.1.1 System compliance
• 5.2.1 Program compliance

To make them better, a formal proposal should be submitted.

For som words there are different Words for compiling and run-time/inzterpretig semenatics, if I understand the things rigth: All word pairs xxx and [XXX] for instance.

Please, make a separate proposal to make such a word better.

I'm also always surprised why three different string types are allowed.

They cannot be disallowed. No type/format can be disallowed in general.

And all corresponding words would then return this standard string token?

It's impossible to incompatibly change the stack effect of existing standard words. You have to introduce new words that return this token instead of a character string ( c-addr u ).

if I start a FORTH system differently, for example via the parameter line, I run into a conflict with this standard.

Could you please clarify what conflict you are talking about? A "parameter line" is implementation dependent.

what instruction must quit receive for its first task, namely resetting the FORTH system?

There is no such task as "resetting the Forth system". See 6.1.2050 QUIT.

They cannot be disallowed. No type/format can be disallowed in general.

I never mentioned that there is only one string type allowed. But I mentioned to tell a system to be consistent using only one type for words input/output and have words to "translate" between types if necessary some where (in programs).

Yes, see:

• 5.1.1 System compliance
• 5.2.1 Program compliance To make them better, a formal proposal should be submitted.

That's not I discuss. A system that complies ... is okay in any Forth standard.

What I discuss is to rethink Forth in some parts. And yes, it's a kind of break.

But sorry, in my current state of deeper and brider knowledge I feel not able to write a full concept yet in form of a proposal. But I think abaout and how to split all things to proposals. I'm not familiar with that yet.

They cannot be disallowed. It's impossible to incompatibly change the stack effect of existing standard words.

No isn't. A new standard may change things.

This is impossible if one insists on the principle of ensuring backward compatibility and the broadest possible compatibility with existing systems under all (or most) circumstances.

In my opinion, this prevents several possible systematizations and streamlinings. And I never mentioned that this should be the very next version, but in future it should be done.

Unfortunately, I've only encountered one opinion on this so far. This also gives the impression to me that my topics aren't generating much interest.

If backward compatibility and compatibility with existing systems are the highest priority here, then my forward-looking discussion is pointless because it will fall on deaf ears.

So I will ask you all two questions.(QUESTION: How do I make a reply to be a survey?)

• What priority has the backward compatibility between Forth standards?

  • Necessary
  • Very important
  • Less important
  • Unimportant
  • Not necessary

• What priority has the compatibility with existing Forth systems?

  • Necessary
  • Very important
  • Less important
  • Unimportant
  • Not necessary

Could you please clarify what conflict you are talking about?

3.4 The Forth text interpreter Upon start-up, a system shall be able to interpret, as described by 6.1.2050 QUIT, Forth source code received interactively from a user input device.

As a system is able to interpret (may be via EVALUATE ?) a command line word list string, starting a forth system, this don't dfit this sentence, or does it?

So no standard system shall be able to do so?

And there is one connecting point to the SOURCE API thoughts, similar to RECOGNIZERS/TRANSLATORS.

There is no such task as "resetting the Forth system". See 6.1.2050 QUIT.

Apologies for not speaking the Forth description language perfectly.

Of course, there are several "reset" states. One is when only the return stack (and control stack) is reset, as with a normal return from interpretation. The other is a "complete" reset via ABORT, which also resets the data stacks (parameters and, if applicable, floats).

And of course, I understand that the dictionary is not reset.

And yes, it's a kind of break.

If a specification introduces significant breaks, than the specification defines a new programming language.

In fact, cases where you actually need to break something are very rare. Simply introduce new words (perhaps in a new namespace; see my other reply) instead of changing the behavior of existing words.

What priority has the backward compatibility between Forth standards?

Very important. We may remove deprecated words from the next version. We may make some things more or less abstract (as the implementation-defined cell size in Forth-94, or the separate floating-point stack in Forth-2012). But we should not allow the word's behavior to change in an incompatible way. Although such cases have historically occurred, see Forth-83 in Annex C: Compatibility analysis.

In general, a standard Forth program should remain standard in the next version of the Standard, possibly with new environmental dependencies.

What priority has the compatibility with existing Forth systems?

A standard Forth system should remain standard, possibly with new environmental restrictions.

Regarding 3.4 The Forth text interpreter:

• Upon start-up, a system shall be able to interpret, as described by 6.1.2050 QUIT, Forth source code received interactively from a user input device.

"Be able" does not mean the system must do that in all circumstances. How to start the system in this mode is system-specific. How to specify the actions the system must perform before interpreting the user input device is system-specific. These things are not standardized at the moment.

The method of selecting a user input device is implementation defined. See 3.2.4 Operator terminal.

As a system is able to interpret (may be via EVALUATE ?) a command line word list string, starting a forth system, this don't dfit this sentence, or does it?

This does not violate the section 3.4, since "A system may contain non-standard extensions, provided that they are consistent with the requirements of this standard".

So no standard system shall be able to do so?

No. Any standard system may do so.

Ok. It looks like we have different thoughts about what a standard shall do.

And I understand, that my thoughts are not welcome here because they look to much forward and not enough back in your eyes. So take my ideas or don't.

And my words and my kind of using the language is different to yours. I'm not able to make my things more clear.

Thank you for your time you spend for my 'fictional' fantasies.

It looks like we have different thoughts about what a standard shall do.

Don't let this stop you from pursuing your endeavor. It can be a new Forth-like language that is not compatible with the Standard Forth. Note, there are many Forth-like languages. Some of them:

• Factor
• 8th
• RetroForth
• StrongForth
• Fift

Regarding this proposal, you may want to change its status to "Retracted".