Annex C: Compatibility analysis

Before this standard, there were several industry standards for Forth. The most influential are listed here in chronological order, along with the major differences between this standard and the most recent, Forth 94.

C.1 FIG Forth (circa 1978)

FIG Forth was a "model" implementation of the Forth language developed by the Forth Interest Group (FIG). In FIG Forth, a relatively small number of words were implemented in processor-dependent machine language and the rest of the words were implemented in Forth. The FIG model was placed in the public domain, and was ported to a wide variety of computer systems. Because the bulk of the FIG Forth implementation was the same across all machines, programs written in FIG Forth enjoyed a substantial degree of portability, even for "system-level" programs that directly manipulate the internals of the Forth system implementation.

FIG Forth implementations were influential in increasing the number of people interested in using Forth. Many people associate the implementation techniques embodied in the FIG Forth model with "the nature of Forth".

However, FIG Forth was not necessarily representative of commercial Forth implementations of the same era. Some of the most successful commercial Forth systems used implementation techniques different from the FIG Forth "model".

C.2 Forth 79

The Forth-79 Standard resulted from a series of meetings from 1978 to 1980, by the Forth Standards Team, an international group of Forth users and vendors (interim versions known as Forth 77 and Forth 78 were also released by the group).

Forth 79 described a set of words defined on a 16-bit, twos-complement, unaligned, linear byte-addressing virtual machine. It prescribed an implementation technique known as "indirect threaded code", and used the ASCII character set.

The Forth-79 Standard served as the basis for several public domain and commercial implementations, some of which are still available and supported today.

C.3 Forth 83

The Forth-83 Standard, also by the Forth Standards Team, was released in 1983. Forth 83 attempted to fix some of the deficiencies of Forth 79.

Forth 83 was similar to Forth 79 in most respects. However, Forth 83 changed the definition of several well-defined features of Forth 79. For example, the rounding behavior of integer division, the base value of the operands of PICK and ROLL, the meaning of the address returned by ', the compilation behavior of ', the value of a "true" flag, the meaning of NOT, and the "chaining" behavior of words defined by VOCABULARY were all changed. Forth 83 relaxed the implementation restrictions of Forth 79 to allow any kind of threaded code, but it did not fully allow compilation to native machine code (this was not specifically prohibited, but rather was an indirect consequence of another provision).

Many new Forth implementations were based on the Forth-83 Standard, but few "strictly compliant" Forth-83 implementations exist.

Although the incompatibilities resulting from the changes between Forth 79 and Forth 83 were usually relatively easy to fix, a number of successful Forth vendors did not convert their implementations to be Forth 83 compliant. For example, the most successful commercial Forth for Apple Macintosh computers is based on Forth 79.

C.4 ANS Forth (1994)

In the mid to late 1980s the computer industry underwent a rapid and profound change. The personal-computer market matured into a business and commercial market, while the market for ROM-based embedded control computers grew substantially. Improvements in custom processor design allowed for the development of numerous "Forth chips," customized for the execution of the Forth language.

In order to take full advantage of evolving technology, many Forth implementations ignored some of the restrictions imposed by the implied "virtual machine" of previous standards. The ANS Forth committee was formed in 1987 to address the fragmentation within the Forth community caused not only by the difference between Forth 79 and Forth 83 but the exploitation of technical developments.

The committee undertook a comprehensive review of a variety of existing implementations, especially those with substantial user bases and/or considerable success in the market place. This allowed them to identify and document features common to these systems, many of which had not been included in any previous standard. This was the most comprehensive review of Forth systems to date, taking eighty-seven days covering twenty-three meetings over eight years. The inclusive nature of the standard allowed the various factions within the community to unify in support of ANS Forth, with many systems providing a compatibility layer.

The committee chose to move away from prescribing stringent requirements as previous standards had, with the specification of a virtual machine. It preferred to describe the operation of the virtual machine, without reference to its implementation, thus allowing an implementor to take full advantage of any technical developments while providing the developer with a complete list of entitlements.

This required the identification of implicit assumptions made by the previous standards, making them explicit and abstracting them into more general concepts where possible. A good example of this is the size of an item on the stack. In previous standards this was assumed to be 16 bits wide. This was no longer a valid assumption. ANS Forth introduced the concept of the cell, allowing an implementation to use a stack size most suited to the environment.

The American National Standards Institution (ANSI) published the ANS Forth Standard in 1994 with the title "ANSI X3.215-1994 Information Systems — Programming Language FORTH". This is referenced throughout this document as Forth 94.

C.5 ISO Forth (1997)

ANSI submitted the Forth 94 Standard to the ISO (International Organization for Standardization) and IEC (International Electrotechnical Commission) joint committee for consideration as an international standard. The ISO/IEC adopted the Forth 94 document as an international standard in 1997, publishing it under the title "ISO/IEC 15145:1997 Information technology. Programming languages. FORTH".

C.6 Approach of this standard

During a workshop on the Forth standard at the EuroForth conference in 2004 it was agreed that Forth 94 required updating.

A committee was formed and agreed that the process should be as open as possible, adopting the Usenet RfD/CfV (Request for Discussion/Call for Votes) process to produce semi-formal proposals for changes to the standard. In addition to general discussion on the comp.lang.forth usenet news group, a moderated mailing list (with public archive) was created for those who do not follow the news group. Standards meetings to discuss CfVs were held in public in conjunction with the EuroForth conference.

The work of the Forth 94 committee was the basis of this standard, informally called Forth 200x. The aim of the Forth 200x committee is to produce a rolling document, with the standard constantly being updated based on discussion of proposals and the corresponding votes. A snapshot document is occasionally produced, with this document being the first.

The Forth 200x committee defined a procedure for proposals. In addition to the formal text of the proposal, they had to include: the rationale behind the change; a reference implementation, or a description of the reason a reference implementation cannot be presented; unit testing for the proposed change, especially for border conditions. See Proposals Process (page ix) for a full description.

C.7 Differences from Forth 94

C.7.1 Removed Obsolete Words

Forth 94 declared seven words as `obsolescent', all but FORGET have been removed from this standard.

Words affected:

#TIB, CONVERT, EXPECT, QUERY, SPAN, TIB, WORD.

Reason:

Obsolescent words have been removed.

Impact:

WORD is no longer required to leave a space at the end of the returned string.

It is recommended that, should the obsolete words be included, they have the behaviour described in Forth 94. The names should not be reused for other purposes.

Transition/Conversion:

The functions of TIB and #TIB have been superseded by SOURCE. The function of CONVERT has been superseded by >NUMBER.

The functions of EXPECT and SPAN have been superseded by ACCEPT.

The function of QUERY may be performed with ACCEPT and EVALUATE.

C.7.2 Separate Floating-point Stack is now Standard

Previously systems could implement either a separate floating-point stack or a combined floating-point/data stack; programs were required to cater for both (or declare an environmental dependency on a particular variant).

Words Affected:

All floating-point words.

Reason:

The developing of software that may be used with either a combined stack or a separate stack is extremely difficult and costly. While some of the systems surveyed provide a combined floating-point/data stack, they all provide a separate floating-point stack.

Impact:

Forth 94 programs with an environmental dependency on a separate floating-point stack become standard programs.

Forth 94 programs with an environmental dependency on a combined stack retain the environmental dependency.

Forth 94 programs (without environmental dependency, i.e., those working on either kind of system) remain standard programs.

Forth 94 systems that implement a separate floating-point stack continue to be standard systems.

Forth 94 systems that implement a combined stack become systems with an environmental restriction of not providing a separate floating-point stack, but a combined stack.

Transition/Conversion:

Any code that has an environmental dependency on the use of a combined floating-point/data stack should be ported to use a separate floating-point stack.

A system that has an environmental restriction on using a combined floating-point/data stack should consider providing a separate floating-point stack.

C.7.3 Using ENVIRONMENT? to inquire whether a word set is present

With the advent of a new standard, it was necessary to review the meaning of word set queries. Compatibility with Forth 94 demands that a word set query produce the same result as for Forth 94; i.e., querying for CORE-EXT returns true only if all the Forth 94 CORE EXT words are present. The question was how to distinguish between word sets described by this and subsequent standards.

The committee considered adding a year indicator to the word set name ("CORE-EXT-2012") or a providing a general query ("Forth-2012") which could be combined with the word-set query. As the committee could find very few examples of the word-set queries being used, it chose not to update the word set-query mechanism, but rather to mark it as obsolescent.

Words Affected:: ENVIRONMENT?
Reason:: The use of the word-set query to inquire whether a word set is present in the system has been marked obsolescent. If present the query indicates the word set, as documented in Forth 94, is available.
Impact:: Forth 94 did not guarantee the presence of these queries. Many systems that provided all the words in a particular word set did not provide the corresponding query. Portable programs are not affected as they could not rely on this function.
Transition/Conversion:: There is no direct equivalent to determine the presence of a whole word set. The 15.6.2.2530.30 [DEFINED] and 15.6.2.2534 [UNDEFINED] words can be used to detect the availability (or otherwise) of individual words.

C.7.4 Additional TO targets

6.2.2295 TO has been extended to act on targets defined with 12.6.2.1628 FVALUE and 8.6.2.0435 2VALUE.

Words affected:: TO

C.7.5 Input/Output return values

Words affected:: All words that return an ior.
Reason:: Forth 94 left the error code (ior) implementation-defined, although it did recommend an ior to be a THROW code. Forth 2012 now requires an ior to be a THROW code.
Transition/Conversion:: Forth 94 programs are not affected. Programs that are dependent on iors being throwable are no longer required to document the dependency.
Forth 94 systems that abided by the recommendation are not affected. Systems that did not heed this advice are required to do so. A number of THROW codes were added to table 9.1 to ease this transition.

C.7.6 Minimum number of locals

Words affected:: (LOCAL), LOCALS|
Reason:: Some programs require more than eight locals.
Transition/Conversion:: Existing programs are unaffected. Systems implementing the locals word set have to be changed to support at least 16 (previously 8) locals.

C.7.7 Number prefixes

Decimal, hexadecimal, binary number literals can now be written irrespective of BASE by using the prefix #, $, %. Also, character literals can be written as 'c'. Standard programs are unaffected. Systems have to be changed to recognize these forms.

See 3.4.1.3 Text interpreter input number conversion.

C.7.8 SOURCE-ID Clarification

When interpreting text from a file, the relationship between the position in the file returned by SOURCE-ID, and the current interpretation position is undefined.

C.7.9 FASINH

An ambiguous condition on r1 being less than 0 was removed.

Existing programs are not affected. Existing systems are unlikely to be affected.

C.7.10 FATAN2

Words affected:: FATAN2
Reason:: The result is now specified more tightly: it is the principal angle (between -pi and pi).
Impact:: Forth 94 compliant programs are not affected.
Transition/Conversion:: Systems may have to change FATAN2 to return the principal angle.