Proposal: Obsolescence for SAVE-INPUT and RESTORE-INPUT

SAVE-INPUT/RESTORE-INPUT are used in some user-space code, for example in Gforth assemblers + cross-compilers. Example:

: .times{  ( n -- input n )
  dup >r 1 > IF  save-input  THEN  r> ;
: .}times  ( input n -- input n-1 / 1 / )
  1- dup 0>
  IF  >r restore-input throw r@ 1 >
      IF  save-input  THEN  r>
  THEN ;
:D

I can only imagine one workaround, which is to copy input to a buffer, and EXECUTE it from there. Obviously, such a workaround increases CPU+RAM usage.

SAVE-INPUT/RESTORE-INPUT are used in some user-space code, for example in Gforth assemblers + cross-compilers.

It's a good finding. The words .times{ and .}times are formally defined in the standard Forth. Although, as I can see, they are not used anywhere.

One problem with this construct is that it can work or fail without any explanation, — since the standard does not guarantee any conditions in which restore-input shall work. So the construct .times{ ... .}times is actually a system-specific means (or it has an environmental dependency concerning behavior of restore-input).

Another problem with this construct is that the body is performed at least once regardless of n. To correctly implement the case of n=0, we need to parse the input stream till the corresponding closing .}times (with possible nesting). But if we parse the input stream in this way, we can save the extracted text into a buffer and just translate (evaluate) it the given number of times, without employing save-input/restore-input.

I believe, we should have an API that helps in such tasks. Under the hood it can save a content or position (depending on the input source kind), and provides a uniform interface. Actually, such an API can be even implemented in a portable way if we standardize two things:

1. When the input stream is a string, the line comment shall skip up to the nearest line terminator or to the end of the input buffer (what is encountered first) (see a comment).
2. A program should not depend on the input source kind from which it's translated. It means, it should not rely that the input buffer contains a single line, and that refill reads a single line (see Portable line-oriented parsing). Probably, some helper words should be standardized for that.

Obviously, such a workaround increases CPU+RAM usage.

It depends on how save-input is implemented. A reliable implementation can also save a fragment of the input stream into the memory. OTOH, repeating reading from the file can take more CPU than reading from memory.

The committee considers this proposal formal and asks the author to change its status to "CfV - Call for Votes" whenever he deems it ready.

Note: The committee likes to point out, that these words cannot be made informal, as they are used to implement interpreted loops.

There is a technique to reuse input that doesn't disturb the current input stream. That is saving and restoring >IN. This may be restricted to the current input buffer, but that may cover a substantial part of the cases.

Clearly the original intent was a possibility to be a factor of INCLUDE , interrupting the current input stream. However as the proposal points out, this is not going to be a useful system word, rather a later burden. I would call this "meddling in matters that should be up to the implementer". This kind of words should be exterminated from the standard. They are almost never needed, and hard to implement. So obsolete these words!

I don't understand what this proposal has to do with "interpreted loops" . All noforths and ciforths (Dutch Forths) have interpreted loops with using SAVE-INPUT and RESTORE-INPUT.

I don't understand what this proposal has to do with "interpreted loops" . All noforths and ciforths (Dutch Forths) have interpreted loops without using SAVE-INPUT and RESTORE-INPUT.

what this proposal has to do with "interpreted loops"

As far as I know, "interpreted loops" are not used in standard programs (i.e., Forth programs that are independent on a particular Forth system). And the systems that provide save-input and restore-input may continue to provide these words.

An alternative way to implement an interpreted loop is to parse the input stream (the loop body) into a buffer and evaluate the buffer.

Concerning possible problems, see #217 New Line characters in a string passed to EVALUATE.

 : EXECUTE-PARSING ROT ROT SAVE SET-SRC CATCH RESTORE THROW ;

The standard does not allow save-input and restore-input to be used this way, because the identity of the input source on restore-input will not be the same as on save-input (but it shall be the same).

not having REFILL and line by line compilation is too revolutionary.

Not quite. When the input source is a buffer, refill loads the next buffer (not the next line). When the input source is a multiline string, refill does almost nothing (see also my proposal for "\").

So, not having "line by line compilation" is not something revolutionary.

I needed save-input and restore-input in order to do lazy compilation in my uCore cross-compiler. The rational was minimizing the code size of applications in a one-pass compilation process. This is what I did:

I restructured often used words in "libraries" that were derived from load files of these words. The application would only need a few of them. Therefore it would have been a waste of code memory space to always compile all of them. The library consisted of the original load file, which I sliced up in "sibling groups" using ~ in front of each group, which consisted of just one definition in most cases. This embellished file could be loaded as usual using INCLUDE, because then ~ would be an immediate no-op. It could also be loaded using LIBRARY <filename> instead. In that case ~ would compile the following sibling group as a - lets call it - library word. Library words are immediate and the parameter list consists of the file and the position of the source code of that group.

When compiling the application, every now and then such a library word would be found and executed in the middle of compiling some application word. This triggers a sequence of events:

1. Delete the unfinished :-definition that requires the library word.
2. Use save-input to remember the file and location of the aborted definition.
3. Change the input stream to point to the file and location in the library file.
4. Compile from the library file until the next ~
5. Use restore-input to get back to where we interrupted comiling the application.
6. Continue compiling the application.

This code can be written in such a way that it works recursively, i.e. a library word may need another library word potentially from a different library etc.

This way the uCore cross-compiler produces optimal code in one pass.

I venture to say that save-input / restore-input is a very valuable application word. If it would have been not there, my code would have to be much more system dependent, fiddeling with the internals of e.g. Gforth with all the headaches this creates when its internal structure is silently modified by the implementors. Therefore: Don't deprecate it! Instead, define it in a strict way so that it will do the same on every system.

it would have been not there, my code would have to be much more system dependent

I see similar code in library.fs and microcross.fs (in microCore-VHDL/microCore at GitHub). It seems, this program is very system-specific and is far from a standard Forth program. Given such strong dependence on the system, even using non-standard save-input and restore-input adds almost nothing to dependency on the system. Or do you think your Forth system will stop providing save-input and restore-input if they become obsolete?

When compiling the application, every now and then such a library word would be found and executed in the middle of compiling some application word. This triggers a sequence of events

Got it. This does not look like one-pass compiling, as, when a word is missing, you translate the definition body again, and again for each missing word.

There are a number of other approaches (without save-input):

• Since you are using a user-defined Forth text interpreter, you can save the portion of the input source from the start of a colon-definition into a buffer and then translate this portion from the buffer.
• Or, the same as for words from the library, it is possible to add markup that allows to extract the whole definition body to lately translate it from a buffer (multiple times, if needed).
• Or, you can load the whole file into a buffer and apply evaluate (with proper \), and change >in instead of using restore-input.
• Or, it is possible to compile a definition into an intermediate code format, and when all missing words are loaded, translate the intermediate code into the target code.
• Or, when a missing word from a library is encountered, instead of discarding the definition, you can save the compiled code into a buffer, load the missing word, and then perform a binary translation of the compiled portion to the new location.
• Or, when a missing word from a library is encountered, you can compiler an unresolved call. After the definition is compiled, you load missing words (translate their definitions) and resolve all unresolved calls (this is also recursive).

Don't deprecate it! Instead, define it in a strict way so that it will do the same on every system.

Somebody should prepare a proposal. And then, if we specify these words more tightly, the system implementers should either make these words stronger, or remove them.

If we deprecate them, system implementers should not do anything, and authors of standard programs should stop using these words; however, such authors already don't use them. Therefore, no one should do anything.