Digest #223 2023-05-05
Contributions
This word introduces the concept of individual lines, and ( is extended to support them, however there is no similar extension to \, which continues to discard the entire rest of the buffer. The use case is a vi-clone screen editor in durexForth, it allows to edit files on disk and press a key to evaluate them as if they were included. That Forth's \ ( included evaluate etc were recently brought to standard, so when the editor evaluates its buffer then \ comments are broken, but if it separates and evaluates each line one-at-a-time then ( comments are broken. It would have to essentially reimplement included on in-memory buffers, maybe provide an non-standard definition of one of ( or \. I'm actually not really sure how it could be fixed. Other than:
I would propose a new section 11.6.2.2535 so that, when blk is 0, \ discards "the rest of the line," where that's defined to be the same as in included here.
Cross-reference durexForth author Johan's question about (. My opinion is that he's suffering from the XY-problem, not seeing that this is a flaw with the standard \. I would be surprised if there are any use cases that rely on \'s interaction with evaluate as currently specified.
Replies
Actually the current behaviour for blocks is that comments work within a block, but end at block boundaries. Lines in blocks are something that LIST, , and the block editor know about, but PARSE and PARSE-NAME just see the block as if it was one line, so a standard ( will do multi-line comments in blocks.
As for your questions:
I have not had problems due to the difference. This may be because I use ( only for stack comments and (rarely) for commenting out code withing a line, and so never use it across lines; instead, I use '' for big blocks of comments. I guess the current scheme might lead to problems if you copy and paste code containing a multi-line comment from a file onto the user input device.
We certainly cannot change how comments are handled in files (it would break the files). If anybody uses open (end-of-block-terminated) comments in blocks, we cannot change the block handling. For the user input device, the question of multi-line comments should not be one of breaking existing code, except if people do things like piping code with open (eol-terminated) comments into a Forth system. So if any change is possible at all, it's that of making 6.1.0080 multi-line.
Alternatively, one could de-standardize the behaviour of end-of-parse-area-terminated '(' comments in order to allow some systems to continue the current behaviour and some to change it to multi-line (and maybe some to produce an error message). But that means that you could not rely on a standard behaviour in this case, so if it's really something you want, it's not desirable.
If you make a proposal, somebody might come up with a real case that would break, but I don't expect it. Somebody might be nervous about breaking some unknown usage of end-of-parse-area-terminated '(' comments. And I guess the biggest problem will be that people will fail to see enough benefit in such a change to merit it. But you will only find out if you try.
No. I expect that several people will come up with real cases that would break.
Johan's durexForth comes with a vi-clone screen editor. It allows you to edit files on disk and press a key to evaluate them as if they were included. In an older version it had noncompliant \ ( evaluate etc that worked together to allow this. After making them compliant the editor cannot support \ comments by evaluating the buffer since it's specified to dump the entire parse area. He fixed it by having the editor split into individual lines first but that breaks ( comments. It cannot support both without essentially rewriting included.
IMO it's a flaw that included introduces the concept of "individual lines" but \ isn't extended to support them for this use case.
Hi Anton, indeed I once again misunderstood the standard, I had the wrong impression about the Block behaviour. I would agree that Interpret behaviour being different is maybe not enough of a problem to bother about. Sorry for wasting your time on a stupid question, although for sure your questions were illuminating to me.
I am interested in your comment about using '' for big block of comments. What word is that? What does it do? All the best /Johan
example - New Line characters in a string passed to EVALUATE
I thought about this problem many times over the years and want to share my view on it.
The real difficulty is once you start to EVALUATE lines like this:
s\" \ \n ( \n ) 1" EVALUATE
By the current standard, the first \ comment will parse and discard the rest of the parse area = the rest of string is not evaluated.
I think it would be preferable if:
- Evaluated strings are interpreted line-by-line
- EVALUATE would make the given string the input source (but NOT the input buffer!)
- REFILL would - similar to what it does for other input sources - make the input buffer the next line from the EVALUATE input source, and return TRUE, if possible.
(would have the same behavior as when parsing text files.
The word I meant is \ (it's tricky to render in markdown). I put it at the start of every commented line, and my editor helps me to do that.
I am closing this as it seems the issue is clarified.
example - New Line characters in a string passed to EVALUATE
I expect that systems don't treat newlines in EVALUATEd strings specially (are there systems that do?).
I expect that programs don't pass strings with newlines to EVALUATE.
If the latter is true, we could change the requirements as you suggest. But that means that you would have to convince all implementors of standard systems to change their implementation of EVALUATE (and they may be nervous about the possibility of the second point not being true).
It may be easier to just have a new word for interpreting multi-line strings. If you want that word to be standardized, you still have the hard work to convince standards committee members of its value. But if system implementors use such a word as factor of INCLUDED, it may have common practice.
This all makes me wonder what's behind the question by Stephen Pelc that started this thread.
This word introduces the concept of individual lines, and ( is extended to support them, however there is no similar extension to \, which continues to discard the entire rest of the buffer. The use case is a vi-clone screen editor in durexForth, it allows to edit files on disk and press a key to evaluate them as if they were included. That Forth's \ ( included evaluate etc were recently brought to standard, so when the editor evaluates its buffer then \ comments are broken, but if it separates and evaluates each line one-at-a-time then ( comments are broken. It would have to essentially reimplement included on in-memory buffers, maybe provide an non-standard definition of one of ( or \. I'm actually not really sure how it could be fixed. Other than:
I would propose a new section 11.6.2.2535 so that, when blk is 0, \ discards "the rest of the line," where that's defined to be the same as in included here.
Cross-reference durexForth author Johan's question about (. My opinion is that he's suffering from the XY-problem, not seeing that this is a flaw with the standard \. I would be surprised if there are any use cases that rely on \'s interaction with evaluate as currently specified.
Cross-reference also ruv's post about exactly this problem.
example - New Line characters in a string passed to EVALUATE
Yes, it would be interesting to hear from Stephen. Personally, I want to achieve following: When loading a text file to RAM and then EVALUATE:ing the buffer, the results would be similar to if the text file was directly INCLUDED.
Since this intent is directly related to file operations, I think it would make sense if this augmented behavior was described in the optional File-Access word set.
Some of the changes would then be straight-forward. For example, for REFILL the standard states in 11.6.2.2125: "When the input source is a text file, attempt to read the next line from the text-input file." One could simply expand this to encompass the EVALUATE string. Same procedure with (. I imagine other changes would be relatively minor.