Digest #34 2018-05-26
Contributions
Problem
FIND has several problems:
It takes a counted string.
Its interface is designed for single-xt+immediate-flag systems (which cannot implement FILE S" correctly in all cases), and there is allowance for a STATE-dependent result to support other systems.
As currently specified (there is a proposal for fixing that), it fails to meet the goal of guaranteeing that the user-defined text-interpreter (outlined in the rationale of COMPILE,).
As a consequence of problem 2, the following implementation of ' is wrong:
: ' bl word find 0= -13 and throw ;
This ' produces the wrong result in the following case on Gforth:
: x ' ; immediate
] x s" [
And that's because Gforth implements S" correctly, and FIND such that the user-defined text interpreter works. The following definition would fix this problem on Gforth:
: ' bl word state @ >r postpone [ find r> if ] then 0= -13 and throw ;
but actually the specification of FIND currently is sufficiently unclear that we cannot be sure that that's guaranteed to work, either.
SEARCH-WORDLIST has a related, but different set of problems:
It produces results with a varying number of stack items. In some situations that makes it more cumbersome to use.
Its interface is designed for single-xt+immediate-flag systems, but this time without allowing a STATE-dependent result. As a consequence, Gforth produces the xt representing the interpretation semantics of the word, independent of STATE, with no way to get the compilation semantics of the word.
In the general case, given that the xt part of the result does not represent a part of the compilation semantics, the 1/-1 part of the result is pointless.
Solution
Introduce FIND-NAME (for the search order) and FIND-NAME-IN (for
specific wordlists). Both take strings in c-addr u form, and both
return the name token of the found word (or 0 if not found). We can
then go from the name token to the interpretation semantics with
NAME>INTERPRET,
and to the compilation semantics with
NAME>COMPILE.
Typical use
: ' parse-name find-name dup 0= -13 and throw name>interpret ;
: postpone
parse-name find-name dup 0= -13 and throw
name>compile swap postpone literal compile, ; immediate
\ user-defined text interpreter
: interpret-word
parse-name 2dup find-name if
nip nip state @ if name>compile else name>interpret then execute
else
... \ process numbers
then ;
\ alternative:
defer name>statetoken ( nt -- ... xt )
: [ ['] name>interpret is name>statetoken false state ! ;
[ \ initialize STATE and NAME>STATETOKEN
: ] ['] name>compile is name>statetoken true state ! ;
: interpret-word
parse-name 2dup find-name if
nip nip state name>statetoken execute
else
... \ process numbers
then ;
Remarks
FIND-NAME and FIND-NAME-IN are natural factors of all words that
look up words in the dictionary, such as FIND, ', POSTPONE, the text
interpreter, and SEARCH-WORDLIST. So implementing them does not cost
additional code in the Forth system, only some refactoring effort.
This approach is not compatible with the cmForth approach and Mark Humphries' approach, because they both use one word header each for interpretation and compilation semantics. This problem already exists for the other words that deal with name tokens, but the present proposal would make name tokens more important, and systems that do not support them less viable. However, both approaches have been known for at least two decades, and have seen little to no uptake in standard systems. And we have good approaches for implementing systems with name tokens, so excluding these approaches is not a significant loss.
Proposal
Add the following words:
FIND-NAME ( c-addr u -- nt | 0 )
Find the definition identified by the string c-addr u in the current search order. Return its name token nt, if found, otherwise 0.
FIND-NAME-IN ( c-addr u wid -- nt | 0 )
Find the definition identified by the string c-addr u in the wordlist wid. Return its name token nt, if found, otherwise 0.
Reference implementation
Implementing FIND-NAME-IN requires carnal knowledge of the system.
: find-name {: c-addr u -- nt | 0 :}
get-order 0 swap 0 ?do ( widn...widi nt|0 )
dup 0= if
drop c-addr u rot find-name-in
else
nip
then
loop ;
Testing
Experience
Gforth has implemented FIND-NAME and (under the name
(search-wordlist)) FIND-NAME-IN since
1996.
No problems were reported or found internally.
Several other systems have been reported to implement FIND-NAME
under this or other names (e.g., FOUND in ciforth).
Replies
Do you want the traditional user-defined text interpreter (e.g., as outlined in the rationale of COMPILE,) to work on VFX? The new description specifies exactly as much as is necessary to guarantee that, nothing more. If you need more breathing space, you would break these user-defined text interpreters.
As for unintended consequences, if someone uses a blemish in the standard like the current description of FIND as an excuse to damn the whole standard, a consequence is to fix the blemish.
Since we cannot use COMPILE, for words that parse or affect the stack, surely NAME>COMPILE has to be able to return the xt of words such as the suggested NDCS, which can handle parsing or stack effects.
OK. If there's no advantage and no one does it, let's rule it out. The most important thing to come out of this long discussion, I think, is that there is a difference between providing an extra definition for default and non-default compiling semantics. The first needs a stack effect of xt -- , should be available to COMPILE, and returns a default flag in FIND. The second can have any stack picture, and returns an immediate flag in FIND. There has been much confusion because of the use of the latter approach, whether through STATE or dual xts, to optimise default semantics. You can't stop a user doing that, of course, but maybe a system shouldn't.
Optimising solely through COMPILE, makes things clearer. An word can then have both a NDCS and an optimised compilation of its interpretation semantics
Let's make it concrete:
: s"-int '"' parse save-mem ;
: s"-comp '"' parse postpone sliteral ;
' s"-int ' s"-comp interpret/compile: s"
s\" s\"" find-name name>compile ( xt1 xt2 )
A correct result at the end of this piece of code is: xt1 is the xt of S"-comp and xt2 is the xt of execute. In compilation state, FIND returns the xt of S"-comp and 1.
Now your idea seems to be a different result for the piece of code above: xt1 should be the xt of S", and xt2 should be the xt of NDCS,. That may be correct for NAME>COMPILE, but what does your FIND do, if you want to support the traditional user-defined text interpreter (which uses only EXECUTE and COMPILE,)?