,---------------.
| Contributions |
`---------------´


,------------------------------------------
| 2020-06-10 02:47:06  ruv  wrote:
| comment - Recognizer for locals
| see: https://forth-standard.org/standard/locals#contribution-137
`------------------------------------------
May a local be unfindable via `FIND` in a standard Forth system?

The answer is not stated explicitly. But the consequence of other clauses is that yes, it may:

1. Locals should be searched before the search order.
2. [`FIND`](https://forth-standard.org/standard/search/FIND) (as well as [`FIND-NAME`](https://forth-standard.org/standard/core/FIND#reply-174)) considers the search order only (if it's present).
3. A system is not obligated to use `FIND` for finding definition names in the Forth text interpreter.

So, it means even more: `FIND` cannot find locals in a standard Forth system that provides the optional Search-Order word set.

In any case, the result is that the locals may be recognized by a recognizer before other Forth definitions.

-----

What does the following phrase mean: "The system need not maintain these identifiers in the same way it does other dictionary entries as long as they can be found by normal dictionary searching processes" ?   Does "they" refer to "other dictionary entries" or to "these identifiers"?




,---------.
| Replies |
`---------´


,------------------------------------------
| 2020-06-10 12:29:04  ruv  replies:
| proposal - Recognizer RfD rephrase 2020
| see: https://forth-standard.org/proposals/recognizer-rfd-rephrase-2020#reply-375
`------------------------------------------
### Why do we need standard token descriptors?

In many cases a user-defined recognizer returns either a token of these basic data types: `xt`, `nt`, `x`, `xd`, `f`, `(c-addr u)`, or a tuple of such tokens. Also, the most standard recognizers (i.e. that can be standardized) return some of these tokens too. So, one argument is that the corresponding descriptors have very high factor of reusing.

The second argument is that they are needed to analyze a result of the standard compound recognizers (if any).

The third argument is that in some cases their can help to create a new user-defined descriptor.

### One approach to create descriptors

If a user-defined recognizer returns a token that is just a tuple of other known tokens, a descriptor for this token can be created from the tuple of the corresponding descriptors. I.e., there is no need in this case to specify interpreting action, compiling action, reproducing action, etc.

For example, in my [comparison](https://ruv.github.io/forth-design-exp/resolver-vs-recognizer.xml) of Recognizer and Resolver APIs  a token is represented as `( d-numerator u-denominator )` pair. And the descriptor `TD-3LIT` for this token can be defined as:

```
TD-2LIT TD-LIT 2  DESCRIPTOR-COMPOSITE{ TD-3LIT }
```

Or, if a user needs a distinct descriptor for an already known token:
```
TD-FLIT 1 DESCRIPTOR-COMPOSITE{ TD-FLIT-SPECIAL }
```

OTOH, at the moment I don't sure that such a method of creating a descriptor is worth to be standardized.

I personally prefer a more easy way, that is based not on the token descriptors but on the token translators. For token descriptors, not every user-defined token descriptor can be created from other standard descriptors. But **any** user-defined token translator can be **always** created from other standard token translators, so we don't need to specifying the particular actions at all.