,---------------. | Contributions | `---------------´ ,------------------------------------------ | 2023-09-10 17:16:18 UlrichHoffmann wrote: | proposal - 2023 Standards meeting agenda (2023-09-13 to 2023-09-15) | see: https://forth-standard.org/proposals/2023-standards-meeting-agenda-2023-09-13-to-2023-09-15-#contribution-306 `------------------------------------------ # 2023 Forth Standards Meeting Agenda 13 Sept 2023 12:00-19:00 MEST 14 Sept 2023 09:00-19:00 MEST 15 Sept 2023 09:00-13:00 MEST On site at Hotel Villa Aricia, Rome, Italy For latest details see chat.forth-standard.org See also: euro.theforth.net, forth-standard.org ## Schedule #### Wednesday, 13th September 2023 (MEST) 12:00 Get together - Setup your gear and smalltalk 12:15 Call to order - get ready 12:30 Session 1 14:30 Bio Break 15:00 Session 2 17:00 Bio Break 17:15 Session 3 19:00 End of main sessions Workshops #### Thursday, 14th September 2023 (MEST) 09:00 Get together - Setup your gear and smalltalk 09:15 Call to order - get ready 09:30 Session 4 11:30 Lunch Break 12:30 Session 5 14:30 Bio Break 15:00 Session 6 17:00 Bio Break 17:15 Session 7 19:00 End of main sessions Workshops #### Friday, 15th September 2023 (MEST) 09:00 Get together - Setup your gear and smalltalk 09:15 Call to order - get ready (please be online by now) 09:30 Session 8 11:15 Bio Break 11:30 Session 9 13:00 End of Standards Meeting afternoon euroForth conference ## Agenda ### Participants - Welcome - Determine the persons present - Meeting transcript ### Review of Procedures - How we organize this meeting - Progress of current work Draft Document update (last draft is from 2019) Are we ready for a new Standard snapshop (Forth2023)? How can we speed up our work? How we can better serve the Forth community? How can we encourage Forthers to submit proposals? What are addtional topics for future standardisation? ### Reports - Chair - Editor - Technical Officer - Treasurer ### Election/Confirmation of officers If you would like to stand for election, please suggest your name and please shortly introduce yourself. We might have to elect (by secret ballot) or confirm a Chair, Editor, Technical Officer and a Treasurer. - Chair (currently Ulrich Hoffman) - Editor (currently Peter Knaggs) - Technical (currently Gerald Wodni) - Treasurer (currently Bernd Paysan) ### Review of Proposals/Contributions Proposals can be retrieved on the fly from forth-standard.org/proposals Please follow the links below for up to date lists of proposals and contribuions. - [Proposals in the state formal](https://forth-standard.org/contributions/proposal/formal) - [Proposals in the state informal](https://forth-standard.org/contributions/proposal/informal) - [Contributions on forth-standard.org](https://forth-standard.org/contributions) This includes also the above proposals. We will skip them on the fly. Revision of contributions since the interim meeting in March 2023 ### Workshop Topics Workshops are topics for discussion outside the formal meeting. We will collect topics on the fly during the meeting's discussions. ### Consideration of proposals + CfV votes Which proposals should go for vote? Any topics for proposal in the pipeline? ### Workshop reports Let's collect the results of our workshops. ### Matters arising Whats up? ### Any other business Something else? ### Adjourn Date of next meeting When shall we three meet again? In thunder, lightning, or in rain? ---- ,---------. | Replies | `---------´ ,------------------------------------------ | 2023-08-05 04:51:11 MitchBradley replies: | proposal - Include a revised 79-STANDARD Specification for "><" To "Core Ext" | see: https://forth-standard.org/proposals/include-a-revised-79-standard-specification-for-to-core-ext-#reply-1031 `------------------------------------------ Have you looked at other current implementations to see how they solve this problem? ,------------------------------------------ | 2023-08-05 09:10:21 rcfg7943 replies: | proposal - Include a revised 79-STANDARD Specification for "><" To "Core Ext" | see: https://forth-standard.org/proposals/include-a-revised-79-standard-specification-for-to-core-ext-#reply-1032 `------------------------------------------ Do you want a word equivalent to htonl, htons, ntohl and ntohs in C as included in arpa/inet.h or netinet/in.h? Or do you want a word to swap bytes of cells? The former has to take the type of processor into account whereas the latter does not. If the former then I don't think it belongs in CORE EXT. If the latter then why? I'm a strong believer in less is more. And I think Forth-83 had that word too. ,------------------------------------------ | 2023-08-06 17:40:19 MitchBradley replies: | proposal - Include a revised 79-STANDARD Specification for "><" To "Core Ext" | see: https://forth-standard.org/proposals/include-a-revised-79-standard-specification-for-to-core-ext-#reply-1033 `------------------------------------------ As the author of Open Firmware, a Forth-based system that became a national standard (IEEE 1275-1994) . Open Firmware was used on millions of computers, of different byte orders and word sizes and alignment restrictions, produced by many different companies, Open Firmware supported portable device drives that would work unmodified across the full range of systems. Thus I can claim relevant experience on this topic The typical use of byte-swizzling is to prepare or decode in-memory data structures that need to be passed to and from another device, whether another computer across a network or an I/O device that is controlled by memory-based descriptors. The data structure will have a defined byte order that might or might not match the host computer's byte order. Furthermore, the host might impose alignment restrictions different from the data structure. If you want to write portable code, then you cannot use a byte-swizzling primitive directly. You need a set of access wrappers akin to the "hton"/"ntoh" suite. Open Firmware drivers typically combine the host-dependent swizzling with unaligned-access, resulting in a set of works like "be-w@", "be-w!", "le-w@", "le-w!", and similar words for 32-bit and perhaps 64-bit access. It is a lot of words, but it actually solves the problem. Just having one swizzler primitive is "minimal", but doesn't actually get the job done. It can be useful as an implementation factor, but it shouldn't appear directly in portable code. Before proposing something, it would be helpful to study modern Forth implementations like VFX Forth, Swift Forth, gforth, etc, to see how they solve this common problem. A proposal that conforms to a modern system's usage is much more likely to succeed than something modeled on a 40+ year old standard that never achieved much commercial traction. ,------------------------------------------ | 2023-08-08 01:11:13 BerndPaysan replies: | proposal - minimalistic core API for recognizers | see: https://forth-standard.org/proposals/minimalistic-core-api-for-recognizers#reply-1034 `------------------------------------------ ## Author: Bernd Paysan ## Change Log: * 2020-09-06 initial version * 2020-09-08 taking ruv's approach and vocabulary at translators * 2020-09-08 replace the remaining rectypes with translators * 2022-09-08 add the requested extensions, integrate results of bikeshedding discussion * 2022-09-08 adjust reference implementation to results of last bikeshedding discussion * 2022-09-09 Take comments from ruv into account, remove specifying STATE involvement * 2022-09-10 More complete reference implementation * 2022-09-10 Add use of extended words in reference implementation * 2022-09-10 Typo fixed * 2022-09-12 Fix for search order reference implementation * 2022-09-15 Revert to Trute's table approach to call specific modes deliberately * 2023-08-08 Remove names for table access words; there's no usage outside POSTPONE seen; POSTPONE can do that without a standardized way. ## Problem: The current recognizer proposal has received a number of critics. One is that its API is too big. So this proposal tries to create a very minimalistic API for a core recognizer, and allows to implement more fancy stuff as extensions. The problem this proposal tries to solve is the same as with the original recognizer proposal, this proposal is therefore not a full proposal, but sketches down some changes to the original proposal. ## Solution: Define the essentials of the recognizer in a RECOGNIZER word set, and allow building upon that. Common extensions go to the RECOGNIZER EXT wordset. Important changes to the original proposal: * Make the recognizer types executable to dispatch the methods (interpret, compile, postpone) themselves * Make the recognizer sequence executable with the same effect as a recognizer * Make sure the API is not mandating a special implementation This replaces one poor man's method dispatch with another poor man's method dispatch, which is maybe less daunting and more flexible. The core principle is still that the recognizer is not aware of state, and the returned translator is. If you have for some reason legacy code that looks like : rec-xt ( addr u -- translator ) here place here find dup IF 0< state @ and IF compile, ELSE execute THEN ['] drop ELSE drop ['] notfound THEN ; then you should factor the part starting with state @ out and return it as translator: : translate-xt ( xt flag -- ) 0< state @ and IF compile, ELSE execute THEN ; : rec-xt ( addr u -- ... translator ) here place here find dup IF ['] translate-xt ELSE drop ['] notfound THEN ; The standard interpreter loop should look like this: : interpret ( i*x -- j*x ) BEGIN parse-name dup WHILE forth-recognize execute REPEAT 2drop ; with the usual additions to check e.g. for empty stacks and such. ## Typical use TBD ## Proposal: XY. The optional Recognizer Wordset A recognizer takes the string of a lexeme and returns a translator xt and additional data on the stack (no additional data for `NOTFOUND`): REC-SOMETYPE ( addr len -- i*x translate-xt | NOTFOUND ) # XY.3 Additional usage requirements ## XY.3.1 Translator **translator:** subtype of xt, and executes with the following stack effect: *TRANSLATE-THING* ( j\*x i\*x -- k\*x ) A translator xt that interprets, compiles or postpones the action of the thing according to what the state the system is in. `i*x` is the additional information provided by the recognizer, `j*x` and `k*x` are the stack inputs and outputs of interpreting/compiling or postponing the thing. # XY.6 Glossary ## XY.6.1 Recognizer Words **FORTH-RECOGNIZE** ( addr len -- i*x translator-xt | NOTFOUND-xt ) RECOGNIZER Takes a string and tries to recognize it, returning the translator xt and additional information if successful, or `NOTFOUND` if not. **NOTFOUND** ( -- ) RECOGNIZER Performs `-13 THROW`. If the exception word set is not present, the system shall use a best effort approach to display an adequate error message. **TRANSLATE:** ( xt-int xt-comp xt-post "name" -- ) RECOGNIZER EXT Create a translator word under the name "name". This word is the only standard way to define a user-defined translator from scratch. "name:" ( j*x i*x -- k*x ) performs xt-int in interpretation, xt-comp in compilation and xt-post in postpone state using a system-specific way to determine the current mode. ## XY.6.2 Recognizer Extension Words **SET-FORTH-RECOGNIZE** ( xt -- ) RECOGNIZER EXT Assign the recognizer xt to FORTH-RECOGNIZE. Rationale: FORTH-RECOGNIZE is likely a deferred word, but systems that implement it otherwise can use this word to change the behavior instead of using `IS FORTH-RECOGNIZE`. **FORTH-RECOGNIZER** ( -- xt ) RECOGNIZER EXT Obtain the recognizer xt that is assigned to FORTH-RECOGNIZE. Rationale: FORTH-RECOGNIZE is likely a deferred word, but systems that implement it otherwise, can use this word to change the behavior instead of using `ACTION-OF FORTH-RECOGNIZE`. The old API has this function under the name FORTH-RECOGNIZER (as a value) and this name is reused. Systems that want to continue to support the old API can support `TO FORTH-RECOGNIZER`, too. **RECOGNIZER-SEQUENCE:** ( xt1 .. xtn n "name" -- ) RECOGNIZER EXT Create a named recognizer sequence under the name "name", which, when executed, tries to recognize strings starting with xtn and proceeding towards xt1 until successful. **SET-RECOGNIZER-SEQUENCE** ( xt1 .. xtn n xt-seq -- ) RECOGNIZER EXT Set the recognizer sequence of xt-seq to xt1 .. xtn. **GET-RECOGNIZER-SEQUENCE** ( xt-seq -- xt1 .. xtn n ) RECOGNIZER EXT Obtain the recognizer sequence xt-seq as xt1 .. xtn n. **TANSLATE-NT** ( j*x nt -- k*x ) RECOGNIZER EXT Translates a name token; system component you can use to construct other translators of. **TRANSLATE-NUM** ( j*x x -- k*x ) RECOGNIZER EXT Translates a number; system component you can use to construct other translators of. ## Reference implementation: This is a minimalistic core implementation for a recognizer-enabled system, that handles only words and single numbers without base prefix. This implementation does only take interpret and compile state into account, and uses the STATE variable to distinguish. Defer forth-recognize ( addr u -- i*x translator-xt / notfound ) : interpret ( i*x -- j*x ) BEGIN ?stack parse-name dup WHILE forth-recognize execute REPEAT ; : lit, ( n -- ) postpone literal ; : notfound ( state -- ) -13 throw ; : translate: ( xt-interpret xt-compile xt-postpone "name" -- ) create , , , does> state @ 2 + cells + @ execute ; :noname name>interpret execute ; :noname name>compile execute ; :noname name>compile swap lit, compile, ; translate: translate-nt ( nt -- ) ' noop ' lit, :noname lit, postpone lit, ; translate: translate-num ( n -- ) : rec-nt ( addr u -- nt nt-translator / notfound ) forth-wordlist find-name-in dup IF ['] translate-nt ELSE drop ['] notfound THEN ; : rec-num ( addr u -- n num-translator / notfound ) 0. 2swap >number 0= IF 2drop ['] translate-num ELSE 2drop drop ['] notfound THEN ; : minimal-recognize ( addr u -- nt nt-translator / n num-translator / notfound ) 2>r 2r@ rec-nt dup ['] notfound = IF drop 2r@ rec-num THEN 2rdrop ; ' minimal-recognizer is forth-recognize ## Extensions reference implementation: : set-forth-recognize ( xt -- ) is forth-recognize ; : forth-recognizer ( -- xt ) action-of forth-recognize ; ### Stack library : STACK: ( size "name" -- ) CREATE 0 , CELLS ALLOT ; : SET-STACK ( item-n .. item-1 n stack-id -- ) 2DUP ! CELL+ SWAP CELLS BOUNDS ?DO I ! CELL +LOOP ; : GET-STACK ( stack-id -- item-n .. item-1 n ) DUP @ >R R@ CELLS + R@ BEGIN ?DUP WHILE 1- OVER @ ROT CELL - ROT REPEAT DROP R> ; ### Recognizer sequences : recognize ( addr len rec-seq-id -- i*x translator-xt | NOTFOUND ) DUP >R @ BEGIN DUP WHILE DUP CELLS R@ + @ 2OVER 2>R SWAP 1- >R EXECUTE DUP ['] NOTFOUND <> IF 2R> 2DROP 2R> 2DROP EXIT THEN DROP R> 2R> ROT REPEAT DROP 2DROP R> DROP ['] NOTFOUND ; #10 Constant min-sequence# : recognizer-sequence: ( rec1 .. recn n "name" -- ) min-sequence# stack: min-sequence# 1+ cells negate here + set-stack DOES> recognize ; : ?defer@ ( xt1 -- xt2 ) BEGIN dup is-defer? WHILE defer@ REPEAT ; : set-recognizer-sequence ( rec1 .. recn n rec-seq-xt -- ) ?defer@ >body set-stack ; : get-recognizer-sequence ( rec-seq-xt -- rec1 .. recn n ) ?defer@ >body get-stack ; Once you have recognizer sequences, you shall define ' rec-num ' rec-nt 2 recognizer-sequence: default-recognize ' default-recognize is forth-recognize The recognizer stack looks surprisingly similar to the search order stack, and Gforth uses a recognizer stack to implement the search order. In order to do so, you define wordlists in a way that a wid is an execution token which searches the wordlist and returns the appropriate translator. : find-name-in ( addr u wid -- nt / 0 ) execute ['] notfound = IF 0 THEN ; root-wordlist forth-wordlist dup 3 recognizer-sequence: search-order : find-name ( addr u -- nt / 0 ) ['] search-order find-name-in ; : get-order ( -- wid1 .. widn n ) ['] search-order get-recognizer-sequence ; : set-order ( wid1 .. widn n -- ) ['] search-order set-recognizer-sequence ; ## Testing TBD