retro from the top down

Table of Contents

1 part 1 : overview

1.1 quick intro to retro

Retro is a concatenative programming language, related to forth, factor, and joy. Retro runs on a portable virtual machine called ngaro, and is self hosting, meaning it is written in itself.

The retro documentation includes a file, docs/Commentary.txt, which describes the process of building a new image from the bottom up. The actual code is in the following files:

image/meta.rx the assembler and metacompiler
image/meta-alt.rx same thing, but from vm primitives
image/kernel.rx the core retro commands

All code in the present file is taken from these sources, as they existed in October of 2012.

This is a top-down view of the bootstrap process, which means that the code will not actually compile in the order presented here. Rather, the point here is to see how all the parts fit together into a unified whole.

We will start at the "top" of the retro user interface, with the listener (the "ok" prompt), briefly discuss each word we encounter, then perform a recursive descent into its definition. The words will be introduced in breadth first order, but the definitions will be presented depth-first, so that we are implementing each word "just in time" for execution in the level above.

1.2 the listener

1.2.1 listen the word that does what you ask it to

t: listen  (   - ) repeat ok 32 # accept tib find process again ;

This is the main loop of the retro interpreter, taken from line 246 of kernel.rx, just under halfway through the file. After this point, the code generates a fresh dictionary, copies the newly-generated image over the original, and performs a virtual reboot of the system.

Once we understand how the listener works, we can simply follow the Commentary.txt file from the heading "Stage 3: Extend the Language" on line 1000. Therefore, we will only concern ourselves here with this the first two phases.

1.2.2 What do all those words mean?

First, note that word t: is simply the standard word : ("colon"), which begins a definition, except that the definition gets stored in the target image. Since t: is the first word of a definition, it should be the first sub-node that we visit, but we will treat it as : and visit that instead.

So. What do the other words mean? The following definitions are taken from doc/Core_Functions.rst, except for # and process, which are invisible after the reboot, and not part of standard retro.

: "- Calls create, changes class to .word, and turns compiler on
listen - Top level interpreter. Reads and process input
( "- Parse for ) and ignore everything it reads
repeat - Start an unconditional loop
ok - Displays the "ok" prompt
32 -n not found, so parsed as integer and pushed to data stack
# n- meta.rx: compile a literal (unnecssary after stage 3)
accept c- Read a string, ending with the specified character. The string is returned in tib
tib -a Returns address of text input buffer
find $-af Search for a name in the dictionary. Returns a dictionary header and a flag
process af- kernel.rx: run the word's class handler if found, otherwise parse as number
again - Jump to the code following the most recent repeat
; - Compile an exit into a function and stop the compiler

We will not show the lines for :, ;, or (...) in other definitions, but will repeat the internal definitions.

1.2.3 Understanding the stack comment.

The second column shows the stack comment for each word. The following symbols are used:

" indicates that the word reads input from the user or script
- separates the input (on the left) from the output (on the right)
# beats me. :(
n a number
c a character
a an address
$ a string
f a flag (boolean)
others it's just a comment, so sometimes ad-hoc names are used

The topmost item is always on the right, so for example ( ca-nf ) means "take an address, then a character off the stack, give back a number and then a flag". (Flags are almost always returned on top, so they can be easily tested with if .)

  1. TODO what does # mean ? ( number? counted string? )

1.2.4 Summary of listen

Let's take another look at listen then, and put it in our own words: 

t: listen  (   - ) repeat ok 32 # accept tib find process again ;

The listener is an infinite loop ( repeat .. again ) that shows a prompt ( ok ), reads characters from input until encountering a space ( 32 # accept , where 32 is the ASCII code for the space character ), looks the word up in the dictionary ( tib find ), and then performs various possible actions, depending on whether the word was found or not, and if so, what class handler is associated with it ( process ).

We will now explore each of these words in detail.

1.3 stepwise refinement and recursive descent

Having investigated the word listen and briefly described its component words, we are now going to give each of those words the same treatment in turn.

We will proceed depth first and left to right. Note that it's possible that a word is used in multiple places, and we may therefore need to define it earlier than the order in which we expected.

2 part 2 : recursive descent

2.1 listen > : ("colon") the word that makes new words

t: :        ( "-  )  create ' .word # last # @, d->class !, ]] vector? ;

2.1.1 TODO summary

2.1.2 breakdown

: "- Calls create, changes class to .word, and turns compiler on
t: "- meta.rx: compile to target image ( bootstrap version of : )
create "- Parse for a name and call header
' n- Place TOS here and increment heap by 1
.word a- Class for normal functions
#    
last -a Variable; pointer to most recent dictionary header
@, - meta.rx: assembler for @ (which fetches a value from the address at TOS)
d->class a-a Given a dictionary header, return the address of the class handler. Use @ to get the class handler.
!, - meta.rx: assembler for ! (which stores a value to the address at TOS)
]] - Turn compiler on
vector? - kernel.rx: either revectors the word immediately or writes two NOPs for revectoring later

2.2 TODO : > t: the compiler's compiler

: t: ( "- ) label: nop, nop, &m, reclass ;

2.3 TODO t: > label: 

2.4 TODO t: > &m, 

2.5 TODO t: > reclass

2.6 TODO listen > ( ("paren") the word that doesn't care what you say

2.7 TODO : > create 

2.8 TODO : > =' the quote

2.9 TODO listen > .word 

2.10 TODO listen > # proto literals

2.11 TODO listen > last at last, you find the words

2.12 TODO listen > @,= assemble and follow me

2.13 TODO listen > d->class pointer arithmetic

2.14 TODO listen > =!, take that, NOS!

2.15 TODO listen > ]] 

2.16 TODO listen > vector?  vector? I hardly know her.

2.17 TODO listen > ; STOP THE PRESS!

2.18 listen > ( paren ) # should have been done by this point

2.19 TODO listen > repeat the word that's just getting started

2.20 TODO listen > ok the word that lets you know everything is ok

t: ok      (   - ) compiler # @, not 0; drop, cr okmsg # puts ;

2.21 TODO listen > 32 the word that wasn't here

2.22 TODO listen > accept the word that wants you to show some character

2.23 TODO listen > tib the word that holds what listen heard

2.24 TODO listen > find the word that reads the dictionary

2.25 TODO listen > process the hidden word that makes the whole thing work

i: process ( af- ) 0 # !if xt:class jump: withClass then drop jump: number

2.26 TODO listen > again the word that picks up after repeat

3 TODO scraps (put these in the right place as we go along)

i: number  (   - ) tib isNumber? 0 # !if jump: build# then jump: notFound
i: build#  (   - ) tib toNumber ' .data # jump: withClass