Does Forth create

In which my head explodes.

create and does> are what is currently causing the unit tests of my Forth interpreter to fail. So what do these things do? Well, if you read the Forth specification, it becomes obvious that the main thing they do is make a nice smoothy out of your brain. Fortunately, I have found an online example that might make things a bit clearer. Here it is:

: n-vector ( n – ) ( i – address)
    create cells allot
    does> swap cells + ;

I’ve modified it to replace floats with cells. Floats is analogous to cells, but the size used is the size of a floating point number. Anyway, this defines an integer array with a name that comes from the interpreter. It works as follows:

create defines a name (taken from the interpreter stream) that puts the data space pointer from the moment of definition onto the stack. This pointer is known as the “data field” of the word. In the code above, create defines such a word and then allocates a number of cells based on the parameter to n-vector.

I’m not sure that the data field is something that has to be intrinsic or if we could just compile it into the definition of the word e.g. .pushNext <datapointer>. This latter seems to be the easiest option, but see below.

The code for create is implemented in tag blog-1277-1.

does> is a bit more confusing. Conceptually, what it does is append the succeeding words in the definition to the last defined word. The way it does it is quite clever: when you execute does> it pushes a return address onto the return stack like any other defined word. This return address is the location of the first word after does> in the definition. does> pulls this return address off the return stack and uses it to construct a goto that is appended to the end of the last definition. So, when you execute the last definition, instead of returning, it jumps into the middle of the definition containing the does>. When does> returns, because you’ve popped its return address, it actually returns to the location where the word definition containing does> would return to, thus, missing out the rest of the words in the word containing does>.

In the case of n-vector, does> adds a goto to the end of the word defined by the create above it that jumps to the swap immediately after the does, it then exits n-vector ignoring the remaining words.

From the implementation point of view, the first point is that does> cannot be a primitive. It needs to be a defined word so that it has a return address to to be pulled off the return stack. We also need a goto primitive. We could implement that by pushing the return address onto the return stack and just returning. However, that risks leaving the goto address on the return stack if we do this trick twice. Also, an explicit goto will be useful to implement tail recursion.

The second point is how do we represent a return/goto address as a constant in code. The simplest solution would seem to be to use the one half of the cell as the word number and the other half as the index in the word.

If we then modify the r> word to deal with return addresses by concatenating the two parts into a single value, we only need to create a new .goto primitive and does> can be implemented in Forth. The code for this is tagged blog-1277-2.

The following trace shows ho it works. The word does1 is defined as follows: does> @ 1 +.

calling DOES1: does> @ 1 +
stack: <0>
return stack: <1> .return(0, 1)
[150, 0] word: does>
calling does>: -9223372036854775724 compile, r> compile,
stack: <0>
return stack: <2> .return(0, 1) .return(150, 1)

At this point, we have just started executing does>. The data stack is empty and the return stack contains two return addresses. The lowest one is for the interpreter level and the return(150) points to offset 1 in word 150, which is the @ in does1. The large negative value in the definition of does> is the integer value of the .goto primitive.

[137, 0] word: lit>data 0x8000000000000054
stack: <1> -7fffffffffffffac
return stack: <2> .return(0, 1) .return(150, 1)
[137, 2] word: compile,
stack: <0>
return stack: <2> .return(0, 1) .return(150, 1)

Now we have pushed the .goto onto the stack and compiled it i.e. appended it to the last defined word.

[137, 3] word: r>
stack: <1> 100000096
return stack: <1> .return(0, 1)

Now we have removed the top return address from the return stack and put it on the data stack.

[137, 4] word: compile,
stack: <0>
[0, 0] word: .s
stack: <0>

Finally, we have compiled the return address i.e. appended it to the last defined word. Note that we skipped over the rest of does1 when we returned and we are executing a word at the interpreter level.