Forth Refactored

In which my head exploded so I had to simplify things

Because my Forth interpreter is effectively event driven i.e. the input parser parses and then calls the interpreter to interpret one word, it gets very complicated when an executing word needs to pull something off the input parser. It has to exit the interpreter loop (which is really several nested loops) and wait for the next input from the input parser. Then, when it gets the input, it has to know to resume where it left off rather than just interpret the word as a new word to be executed. This means a lot of state is needed and it all has to be tested. My implementation of evaluate added to that in such a way that it was virtually impossible to figure out how to make it work.

There are three reasons for exiting the execution loop in resume()

• The current word of the input parser is finished – tested by the word number being 0 (by convention the currently interpreting word is word 0) and the word index being at the end of word 0.
• The current word is a word that needs something from the interpreter – tested by the state being .needsWordName
• evaluate is being executed and words are needed from the input parser to evaluate – tested by a flag called shouldExitToParser

Ultimately, the goal is to control exiting from the execution loop solely by the flag shouldExitToParser. I started by targeting the .needsWordName state. This was needed because several words need a word name. The primitive .char is one example. The sequence of steps needed to execute it is more or less as follows:

save current state and word to the control stack
change state to .needsWordName
exit to parser
<< get the next word >>
restore the state and word
complete the .char 

By eliminating the need to change to the .needsWordName state, we could obviate the need to save the state. Also, we wouldn’t need to to split the execution of the primitive, the first part in execute() and the second part in complete().

The refactoring is accomplished by creating a new primitive called .parseWord that simply sets the shouldExitToParser flag. The previous functionality is then implemented by changing the .char primitive to only do the bit after getting the word and creating a new Forth definition as follows:

: char .parseWord .char ;

Having done this for all of the words that need to get a word from the input parser, I have found several advantages:

• the complete() function goes away
• the .needsWordName state goes away. There are now only two machine states, which means it is possible we can replace it with a simple boolean.
• all this leads to a performance increase of about 10%

The code for this stage of the refactoring is in tag blog-1338-2.