Forth TX16 - new Forth interpreter for Commander X16

[vasyl]

I've just uploaded the first version of Forth TX16 (or ForthX16, I haven't decided what sounds better yet) to Github: https://github.com/VasylTsv/ForthX16
This a complete and enhanced port of my older weekend project, Forth for a certain ancient virtual machine. Unlike that one, this is written in proper 6502 assembly and should run on the hardware just fine - though I've only tested in on the emulator.
It is completely compliant with Forth 2012 and implements the core subset and some extensions. Runs the entire relevant subset of the popular test suite without errors. Note that this is a large stack version of Forth - unlike many other 8-bit implementation, it has both data and return stack set to 1K each. This may make it slightly slower, but will allow much more complex programs.
I will continue working on it - my goal is to make it a bit smaller (and maybe even a bit faster) to the point when it could be placed in 8K ROM. Haven't tested in on C64 yet, but getting it working there is also among the goals. Of course, some features and such. I will likely upload some older code samples and benchmarks from different sources - the interpreter seems to be very compatible with everything I throw at it.
So, if you are into the Forth, take it for a spin, and provide any suggestions and feedback here.
There were two issues I've ran into while working on this port.

• It is expected that some standard file system features cannot be implemented (like positioning in files). This is fine, the file support is somewhat limited as a result. But I could not get OPEN call to return an error on a missing file. This causes a number of small issues that I still need to investigate.

• I could not implement asynchronous KEY word with KERNAL functions - my first though was to use SCNKEY and then kill the keyboard buffer, but there is no function to clear the keyboard buffer in X16, and the buffer itself is not where it was on C64.

That said, I don't have past experience with programming C64 (grew up on Atari myself) and my 6502 is a bit rusty. So, again, all suggestions are welcome.

[Stefan]

I think you must try to read one byte to get file not found.

How are source files formatted?

I’m very interested in Forth. However, I might not be smart enough to learn it

[vasyl]

Do you mean the Forth files that the interpreter accepts? Those are just regular (sequential) text files with single CR for newline (I have plans to get the interpreter less picky about line endings and tabs though). Lines should not be longer than 100 characters, but that's also pretty reasonable requirement for a small system file format. I basically used Notepad++ for everything, set the line ending to "Macintosh (CR)" style, and made sure everything is in upper case. I did not support the old block format, there's just no point.

I will have to experiment with the single byte read, thanks for the suggestion. It does make certain sense, even if very inconvenient. But it was quite expected that the file I/O is going to be a bit of pain. Frankly, I was surprised when I got not just file loading, but nested loading.

[Stefan]

Yes, those are the files I was asking about.

The text editor I’ve been working on outputs CR style line breaks if you are in PETSCII mode. It might work out of the box for your Forth.

[vasyl]

I think I've figured out that issue with opening files - it appears that OPEN does not set C flag (not sure if it by design), so it should be followed by READST. Did not help that there were some bugs in my implementation of FILE-STATUS word that triggered as soon as I've fixed FILE-OPEN. In any case, uploaded a fixed version. There are still issues around the file handling, but it is more solid now.
While at it, tried it on a C64 emulator. It kinda works, but there are some artifacts. Most likely, some differences in memory map between C64 and X16, nothing major. I will fix it in the future (or make a separate version for C64 if it makes more sense).

[vasyl]

Had another round of updates, these should make the system nicer to use:
- The interpreter will now accept all variations of line endings
- TAB characters in input files are treated as spaces
- STOP key will interrupt the execution

[vasyl]

Rather significant rewrite, but I have achieved a major milestone - the entire core fits into 8K! There were some compromises, but to offset that, this version supports search order per the standard.
The code is in rather rough shape - I had to move a lot of pieces, and there were a lot of changes. It is using token threaded model now with most tokens being just one byte in size. Overall, this may well be the most compact Forth core out there.
In any case, it is functional, but there are a few feature that may be broken - WORDS is incomplete, and I would not recommend trusting FORGET and MARKER, these are most likely broken. These will be fixed in the future revisions. I will be also doing a major cleanup: there are incorrect and obsolete comments all over the place, some things need to be documented, there are dozens of small optimizations still possible (I estimate I can save ~200 bytes still and I need that slack to fix issues).

[BruceRMcF]

Rather significant rewrite, but I have achieved a major milestone - the entire core fits into 8K! There were some compromises, but to offset that, this version supports search order per the standard.

Excellent job! Is it intended to reside in LowRAM or in a HighRAM segment with compiled code in LowRAM? My inclination would be the former, with HighRAM accessed using the already worked out BLOCK approach.

The code is in rather rough shape - I had to move a lot of pieces, and there were a lot of changes. It is using token threaded model now with most tokens being just one byte in size. Overall, this may well be the most compact Forth core out there. ...

Yes, whenever I have tinkered with it, either token or bit/token threaded models come in with the lowest footprints on a 65xx system.

[vasyl]

Thanks! I did not experiment with the split memory yet, but that's the idea to have it easily configurable for the core and the compiled code to reside in different memory segments - and the core is intentionally immutable. I actually did not even check how to load code high on X16 (or C64 for that matter), my experience is more with Atari 8-bits where it is a very simple thing to do, I assume this architecture has some mechanisms as well.

[vasyl]

New big update with a lot of fixes - no more issues with WORDS, FORGET, etc. A lot of other small changes, like improved error messages, faster token lookup, etc. The interpreter is now case-insensitive - combined with support for all flavors of line endings this means that it will accept a lot more original files as is.
Checked the more advanced features of ACME macro language and found quite a few to improve the code organization.
And with all these improvements, it still fits 8K with 20 bytes to spare. I am quite surprised at how many bytes I've managed to save already with all of the 6502 trickery - and I am pretty sure I could find another 50 bytes or so if I need them.
Not stopping here. The code still need reorg and documenting, I need to create a toolkit to make it a proper programming tool. It almost works on C64, will probably branch for that version. Atari version is also in plans.