Commander X16

It might be hard to make everyone happy.

Maybe I should let the keyboard shortcuts be user configurable.

The current source code is not very far off. The shortcuts are just a list of PETSCII/ASCII values stored within the executable.

This could be changed so that the editor loads the shortcut list from a file on startup.

On 1/27/2022 at 4:46 AM, Stefan said:

It might be hard to make everyone happy.



Maybe I should let the keyboard shortcuts be user configurable.



The current source code is not very far off. The shortcuts are just a list of PETSCII/ASCII values stored within the executable.



This could be changed so that the editor loads the shortcut list from a file on startup.

That's how I would go. Everyone wants something different, and you can't please everyone with a fixed layout. Personally, for example, I prefer a more Windows standard shortcut set, with Control-C for copy, Control-V for paste, Control-H for search/replace, and so on. While I can get by on Nano, I find its shortcuts to be obtuse and often frustrating, since I rarely use it and spend my time on Visual Studio and Notepad++.

On the other hand, someone who spends their time in Linux or Unix would likely be most comfortable with Nano or maybe even VI if you're old school enough.

 

 

User-configurable key bindings are now in version 0.4.3.

I also made a simple tool to create the config file, also in the download section (X16EDIT-KEYBINDINGS.PRG).

And finally, I have now tried to support both Ctrl+key and left Alt+key combinations in the editor. Ctrl+key worked fine on MacOS, but there are a lot of problems at least in the emulator on Windows and Linxux as the emulator uses Ctrl+key combinations for its own purposes. Let me know how this works out, as I do not have the X16 Emulator set up on Windows or Linux myself.

The help screen in X16 Edit is embedded into the executable as ASCII text, and takes quite a lot of space.

I've been looking for ways to make the executable smaller. One interesting option is to compress the text with Huffman coding:

https://en.wikipedia.org/wiki/Huffman_coding

After a lot of fiddling, I finally got it to work.

The compression is done by a python script. Decompression is done in 65C02 assembly within the X16 Edit executable.

But how well does it work? Some metrics:

• 
  Original help screen text size: 1,818 bytes
  


• 
  Compressed text size: 1,093 bytes (60 % of the original)
  


• 
  Lookup tables needed for decompression: 194 bytes 
  

And the 65C02 decompression code also takes some extra space. In my first working implementation, the executable became 422 bytes smaller, a saving of about 23 %.

Another issue is speed. The help screen is not displayed instantly, but I think it still is quite fast. I uploaded a video so you may see the decompression code in action for yourself.

Have you tried using lzsa and the kernal's memory_decompress() routine?  If it works it would at least save you the need to include your own decompression routine

On 2/18/2022 at 10:24 PM, Stefan said:

The help screen in X16 Edit is embedded into the executable as ASCII text, and takes quite a lot of space.



I've been looking for ways to make the executable smaller. One interesting option is to compress the text with Huffman coding:



https://en.wikipedia.org/wiki/Huffman_coding



After a lot of fiddling, I finally got it to work.



The compression is done by a python script. Decompression is done in 65C02 assembly within the X16 Edit executable.



But how well does it work? Some metrics:

• 
  Original help screen text size: 1,818 bytes
  


• 
  Compressed text size: 1,093 bytes (60 % of the original)
  


• 
  Lookup tables needed for decompression: 194 bytes 
  

And the 65C02 decompression code also takes some extra space. In my first working implementation, the executable became 422 bytes smaller, a saving of about 23 %.



Another issue is speed. The help screen is not displayed instantly, but I think it still is quite fast. I uploaded a video so you may see the decompression code in action for yourself.



Compressed help screen.mov

The text compression I'm using for Asteroid Commander is a little different and might work better for you. First, I put all the text I have so far in a single file. Then I copied it into a word cloud generator, and set that up to not ignore small common words like a, the, an, at, etc.

From that, I created a vocabulary of the most common words. I chose space for 384 words, but I have a lot more text; you might be able to get away with 64 words. So my vocabulary list is just two byte pointers to the start of the word. The vocabulary is stored in VRAM, the pointers are in low RAM. All the words are stored as their ASCII codes, but on the last letter bit 7 is set to 1, similar to the way error messages are stored in ROM.

To compress the text with only 64 words in the vocabulary, use codes 80-BF to represent those 64 words. Then to decompress, when a code in that 80-BF range is encountered, print a space and then extract the vocabulary word from VRAM.

To get 384 words I'm using codes 60-BF and E0-FF for one set of words (the short words) and the code 01 followed by another byte for a set of 256 words.

I'm getting about 3:1 compression using this method.

On 2/19/2022 at 11:57 AM, desertfish said:

Have you tried using lzsa and the kernal's memory_decompress() routine?  If it works it would at least save you the need to include your own decompression routine

Hi @desertfish!

I didn't remember that there was such a function in the Kernal.

I did a quick test just now.

Compiling the lzsa utility from the Github master worked without any issues on MacOS.

I compressed the help text with the following command, as advised in the X16 PRG:

• 
  lzsa -r -f2 <original_file> <compressed_file>
  

The output is a binary file that cannot be handled by a normal text editor. I imported the compressed file with the .incbin command supported by CA65.

The original file was 1,817 bytes, and the compressed file became 1,149 bytes (63 % of the original). Surprisingly, my own Huffman implementation did better, resulting in compressed size of 1,093 bytes (60 % of the original).

But as you said, I need not store my own decompression tables or routines, and the Kernal routine, which worked perfectly by the way, will in the end be more efficient.

My Huffman code decompresses directly to screen VRAM. As far as I understand, this is not possible with the Kernal lzsa decompress function. So you first need to decompress to RAM, and then copy from RAM to VRAM to actually display the text.

I also looked briefly at the method described by @Ed Minchau

I would assume that in order to gain compression a word that is replaced by a code needs to

• 
  be longer than one character
  


• 
  occur more than one time
  

Analyzing my help text there are:

• 
  25 words occurring 2 times
  


• 
  5 words occurring 3 times
  


• 
  5 words occurring 4 times
  


• 
  2 words occurring 5 times
  


• 
  1 word occurring 6 times
  


• 
  1 word occurring 7 times
  


• 
  1 word occurring 8 times
  


• 
  1 word occurring 9 times
  

A total of 41 words.

I might try to combine lzsa compression with the Ed's method to see where I end up.

On 2/23/2022 at 11:49 PM, Stefan said:

I also looked briefly at the method described by @Ed Minchau



I would assume that in order to gain compression a word that is replaced by a code needs to

• 
  be longer than one character
  


• 
  occur more than one time
  

Analyzing my help text there are:

• 
  25 words occurring 2 times
  


• 
  5 words occurring 3 times
  


• 
  5 words occurring 4 times
  


• 
  2 words occurring 5 times
  


• 
  1 word occurring 6 times
  


• 
  1 word occurring 7 times
  


• 
  1 word occurring 8 times
  


• 
  1 word occurring 9 times
  

A total of 41 words.



I might try to combine lzsa compression with the Ed's method to see where I end up.

It's not just the word that you're compressing here.  Every word has a preceeding space, so if you encounter one of these special codes, print a space and then the word.  You end up getting the word plus the preceeding space at the cost of just one byte, the same as just a space.  So, even one-character words like a or I can be compressed this way.

Yes, I guess that is possible.

But you cannot assume that every word boundary is marked by a blank space. In my help text, for instance, words may be preceded by line breaks instead of blank spaces. The decompression routine could, of coarse, take that into account, and output a preceding blank space only if the previous character was not a line break. This might somewhat reduce the gain of replacing one letter words by a code.