For the next version of Prog8, I've added two new routines to its "compression" library.
The library already has packbits-RLE routines, but these two will be added:
- ZX0 decoder https://github.com/einar-saukas/ZX0 - I learned about this one from your own tests, it's super fast and has very good compression ratio
- TSCrunch decoder, https://github.com/tonysavon/TSCrunch - extremely fast decrunching (approaching RLE speeds) and still having a decent compression ratio
Compression Unpackers (Prog8)
-
desertfish
- Posts: 1123
- Joined: Tue Aug 25, 2020 8:27 pm
- Location: Netherlands
Re: Compression Unpackers (Prog8)
Nice, I'm a big fan of TSCrunch...did you find it from my discord suggestion or run across it on your own?
Do you plan to handle all the different ways to decompress?
1. mem to mem
2. mem to inplace
3. mem to vram
*my current mem to vram is limited to $0000 - $FFFF vram, but it works
Do you plan to handle all the different ways to decompress?
1. mem to mem
2. mem to inplace
3. mem to vram
*my current mem to vram is limited to $0000 - $FFFF vram, but it works
-
desertfish
- Posts: 1123
- Joined: Tue Aug 25, 2020 8:27 pm
- Location: Netherlands
Re: Compression Unpackers (Prog8)
I found it as part of a list of compressors for 8 bit systems
I think I will eventually add the inplace decompression routine from it as well, because that is handy when loading a block of compressed data and decompressing it without requiring a second memory buffer / ram bank
I don't think I'm able to change the routine in a performant way to enable direct-to-vram decompression.
The problem seems to be with a lot of those algorithms that they need back-reference copies and those are expensive to do from vram. It looks like its much faster to just decompress into regular ram and then use an optimized copy loop to write all of it to vram. So I think you are limited to 8Kb chunks for this. Unless someone can make the routine bank-aware
I think I will eventually add the inplace decompression routine from it as well, because that is handy when loading a block of compressed data and decompressing it without requiring a second memory buffer / ram bank
I don't think I'm able to change the routine in a performant way to enable direct-to-vram decompression.
The problem seems to be with a lot of those algorithms that they need back-reference copies and those are expensive to do from vram. It looks like its much faster to just decompress into regular ram and then use an optimized copy loop to write all of it to vram. So I think you are limited to 8Kb chunks for this. Unless someone can make the routine bank-aware
-
desertfish
- Posts: 1123
- Joined: Tue Aug 25, 2020 8:27 pm
- Location: Netherlands
Re: Compression Unpackers (Prog8)
TSCrunch in-place decrunch routine has been added too. See description at
https://prog8.readthedocs.io/en/latest/ ... perimental
It's a bit weird because tscrunch in this mode requires the data files to be prg files (with a 2-byte load header) but I figured it out and it works beautifully
https://prog8.readthedocs.io/en/latest/ ... perimental
It's a bit weird because tscrunch in this mode requires the data files to be prg files (with a 2-byte load header) but I figured it out and it works beautifully
Re: Compression Unpackers (Prog8)
here is my vera ram version if you want to look/test.
it's NOT completely optimized yet, wanted to get it working first then fine tune.
*I don't handle above $FFFF yet
I really need to do some speed tests to see how it's landing compared to the ram then copy as you suggested.
it's NOT completely optimized yet, wanted to get it working first then fine tune.
*I don't handle above $FFFF yet
I really need to do some speed tests to see how it's landing compared to the ram then copy as you suggested.
tscrunch.zip- (1.36 KiB) Downloaded 31 times