Page 1 of 1
Compression Unpackers (Prog8)
Posted: Fri Jul 28, 2023 12:40 am
by badmai
I was testing different compression methods, maybe useful to someone.
LZSA A2(what the cx-16 uses in kernal)
LZSA A1
LZ4 (used -12 so it's "HC")
RLE - just a standard run length
SAL/ZX0 - used the salavdor compressor
From the screenshot....Times are in jiffies cause i'm not sure about another timing method on the cx16 yet
Load - lower is better, how long to load the data
Size - file size
CRC - of the packed data
Unpack - lower is better, how long did it take to unpack
Check - crc of the unpacked data, should match the "IMAGE.BIN" (60408)
*Screenshot is using sdcard, so should match close to real hardware
Included source code (Prog8), and sdcard version
LOAD"TEST.PRG",8,1
RUN
Links to the great projects this is testing:
LZSA
https://github.com/emmanuel-marty/lzsa/tree/master
LZ4
https://github.com/lz4/lz4
Salavdor
https://github.com/emmanuel-marty/salvador
Re: Compression Unpackers (Prog8)
Posted: Fri Jul 28, 2023 12:50 am
by desertfish
- the LZSA decompression is that using the kernal routine or did you write a decompressor for that as well?
- wow that Salvador is super tiny compressed size and very fast decompression!
- I don't see the prog8 source code in the zip file...
Re: Compression Unpackers (Prog8)
Posted: Fri Jul 28, 2023 12:58 am
by badmai
Yes that is the kernal routine, why I decided to do these tests, so slow!!
I just downloaded to test, it's in a folder called "PROG8_SRC" ?
yes I should say salvador is the packer, the method is called ZX0
Re: Compression Unpackers (Prog8)
Posted: Fri Jul 28, 2023 1:08 am
by desertfish
I was dumb, somehow I glossed over the src folder, dunno why that happened. thanks.
why is the ZX0 one so fast though? in this graph it is only average
https://introspec.retroscene.org/compre ... 210128.png
Re: Compression Unpackers (Prog8)
Posted: Fri Jul 28, 2023 1:31 am
by badmai
I'm not sure really
this was a 1 sample test, just to test the compressor and unpack some data
if you do some tests make sure you use the "-classic" flag, I missed that and the unpacker was trying to unpack v2
Edit- Give me a few minutes, I'll post all my setting used so we are on the same page....
Re: Compression Unpackers (Prog8)
Posted: Fri Jul 28, 2023 1:44 am
by badmai
here are the parameters for each packer I used for testing:
lzsa -f2 -r -m5 image.bin IMAGE.LZA
lz4 -12 image.bin IMAGE.LZ4
lzsa -f1 -r -m5 image.bin IMAGE.LZ1
salvador -classic -v image.bin IMAGE.SAL
Re: Compression Unpackers (Prog8)
Posted: Mon Aug 26, 2024 1:33 am
by badmai
revisiting some compression tests...
I compressed some 64x64 @4bit sprite frames trying to decompress real time
Original: 2048 bytes per frame or 16K for all
most frames compressed to under 600 bytes
16384 to 3704 bytes for all frames
Decompress seems to take about 1/2 a frame for a single sprite
note:
I modified the decompress routine to also write to vera memory for the sprite
Since we don't have random access to video memory, a ram buffer is required for the decompressed size
as the buffer is filled, it also fills vera memory
Demo shows 3 sprites animated only using 2K instead of 16K for all frames loaded to VRAM
*You can press ESC to exit demo
Try It Now!
Re: Compression Unpackers (Prog8)
Posted: Mon Aug 26, 2024 4:54 pm
by desertfish
in the sprite code above, why is there a custom written decompression routine? Can't it use the one built into rom ?
Re: Compression Unpackers (Prog8)
Posted: Mon Aug 26, 2024 9:40 pm
by badmai
Speed, LZSA1-m5 is alot faster to unpack and I wanted to test the fastest I currently have.
The ROM routine would work fine, just not the fastest option.
Re: Compression Unpackers (Prog8)
Posted: Tue Aug 27, 2024 2:54 pm
by desertfish
aha it's a different compression format, I didn't notice that.