So one thing that has been annoying me lately is the size of programs. So, I made a tool today that will compress programs into the smallest size possible. There is literally no change to execution, (other than just requiring more ram before starting), and the extraction routine is quite small, only 150 bytes or so. It currently works on any asm program; if enough people care I might add support for basic programs and raw binary input.

Download: https://github.com/MateoConLechuga/ConvHex/releases/latest

Here's some of the results I tested using a few programs:

Oiram (by Mateo): 36299 -> 17508 bytes
Atomas (by Rico): 11678 -> 6158 bytes
PacMan (by Mateo): 18916 -> 10342 bytes
Space Invaders (by JamesV): 5551 -> 3851 bytes

And my personal favorites:

MinesweeperCE (by Merth): 24765 -> 7144 bytes
This Is The Only Level CE (by JWinslow): 25960 -> 9785 bytes

Wow, this is looking very promising! I only wonder why some programs are like 40% compressed, but MinesweeperCE is 62%, is that because of the compression algorithm? Well, keep up the good work

Probably due to different code-to-data ratio (especially low-entropy data that is probably better compressible).
A somewhat far-fetched (but not so much, depending on what Mateo uses) example could be: "AAAAAAAA" is compressed into "8A" (for instance), but "0A42F8CD" stays the same.

Anyway, very nice work I can see this as an option to the PB (well, a (default?) flag in the toolchain's Makefile)

This is a great idea! I just have two questions:
1. will this program be integrated into Cesium and/or Doors?
2. If you decide to compress Basic programs, How will the editor work? Will the program automatically get uncompressed?

He sais the routine is implemented into the program itself, thus doesn't need a shell; however, perhaps a shell could have an option to try and compress programs for you?

Is this LZ77, DEFLATE, or just plain RLE?

zip -9 is barely better, so it's pretty good.
Lionel suggested PuCrunch at some point, which has similar compression rates too, so maybe Mateo ported that

Basic programs will be transformed into asm programs and run like them basically. They won't be able to be edited; it would be more for the final release of programs.

It's the zx7 compression algorithm that I stumbled across when looking at z80 compilers. It is a part of the z88dk development kit; a really awesome array of tools that I'm surprised no one here has really taken advantage of in the past.

Yeah, I was thinking a flag would be nice to add; although it would need to be added to convhex instead. That's a rather simple task; this is more of a standalone program to convert already assembled programs.

Wow. This is amazing, Mateo!.

Yep, ConvHex would probably be the right place for that.
Also, just in case you hit a badly compressible (if at all) file (high entropy everywhere), you'd have to check if the output is actually smaller than the input, and if not, well, keep the uncompressed version

Very nice work !

It could be cool if you can compress basic programs

Wow, that really decreases the size! Do you compress the program on the computer before uploading it to the calculator?

If a basic program was transformed into an asm program by the compressor, would the programs run as fast as asm?

But, wouldn't the decompressed version still be copied to RAM, thereby taking up the size of the compressed version + decompressed version (assuming the compressed version was also in RAM)?

If so, I think this would have better use as a special archiving tool, when moving big stuff to and from flash.

Yes but this is what happens anyway. I don't really get your point.

Anywho, no basic programs won't be faster for the the simple fact they are basic programs.

Smaller size != Faster. this isnt changing a program's final execution code, just adding a step beforehand to decompress it.

This is great, Mateo! Reminds me of Lite86 back in the day

I am currently working on a Huffman encoder/decoder in Hybrid Basic for the 83+ and co. monochrome calcs, in case anyone is interested. But development is REAL slow.

I have released this version of the compression program. There are many different options and sources of input; it allows for .hex, .bin, and .8xp files as input. BASIC program compression simply isn't possible because BASIC programs have too much variance in the bytes.

Download: https://github.com/MateoConLechuga/ConvHex/releases/latest

To create a self extracting program for example:

Code:


I may make this an on-calc compressor as well; and I plan to make it a (default?) option in the C toolchain.

I'm having some difficulty running this application. I downloaded it and ran it but nothing happened. So I ran it again and a window popped up but immediately disappeared. Any idea what I'm doing wrong? (windows firewall blocked it at first because it was 'not a commonly downloaded program'. But I clicked 'run anyways' so I don't think the firewalls the problem)

Maybe you should read the readme
You should run it via the command line, not double click on it.

If you don't know how the command line works, open a new text file, write this:

Code:
, save it as "compress.bat" and double click this file. That should work as well

An example is my ICE program:

Code: