Imagine a bacon-wrapped Ferrari. Still not better than our free technical reports.
See all our reports

Download the Internet, in Milliseconds!

UPDATE: if you’re looking for zt-zip, ZIP archive processing library with a modern and fluent API that works both on Java and Android and supports packing, unpacking, repacking, filesystem permissions, etc, please refer to its Github repository: zt-zip.


You’ve hopefully already heard of Optimizer for Eclipse, the first free tool from ZeroTurnaround, which speeds up your Eclipse installation by finding and fixing config issues in your code. A number of ideas were considered for the next free tool, including Optimizer for IntelliJ, a tool that slows down the IntelliJ IDEA giving the illusion that Eclipse is faster than it is, and Optimizer for Netbeans, an IDE removal tool.

However, none of these ideas made the cut. Today ZeroTurnaround is pleased to announce the second free tool in our portfolio — this one has more impact and is more disruptive to the market than the first.

ZipRebel

Introducing ZipRebel

Our new free tool, ZipRebel, released earlier today and available on github, is a file compression tool that is so powerful it can reduce the entire internet to less than a byte of data. I know this is a very powerful and bold claim but as you’ll see it’s backed up by fact and science.

A Brief History of Data

Let’s take it back to logic voltage levels, where different voltages on a wire represent different signals. 0 to 2 volts might represent logical 0, and 3 to 5 volts represent logical 1. Moving into the digital wave format, we find we have two values a wave can take; 0 or 1. A string of these values represents a number, for example in this image below we see a representation of the number 69.

DigitalSignal69

We clearly understand that 1 is a value, it’s data, it’s a payload, whereas 0 is nil, no data, a separator of actual data. This distinction is where our problem lies and our productivity suffers in general. Our productive work, in this case data (1), is split up by unproductive gaps (0), similar to our more familiar application server restart or application redeploy cycle we infamously fix with our JRebel tool.

A bit, or binary digit, in computer science can take the value 0 or 1. Doesn’t sound too crazy, but let me put it another way – irrespective of whether your bit contains data (1) or is empty (0) it takes up the same amount of space. That’s right, the internet is half filled will emptiness. That’s half full of nothing, taking over the intertubes, just like the neverending story.

Stand back we’re about to do science.

So how does ZipRebel work? How is it different? Let’s take a look at this single byte compression example.

01011001

Here we have 4 bits of data (1’s) and 4 spacers (0’s). Yeh, 4 bits of nothing, packing half of our byte with useless nothingness. ZipRebel removes the empty information, as follows:

1111

Check that out! We just halved our storage size from a byte to a nibble! CHOMP! Also we’re left with nothing but pure data. No waste, no empty bits.

And get this, we can go much further – our 4 bits of pure data is currently being viewed as a raw string of bits. We now know every bit is data, so why not count the number of bits and represent it as a binary number? We have 4 bits of data so we can say it’s equivalent to 100. 100 is binary for 4, so it equates to 1111 in pure data form.

OK, that’s another 25% we just knocked off our nibble to make a tri-bit (yeh, I just made that up, so what!). Now, you’re super bright, that’s why you’re a RebelLabs reader, I expect you see where we go from here, right? We’ve found ourselves in a position where we have empty bits again! Yeh, bits with no data! Can’t have that, we can eliminate this nothingness further (see, recursion really is a good thing) by eliminating the empty bits, leaving us with:

1

Yep, we’ve taken a byte of data, and have represented it as a single binary digit. ZipRebel provides you with a new file format extension, ZR. Oh, and you’re welcome to use this revolutionary tool for developers (patents pending) for free. I know what you’re thinking…

omg

…right?

Does it scale?

Hell yeah it scales! I’ll let my results do the talking. Here you can see numerous tests against a number of files, including the original byte we used previously, a 25 year JRebel and XRebel license for you to download and use. We also tested with the Jenkins war file and even the full blown internet, both of which we managed to squish into a single bit.

File description Initial size Final size
01011001 example (1 byte) example.ZR (1 bit)
25 year JRebel license jrebel.lic (975 bytes) jrebel.lic.ZR (1 bit)
25 year XRebel license xrebel.lic (784 bytes) xrebel.lic.ZR (1 bit)
Jenkins CI jenkins.war (53.9 MB) jenkins.war.ZR (1 bit)
JIRA atlassian-jira-6.4.tar.gz (227 MB) atlassian-jira-6.4.tar.gz.ZR (1 bit)
The Internet theInternet- (1 Zettabyte) theInternet.ZR (1 bit)

I can haz teh internetz?

If that’s what you want, yes you absolutely can download the internet! We’re not the first tool to attempt this of course. The Maven build tool attempts this on a regular basis, but lacks the large scale compression technique we bring to the table, hence the poor performance. We optimize this download and I’m pleased to say are in discussions with Maven to integrate this into future releases, optimizing their internet download stage.

What’s next?

Having spent a short time designing and developing ZipRebel, our attentions have now focused to our next tool, UnzipRebel. However it seems to be a little trickier than we first expected, but you’ll be glad to know we’d like to increase our investment in the tool and you can help!

UnzipRebel

We’re crowdsourcing our funding on Kickstarter, providing people who fund us with commercial software licenses to kick ass tooling in return, all in the ZipRebel of course, saving download time. We’re aiming for a 2016 release, perhaps around the April 1st timeframe, so between now and then our recommendation is to just zip it! Here’s some more detail about our Kickstarter campaign:

UnZipRebel-full-2-1

Here’s a testimonial, for one happy ZipRebel user, Mr Z. Ippit, who’s also an excited potential UnzipRebel user.

After using ZipRebel as a beta for some time now, I’ve managed to massively reduce my SDD usage. All my uber-important documents have been converted to ZipRebel format, and I’ve deleted all originals which saves me over 2GB of space! I can’t wait for UnzipRebel to be released, so I can retrieve my contracts, insurances, licenses and legal documents!

*N.B* It is not recommended you delete original files, even after UnzipRebel has been released. I’m a big believer in natural selection, so didn’t really want to add this disclaimer, but you know… ;)

  • Neeme Praks

    This is very cool! Any chance of packaging this as a file-system implementation that would apply the compression on-the-fly? ;-)

  • Oleg Šelajev

    This is a great idea, can you please submit a Github issue, so we can consider it among all other possible vectors of enhancing ZipRebel.

    Here’s the repository again: https://github.com/zeroturnaround/ziprebel

  • Faisal Feroz

    April Fool :P

  • Awesome prank ;)

  • Francesco Zanitti

    You missed the ultimate obvious optimization:

    instead of storing 1, you could store ~1 (since the ‘~’ is invertible).

    And ~1 = 0, and by applying the compression scheme again, we get the empty value, resulting in an empty file and unlimited compression rate.

  • Simon Maple

    I’m not sure you’re taking this seriously Francesco.

    By calling 1, ~1 we’re altering the data payload. That’s lossy compression, which isn’t our game.

  • Simon Maple

    Prank? :)

  • That’s what I said. I said “what joke?” http://redd.it/311cww

  • Francesco Zanitti

    I am deadly serious, since ~ is invertible. You can always recover the initial (ziprebel-compressed) data by calling ~(decode contents) where the decode function is defined as follow (assuming that “contents” is a string containing the contents of the compressed file, using my optimization):

    decode “” = 0

    So now you can supply the result of ~(decode contents) to unzip-rebel and recover the full data. And by the way I am really looking forward for it!

  • Simon Maple

    Francesco, my friend, you just earned your spot on the UnzipRebel development team. We need out the box thinkers just like you to make the impossible, possible.

  • Francesco Zanitti

    Thank you! I am sure Shannon would be proud of this breakthrough invention!