Compression leading to a speed increase?

General Questions
#1

Ok, something occurs to me.

These days, the CPUs we have are a gazillion times faster than the mechanical parts (hard drives) we have in our computers. Anything having to do with loading data from the hard drive, the actual hard drive mechanics are the bottleneck in 99.9% of situations.

Coming up with numbers out of my ass, let’s suppose it takes 1 second to read a 1 megabyte file from the disk. Let’s say that we can compress that same file by half, and it takes .2 seconds to decompress (or shorter, relatively, with a fast CPU). These numbers seem reasonably proportional to me, but correct me if I’m wrong.

Now, the slowest part of the system, the mechanics of the hard drive, only have to read 500k of data - in half a second. Add decompression overhead on top of that, .2 seconds, and you have .7 seconds total, rather than the 1 second of the original file.

So it strikes me that now that our processors are way, way, way faster than our hard drives, the decompression process can work faster than the reading process - meaning compression would result in both a space and time efficiency saving.

But I’ve never read about compression ever being used as a performance thing, so I assume it’s not practical. Why not?

#2

It does occur on modern graphics cards. Compressed textures read off of the HD, up the AGP and into the GPU, which uses its hardware to decompress the texture, and then renders it and send it too your monitor. I have a feeling that only the compression of an agreed upon data packet at an greed upon rate is fessible in this day and age. You CPU sees too many different types of data to bother setting up a compression rountine for any one of them, while your graphics card deals with textures all of the time under the agreed upon standards of DirectX.

#3

Most files that are large enough to be limited by drive speed are already compressed to some degree. Basically any picure, audio or video file is compressed significantly, as are the large data files used by lots of programs, especially games. You can compress them further with a compression program, but the gains are pretty minimal in most cases. If we could compress the data to arbitrary sizes with proportional performance losses in decompression I’m sure we would use it, but you just can’t compress most of these files much more and most other files are too small to make much difference.

#4

Oddly enough, I thought this was going to be a question about this, when in fact I can use it to answer your question :slight_smile:

Basically, the answer is yes, you can get a performance increase with compression (at least with graphics), and this applies even more so to graphics over a network (which is even more of a bottleneck than drive access).

#5

That article you just pasted seems to make no sense. Graphics compression has nothing to do with network data transfer necesary for multiplayer games. It’s not like they’re transmitting graphics over the line.

#6

Not necessarily in real time as you play. But graphics would be networked either for one-time downloading to install locally, or possibly each time you start up the game.

And maybe even in real time. As I understand the concept behind Java, it feeds you just as much of the software as you need to complete a commanded task. There’s no reason the graphical background can’t be used the same way, except of course for transfer speed.

But this us just a WAG. IANA programmer.

#7

It depends on your specific system, but in general, yes, you are correct. Compressed drives and working with compressed files can be significantly faster, even with the added overhead of compression/decompression.

It’s not recommended as a general thing to improve performance because there are so many variables involved with the particular machine and usage that it’s impossible to say that everyone would get a speed increase. Some people are running their machines pretty hard all the time, and have a lot of CPU usage. Others who mainly use theirs for Office Apps barely use more than 5% of their CPU power on average, and would benefit.

However, a lot of machines are coming out now with 1 GB or more of RAM, and even with the lumbering hideous behemoth that is XP, they typically can end up with most things in memory, and little disk thrashing.

#8

I used to teach “File Processing” many, many times. The point in the OP was made in the texts and I routinely discussed it in my lectures. I also mentioned other issues such as greatZebu’s point: compressing an already compressed file is a loss. This is another point against using whole disk compression methods.

The advantage of decompression over file read speed has been in general true for Intel CPUs since the 386 days. Stacker used to sell hardware (de)compression cards to speed things up but they went away after the 286 generation.