httpd-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
Subject Re: [PATCH] mod_deflate extensions
Date Thu, 21 Nov 2002 14:36:30 GMT

>> Henri Gomez wrote...
>> - Put part of zlib code in Apache 2.0 source ?
> Jeff Trawick wrote...
> that is what I suspect to be the safest, easiest-to-understand way...
> the build would work like on Windows, where the project file for
> mod_deflate pulls in the right parts when building
> Henri Gomez also wrote...
> We could grab the compression part, like does mod_gzip, mod_gzip even 
> use zlib includes.

If the discussion is once again open about including compression
code in a base Apache source release then let me clarify what
Henri just said...

The compression engine in the orginal mod_gzip ( and the
current one in the 1.3 series mod_gzip ) is NOT based
on ZLIB at all. It is based on the ORIGINAL public 
domain GZIP/LZ77 code that pre-dates ZLIB. That LZ77 code
is, in many ways, much 'simpler' than ZLIB since ZLIB
became this kind of 'all things to all people' package and the
I/O and API interface(s) got a lot more complicated than
the original GZIP stuff. ZLIB was meant to be 'easier
to deal with' at the API level but the sacrafice was
more overhead during the compression itself.

The original GZIP code was not thread-safe so the engine
in the original mod_gzip was a thread-safe rewrite that
uses a Finite State Processor ( like SSL codebases ).

Every compression task has a 'context handle' just like
SSL does to maintain the integrity of all the compression
tasks that might be running at the same time. There 
are no 'globals' as there are in GZIP ( and ZLIB? ).

It also got 'tweaked' to do straight pointer based
in-memory compressions which made it a better candidate
for a 'real-time' compression engine. A lot of the
time-consuming things in GZIP were also speeded up with
better loops and code that produced better .ASM output.

So... to make a long story short... mod_gzip actually
includes some original GZIP 'headers' and uses the
same functions names ( deflate/inflate, etc. ) as
legacy GZIP and ZLIB... but those headers were not
from ZLIB.

They didn't change the headers much when they created
ZLIB from GZIP but there are some subtle differences.

NOTE: The original sumbmission of mod_gzip for Apache
2.0 used the same built-in compression engine as the
1.3 series. It was already thread-safe so it would
have been fine for 2.0. There was a LATER re-submission
of mod_gzip for Apache 2.0 that followed an intense
discussion about NOT using such code and it is more
like mod_deflate and does, in fact, just include
ZLIB headers and has the same build expectations
as mod_deflate. BOTH VERSIONS are still sitting there
somewhere in the Apache archives. They were both
submitted some long time ( years now? ) ago.

I also refer you to the discussion thread regarding the
original inclusion of mod_deflate which contains some
'advice' posted to the Apache forum from Dr. Mark Adler
( one of the original authors of all this GZIP/ZLIB LZ77
code ).

He suggested that compiling your OWN version of GZIP/ZLIB
was pretty much the only 'sane' thing to do and I agree
100%. There are actually a number of 'flaky' distributions
of GZIP/ZLIB 'out there'.

He ( Mark Adler ) himself pointed out that there are 
some 'patches' floating around for GZIP/ZLIB that never 
made it into ANY standard distribution and only by applying 
them yourself to your own compiled OBJ/LIB could you be
sure what you are actually using and what shape it is in.

Now that Apache is multi-thread capable and there are some
known issues with legacy GZIP/ZLIB distributions and 
multi-threading it makes even more sense to know EXACTLY 
what 'version' of whatever compression code you are dealing 
with when the threading bug reports start coming in.

> Jeff Trawick also wrote...
> The reason I'm being a real stick-in-the-mud about this is because
> Apache users already seem to have enough trouble getting Apache to
> build out of the box, usually because of some screwy setup or other
> unexpected conditions that nobody can ever reproduce, and it is very
> difficult for me to accept that some quick change to turn on
> mod_deflate is going to do anything other than increase these
> problems, particularly since I committed various "user errors" myself
> getting consistent builds of mod_deflate and zlib.  (Heck, I even
> broke the cvs executable on one machine before I found out what was
> going on :) )

I think you underestimate your legacy user base.

Everyone already accepts the fact that building Apache
is usually an absolute nightmare, especially if you
are trying to get any kind of SSL going. 

It hasn't slowed anyone down that I can tell.

They ( historically ) just stay up all night and figure it out.
That's how you build Apache.

If they REALLY WANT some 'feature' in their Server... they
will tough it out and get it done. That's the way it's 
always been with Apache so no one would be shocked if some LIB 
wasn't in the right place or a makefile needed tweaking.

Kevin Kiley

View raw message