httpd-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From toki...@aol.com
Subject Re: canned deflate conf in manual -- time to drop the NS4/vary?
Date Fri, 04 Jun 2010 08:51:56 GMT

> Mark Nottingham wrote...
>
> On 02/06/2010, at 9:00 AM, tokiley@aol.com wrote:
>
> > > Sergey wrote...
> > > That's new to me that browsers don't cache stuff that has Vary only on 
> > > Accept-Encoding - can you post some statistics or describe the test you ran?
> > 
> > Test results and statistics...
> > 
> > Apache DEV forum...
> > http://www.pubbs.net/200908/httpd/55434-modcache-moddeflate-and-vary-user-agent.html
>
> I don't see anything there but anecdotal evidence

I think you need to do a reboot on your definition of 'anecdotal'.

The thread above was a focused discussion about what ACTUALLY
happens if you try to 'Vary:' on 'User-Agent' in the real world
these days accompanied by some additional (relevant) information about
what COULD (actually) happen if you (alternatively) try to 'Vary:' on
'Accept-encoding:'. If you still think any of it 'lacks veracity'
and is 'not trustworthy' then my only suggestion would be to spend
a little time on Google or Bling. It's an ongoing 'story'.

> certainly no reproducible tests.

What sort of tests would you like to see?

Anyone with access to certain browsers can 'reproduce'
the reported results.

>> apache-modgzip forum...
>> http://marc.info/?l=apache-modgzip&m=103958533520502&w=2
>
> Seven and a half years old, 

Yea. Wow. Boggles the mind that it's still relevant, doesn't it?

> and again anecdotal.

See above regarding use/misuse of the word 'anecdotal'. 

The tests (described in the link) were done using a kernel debugger
and a lot of those (unpatched) browsers are still in use TODAY.
I've heard kernel debuggers called a lot of things but 'anecdotal'
is not on the list.

>> Etc, etc. Lots of discussion about this has taken place over
>> on the SQUID forums as well.
>
> Yes; most of it in the past few years surrounding the ETag bug in Apache, not 
> browser bugs. 
>
> Regards,
> Mark Nottingham

The 2.5 release of SQUID ( Early 2004 ) was the very FIRST version of that 
Proxy Server that made any attempt to handle 'Vary:' headers at all. Prior to
that, they were just doing the same thing all the browsers would. If a 'Vary:'
header of ANY description arrived in the stream, it was simply treated as if
it was 'Vary: *' ( STAR ) and there was no attempt to cache it at all.

There was a huge discussion about ALL of this in late December of 2003 
over in SQUID land as they were trying to get 2.5 out the door.

I believe, at that time, it was Robert Collins who got 'tagged' to do the 
'Vary:' part and Henrik Nordstrom took the whole 'ETag' part on his shoulders.

If you Google 'Vary Accept-Encoding Browsers SQUID' but also include
Robert Collins name you'll find more than if you use 'Henrik's' name
since he was ultra-focused on the ETag thing. ( He still is ).

Regardless, they were both VERY MUCH interested in the 'Browser bugs' 
surrounding all of this since they both realized that SQUID was going to 
'take the heat' if/when the whole 'Vary:' scheme came alive and things suddenly 
got weird 'on the last mile'.

In the end, they did a good job implementing 'Vary:' in SQUID 2.5 but it
really has been an ongoing 'adventure' that continues to this very day.

Only about 12 months ago one of the SQUID User's forum lit up with another
'discovered' problem surrounding all this 'Vary:' stuff and this had
to do with non-compliance on the actual 'Accept-Encoding:' fields
themselves coming from Browsers/User-Agents. ( Browser BUGS ).
In some cases the newly discovered problem reflects the same nightmare 
seen TODAY with the out-of-control use of 'User-Agent'. 

Too many variants being generated.

Squid User's Forum...
http://www.pubbs.net/200904/squid/57482-re-squid-users-strange-problem-regarding-accept-encoding-and-compression-regex-anyone.html

Here's just a sampling of what was being shown from REAL WORLD
Server logs just 12 months ago... 

Accept-Encoding: FFFF, FFFFFFF
Accept-Encoding: mzip, meflate
Accept-Encoding: identity, deflate, gzip
Accept-Encoding: gzip;q=3D1.0, deflate;q=3D0.8, chunked;q=3D0.6,
identity;q=3D0.4, *;q=3D0
Accept-Encoding: gzip, deflate, x-gzip, identity; q=3D0.9
Accept-Encoding: gzip,deflate,bzip2
Accept-Encoding: nnnnndeflate
Accept-Encoding: x-gzip, gzip
Accept-Encoding: gzip,identity
Accept-Encoding: gzip, deflate, compress;q=3D0.9
Accept-Encoding: gzip,deflate,X.509
Yada, yada, yada...

To this day... not even Firefox and MSIE 7 'do the same thing'
with regards to this header. Though SEMANTICALLY identical... the 
following is STILL causing some problems for people that weren't
tickety-boo with their parsing code...

Firefox sends this...
Accept-Encoding: gzip,deflate

MSIE sends this...
Accept-Encoding: gzip, deflate

That one even bit the SQUID folks in the butt for a couple of revisions and
they are STILL trying to arrive at the best 'normalization' parsing for 
this sort of thing.

Here's a thread from less than 60 days ago detailing this 'How do we normalize
this Accept-Encoding stuff?' issue with SQUID...
http://www.techrawr.com/2010/04/10/normalize-accept-encoding-headers-haproxy-squid-cache/

This isn't 'ancient history'.

All these issues are still being 'worked out' even though
they've been lying on the table for over a DECADE...

...which is why my 'advice' on the whole 'fast by default' thing was
to slow-down and take a deep breath before 'enabling' mod_deflate
'out of the box'. People need to do some homework before introducing
all this into their sub-nets and in-house proxies if they haven't already 
and be prepared for some ongoing 'gotchas'.

People get REALLY PISSED these days when everything was running along
just fine and suddenly there are 'problems'. Heads can roll.

Everyone tends to THINK this stuff has all been worked out already but
nothing could be further from the truth. Proceed with caution.

Yours
Kevin Kiley



Mime
View raw message