Date: Sun, 9 Mar 1997 23:57:04 -0700 (MST)
From: Marc Slemko <marcs@worldgate.com>
To: new-httpd@hyperreal.com
Subject: Re: 30 second response time from apache and zeus servers (fwd)
In-Reply-To: <Pine.BSI.3.95.970309223939.10327A-100000@taz.hyperreal.com>
Message-ID: <Pine.BSF.3.95q.970309234245.21863A-100000@valis.worldgate.com>
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=US-ASCII
Content-Transfer-Encoding: QUOTED-PRINTABLE
Sender: new-httpd-owner@apache.org
Precedence: bulk
Reply-To: new-httpd@hyperreal.com

The latest BIND does do negative caching:

  NCACHE (origin: USC/ISI)
=09  enables negative caching. We cache only authoritative NXDOMAIN or
  authoritative NOERROR with zero RR count. Non-authoritative NXDOMAIN answ=
ers
  now contain NS records in the authority section. Non-authoritative NOERRO=
R
  responses have no authority or additional records to differentiate them f=
rom
  referrals. They are cached for NTTL secs (currently 10 minutes) and are t=
imed
  out when the ttl expires.
=09  you probably want this, it is on by default.

Shorter ttl than average, but it really has to be that way.  The
archaic 4.9.3-P1 on taz almost certainly doesn't have negative
caching.

It should not take 30 seconds to respond to an unresolved hostname.
In some cases where the server is not answering, yes, but in this
case the server currently responds with a nxdomain right away so there
should be no delay unless something is broken somewhere.

It takes a lot more effort than most people think to implement
an internal resolver with cache (Netscape sure can't do it right...)
and it makes sense to use named.  In any case, any sort of caching,
including both in memory document and hostname caching, isn't pretty
in Apache right now because you need some sort of IPC to share
between processes.  Tell him that this has nothing to do with Apache
as a server but simply has to do with the decision of the admin of
that particular website to punish those who can't setup proper
reverse DNS entries.  <g>  Oh yea, and tell him it is lame to use
HTML in mail for no reason.

I think Roxen has a seperate process that handles DNS lookups.

On Sun, 9 Mar 1997, Brian Behlendorf wrote:

>=20
> Nice.  So, should I turn off reverse DNS, recommend to him that he
> properly configure reverse DNS, somehow tune my named to cache failures
> (how?) or what?  Kinda cool that he has to pick something relatively puny
> to critique.  :)
>=20
> =09Brian
>=20
> --=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D=
-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=
=3D--
> brian@hyperreal.com     http://www.apache.org     http://www.organic.com/=
jobs
>=20
> ---------- Forwarded message ----------
> Date: Sun, 09 Mar 1997 22:06:10 -0800
> From: Steve Kirsch <stk@infoseek.com>
> To: brian@apache.org, damian@zeus.co.uk
> Cc: cjl@infoseek.com, wchang@infoseek.com, jackiew@infoseek.com,
>     adf@infoseek.com, cshih@infoseek.com
> Subject: 30 second response time from apache and zeus servers
>=20
> I wanted to alert you guys that both the www.apache.org and
> www.zeus.co.uk
>=20
> have 30 second response times when I tried access from my home machine
> (which
>=20
> uses a dynamically assigned IP address).
>=20
>=20
> Apparently, I suspect you are trying to do a reverse name lookup on
> every
>=20
> HTTP request, and are not responding to the request until you have a
> response from
>=20
> your DNS. And if the response is unknown, you are continuing to look up
> the same
>=20
> address each time, rather than cache it as being unknown.
>=20
>=20
> As a result, people like me accessing from home get abysmal response time
> from your
>=20
> servers. As the log below shows, the TCP packets are getting through just
> fine, but
>=20
> there is a 30 second lag time until the server sends a response.
>=20
>=20
> So I do find it quite ironic that the supposedly "fastest" servers on the
> planet,
>=20
> are, in certain cases like mine (which are not all that unique), the
> slowest servers
>=20
> on the planet. =20
>=20
>=20
> Reverse name lookup is more effectively done by a log analysis tool, than
> on-the-fly.
>=20
>=20
> It's possible that there are other causes, so I welcome your response.
>=20
>=20
> Here's the snoop trace for apache. The one for zeus is similar:
>=20
>=20
> <bigger>snoop -x48,100 -tr -d le0 between 198.5.208.92 taz.apache.org
>=20
> Using device /dev/le (promiscuous mode)
>=20
>   0.00000 198.5.208.92 -> taz.apache.org TCP D=3D80 S=3D1128 Syn Seq=3D34=
05100
> Len=3D0 W
>=20
> in=3D8192
>=20
> =20
>=20
>            0: 2000 8aed 0000 0204 05b4 0261               ..........a
>=20
> =20
>=20
>   0.00597 taz.apache.org -> 198.5.208.92 TCP D=3D1128 S=3D80 Syn Ack=3D34=
05101
> Seq=3D103
>=20
> 1158339 Len=3D0 Win=3D8576
>=20
> =20
>=20
>            0: 2180 153f 0000 0204 0218 1818              !..?........
>=20
> =20
>=20
>   0.15405 198.5.208.92 -> taz.apache.org TCP D=3D80 S=3D1128   =20
> Ack=3D1031158340 Seq=3D
>=20
> 3405101 Len=3D0 Win=3D8576
>=20
> =20
>=20
>            0: 2180 2960 0000 3f88 6f50 1022              !.)`..?.oP."
>=20
> =20
>=20
>   0.15723 198.5.208.92 -> taz.apache.org TCP D=3D80 S=3D1128   =20
> Ack=3D1031158340 Seq=3D
>=20
> 3405101 Len=3D0 Win=3D32768
>=20
> =20
>=20
>            0: 8000 cadf 0000 3f88 6f50 1022              ......?.oP."
>=20
> =20
>=20
>   0.30480 198.5.208.92 -> taz.apache.org TCP D=3D80 S=3D1128   =20
> Ack=3D1031158340 Seq=3D
>=20
> 3405101 Len=3D468 Win=3D32768
>=20
> =20
>=20
>            0: 8000 9dd0 0000 4745 5420 2f64 6f63 732f    ......GET
> /docs/
>=20
>           16: 6d6f 642f 6d6f 645f 6173 6973 2e68 746d  =20
> mod/mod_asis.htm
>=20
>           32: 6c20 4854 5450 2f31 2e30 0d0a 4163 6365    l
> HTTP/1.0..Acce
>=20
>           48: 7074 3a20 696d 6167 652f 6769 662c 2069    pt: image/gif,
> i
>=20
>           64: 6d61 6765 2f78 2d78 6269 746d 6170 2c20   =20
> mage/x-xbitmap,
>=20
>           80: 696d 6167 652f 6a70 6567 2c20 696d 6167    image/jpeg,
> imag
>=20
>           96: 652f 706a                                  e/pj
>=20
> =20
>=20
>   0.34918 taz.apache.org -> 198.5.208.92 TCP D=3D1128 S=3D80     Ack=3D34=
05569
> Seq=3D103
>=20
> 1158340 Len=3D0 Win=3D8576
>=20
> =20
>=20
>            0: 2180 278c 0000 0000 0000 0000              !.'.........
>=20
> =20
>=20
>  27.62119 taz.apache.org -> 198.5.208.92 TCP D=3D1128 S=3D80     Ack=3D34=
05569
> Seq=3D103
>=20
> 1158340 Len=3D512 Win=3D8576
>=20
> =20
>=20
>            0: 2180 0770 0000 4854 5450 2f31 2e31 2032    !..p..HTTP/1.1
> 2
>=20
>           16: 3030 204f 4b0d 0a44 6174 653a 2053 756e    00 OK..Date:
> Sun
>=20
>           32: 2c20 3039 204d 6172 2031 3939 3720 3138    , 09 Mar 1997
> 18
>=20
>           48: 3a30 353a 3330 2047 4d54 0d0a 5365 7276    :05:30
> GMT..Serv
>=20
>           64: 6572 3a20 4170 6163 6865 2f31 2e32 6238    er:
> Apache/1.2b8
>=20
>           80: 2d64 6576 0d0a 436f 6e74 656e 742d 5479  =20
> -dev..Content-Ty
>=20
>           96: 7065 3a20                                  pe:
>=20
> =20
>=20
>  27.62389 taz.apache.org -> 198.5.208.92 TCP D=3D1128 S=3D80     Ack=3D34=
05569
> Seq=3D103
>=20
> 1158852 Len=3D512 Win=3D8576
>=20
> =20
>=20
>            0: 2180 1a02 0000 6520 7072 6f63 6573 7365    !.....e
> processe
>=20
>           16: 6420 6279 0a74 6869 7320 6d6f 6475 6c65    d by.this
> module
>=20
>           32: 2e0a 3c21 2d2d 2570 6c61 696e 7465 7874  =20
> ..<<!--%plaintext
>=20
>           48: 2026 6c74 3b3f 494e 4445 5820 7b5c 7474     &lt;?INDEX
> {\tt
>=20
>           64: 2068 7474 7064 2f73 656e 642d 6173 2d69   =20
> httpd/send-as-i
>=20
>           80: 737d 206d 696d 6520 7479 7065 2667 743b    s} mime
> type&gt;
>=20
>           96: 202d 2d3e                                   -->
>=20
> =20
>=20
>  28.10669 198.5.208.92 -> taz.apache.org TCP D=3D80 S=3D1128   =20
> Ack=3D1031159364 Seq=3D
>=20
> 3405569 Len=3D0 Win=3D32768
>=20
> =20
>=20
>            0: 8000 c50b 0000 43e5 7c50 1022              ......C.|P."
>=20
> =20
>=20
>  28.11847 taz.apache.org -> 198.5.208.92 TCP D=3D1128 S=3D80     Ack=3D34=
05569
> Seq=3D103
>=20
> 1159364 Len=3D512 Win=3D8576
>=20
> =20
>=20
>            0: 2180 65cb 0000 7470 642f 7365 6e64 2d61  =20
> !.e...tpd/send-a
>=20
>           16: 732d 6973 2061 7369 733c 2f63 6f64 653e    s-is
> asis<</code>
>=20
>           32: 3c2f 626c 6f63 6b71 756f 7465 3e0a 7468  =20
> <</blockquote>.th
>=20
>           48: 6973 2064 6566 696e 6573 2074 6865 203c    is defines the
> <<
>=20
>           64: 636f 6465 3e2e 6173 6973 3c2f 636f 6465  =20
> code>.asis<</code
>=20
>           80: 3e20 6669 6c65 2065 7874 656e 7369 6f6e    > file
> extension
>=20
>           96: 2061 7320                                   as
>=20
> =20
>=20
>  28.12133 taz.apache.org -> 198.5.208.92 TCP D=3D1128 S=3D80     Ack=3D34=
05569
> Seq=3D103
>=20
> 1159876 Len=3D512 Win=3D8576
>=20
> =20
>=20
>            0: 2180 86e5 0000 2061 7265 2073 656e 7420    !..... are=20
> sent
>=20
>           16: 3c65 6d3e 6173 2069 733c 2f65 6d3e 2073    <<em>as is<</em>
> s
>=20
>           32: 6f20 6173 2074 6f0a 7465 6c6c 2074 6865    o as to.tell
> the
>=20
>           48: 2063 6c69 656e 7420 7468 6174 2061 2066     client that a
> f
>=20
>           64: 696c 6520 6861 7320 7265 6469 7265 6374    ile has
> redirect
>=20
>           80: 6564 2e0a 3c62 6c6f 636b 7175 6f74 653e  =20
> ed..<<blockquote>
>=20
>           96: 3c63 6f64                                  <<cod
>=20
> =20
>=20
>  28.12392 taz.apache.org -> 198.5.208.92 TCP D=3D1128 S=3D80 Fin Ack=3D34=
05569
> Seq=3D103
>=20
> 1160388 Len=3D412 Win=3D8576
>=20
> =20
>=20
>            0: 2180 1489 0000 4f44 5926 6774 3b20 3c62    !.....ODY&gt;
> <<b
>=20
>           16: 723e 0a26 6c74 3b2f 4854 4d4c 2667 743b  =20
> r>.&lt;/HTML&gt;
>=20
>           32: 0a3c 2f63 6f64 653e 3c2f 626c 6f63 6b71  =20
> .<</code><</blockq
>=20
>           48: 756f 7465 3e0a 4e6f 7465 733a 2074 6865    uote>.Notes:
> the
>=20
>           64: 2073 6572 7665 7220 616c 7761 7973 2061     server always
> a
>=20
>           80: 6464 7320 6120 4461 7465 3a20 616e 6420    dds a Date:=20
> and
>=20
>           96: 5365 7276                                  Serv
>=20
> =20
>=20
>  28.49997 198.5.208.92 -> taz.apache.org TCP D=3D80 S=3D1128   =20
> Ack=3D1031160388 Seq=3D
>=20
> 3405569 Len=3D0 Win=3D32768
>=20
> =20
>=20
>            0: 8000 c10b 0000 f882 ef50 1080              ......=F8..P..
>=20
> =20
>=20
>  28.58074 198.5.208.92 -> taz.apache.org TCP D=3D80 S=3D1128   =20
> Ack=3D1031160801 Seq=3D
>=20
> 3405569 Len=3D0 Win=3D32356
>=20
> =20
>=20
>            0: 7e64 c10a 0000 f882 ef50 1080              ~d....=F8..P..
>=20
> =20
>=20
>  28.75340 198.5.208.92 -> taz.apache.org TCP D=3D80 S=3D1128 Rst Seq=3D34=
05569
> Len=3D0 W
>=20
> in=3D0
>=20
> =20
>=20
>            0: 0000 3f7b 0000 43fc 4c50 1022              ..?{..C=FCLP."
>=20
> =20
>=20
> </bigger>
>=20