Return-Path: <new-httpd-owner-new-httpd-archive=hyperreal.org@apache.org>
Delivered-To: new-httpd-archive@hyperreal.org
Received: (qmail 4290 invoked by uid 6000); 10 Jan 1998 23:48:41 -0000
Received: (qmail 4283 invoked from network); 10 Jan 1998 23:48:40 -0000
Received: from valis.worldgate.com (marcs@198.161.84.2)
  by taz.hyperreal.org with SMTP; 10 Jan 1998 23:48:40 -0000
Received: from localhost (marcs@localhost)
	by valis.worldgate.com (8.8.7/8.8.7) with SMTP id QAA12944
	for <new-httpd@apache.org>; Sat, 10 Jan 1998 16:48:30 -0700 (MST)
Date: Sat, 10 Jan 1998 16:48:30 -0700 (MST)
From: Marc Slemko <marcs@worldgate.com>
To: Apache - BYOC <new-httpd@apache.org>
Subject: interesting bug with zb and Nagle and FreeBSD and lo0
Message-ID: <Pine.BSF.3.95q.980110164110.1383N-100000@valis.worldgate.com>
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=US-ASCII
Sender: new-httpd-owner@apache.org
Precedence: bulk
Reply-To: new-httpd@apache.org

---
Server:                 Apache/1.3b4-dev
Document Length:        1316
Concurency Level:       1
Time taken for tests:   10.060 seconds
Complete requests:      51
Failed requests:        0
Keep-Alive requests:    51
Bytes transfered:       81448
HTML transfered:        67116
Requests per seconds:   5.07
Transfer rate:          8.10 kb/s

Connnection Times (ms)
           min   avg   max
Connect:     0     0     0
Total:      56   197   201
---

We kick ass!  5 requests per sec using keepalives on a p166 getting a ~1k
document. 

Quite interesting, since getting / instead of /index.html gets 2 order of
magnitude better rates. 

Looking further:

# request
16:24:45.303916 localhost.worldgate.com.3310 > localhost.worldgate.com.http: P 437:537(100) ack 6390 win 57344 <nop,nop,timestamp 135124 135124,nop,nop,cc 92000> (DF)
# ack (200ms!)
16:24:45.502068 localhost.worldgate.com.http > localhost.worldgate.com.3310: . ack 537 win 57344 <nop,nop,timestamp 135124 135124,nop,nop,cc 92001> (DF)
# request
16:24:45.502122 localhost.worldgate.com.3310 > localhost.worldgate.com.http: P 537:546(9) ack 6390 win 57344 <nop,nop,timestamp 135124 135124,nop,nop,cc 92000> (DF)

That makes it more obvious.  We have delayed ack acting on the server
delaying the ack for 200ms, and Nagle working on the client preventing
it from sending the end of the request until it gets the ack.  This gives
almost exactly 5/sec.

The fix is to change zb to disable Nagle, but I am confused about WTF
FreeBSD is generating two packets.  Heck the mtu of lo0 is 16k.
The same thing happens if I send across the network to another machine
from this one.  It comes from a single write() and 109 bytes isn't
that much.  It does this 100% of the time.