httpd-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "William A. Rowe, Jr." <wr...@rowe-clan.net>
Subject New Win32 apr_stat(), apr_lstat() performance
Date Sun, 12 Nov 2000 18:15:31 GMT

For anyone interested (FirstBill, Tim, etc.)

I was very interested in the stat performance, since it is once of our
weakest links in server performance.  I hashed and dumped out a list of
2000 files, randomized by hash key, off of a 60,000 file NTFS volume.
Here are the results, I'm satisfied that this is where we will leave
things for a while (no group/user/perms from apr_stat()/apr_lstat() on
Win32, at least not by default.)  I'm spending my time in Apache itself
right now, and as I see requirements, I'll review the issue again.  This
simply proves group/user/perms are all too expensive to implement today.

Using the oldest apr_stat method (the only method available for Win95):

Stat'ed 2000, failed 0 requests
using the FindFirstFile (suffers wildcard expansion) method
                        630 ms (kernal)
  +                      80 ms (user)
  =                     711 ms total cpu time

Stepping up to the Win98 method (that still can't return device/inode):

Stat'ed 2000, failed 0 requests
using the GetFileAttributesEx GetFileExInfoStandard method
                        460 ms (kernal)
  +                      80 ms (user)
  =                     540 ms total cpu time

Then I started looking at the new CreateFile/GetFileInformationByHandle
and CloseHandle methods.  We use this method now for two very simple 
reasons.  We recover device and inode data that will be used for later
optimization, and we detect non-file objects including pipes and other
dangerous targets.  First, ignoring users, groups and permissions:

Stat'ed 2000, failed 0 requests
using the GetFileInformationByHandle BACKUP_SEMANTICS OPEN_NO_RECALL method
with additional query to GetFileType 
                        690 ms (kernal)
  +                      90 ms (user)
  =                     781 ms total cpu time

Gathering the owner user and group SID, but ignorning permissions:

Stat'ed 2000, failed 0 requests
using the GetFileInformationByHandle BACKUP_SEMANTICS OPEN_NO_RECALL method
with additional query to GetFileType and GetSecurityInfo
                       8852 ms (kernal)
  +                    7891 ms (user)
  =                   16744 ms total cpu time

Gathering the owner user and group SID, as well as the DACL (discressionary
permissions, rather than system permissions, or SACL, which we are unlikely
to have access to) for the owner, group and world:

Stat'ed 2000, failed 0 requests
using the GetFileInformationByHandle BACKUP_SEMANTICS OPEN_NO_RECALL method
with additional query to GetFileType and GetSecurityInfo
                      16523 ms (kernal)
  +                   15502 ms (user)
  =                   32026 ms total cpu time

Note all of these stats vary +/-15% per invocation.  This is a very hard
benchmark against a 2GB compressed NT 4.0 NTFS partition, with the list of
2000 files randomized from some 60,000 files on disk.  They aren't even
recently defragmented, although I did assure they reside in the cache.
Consider this a worst case scenario, since it's a 300mhz box using a 
single channel narrow scsi 4gb drive.  I've benchmarked to show the 
difference between methods, not absolute times.

The benchmark code is available by request, I'd post it up to
dev.apache.org/~wrowe if anyone is interested.


Mime
View raw message