From users-return-7450-apmail-subversion-users-archive=subversion.apache.org@subversion.apache.org Wed Feb 02 09:10:21 2011 Return-Path: Delivered-To: apmail-subversion-users-archive@minotaur.apache.org Received: (qmail 56090 invoked from network); 2 Feb 2011 09:10:21 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (140.211.11.3) by minotaur.apache.org with SMTP; 2 Feb 2011 09:10:21 -0000 Received: (qmail 5892 invoked by uid 500); 2 Feb 2011 09:10:20 -0000 Delivered-To: apmail-subversion-users-archive@subversion.apache.org Received: (qmail 5600 invoked by uid 500); 2 Feb 2011 09:10:18 -0000 Mailing-List: contact users-help@subversion.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Delivered-To: mailing list users@subversion.apache.org Received: (qmail 5587 invoked by uid 99); 2 Feb 2011 09:10:17 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 02 Feb 2011 09:10:17 +0000 X-ASF-Spam-Status: No, hits=2.9 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_NONE,SPF_HELO_PASS,SPF_NEUTRAL X-Spam-Check-By: apache.org Received-SPF: neutral (athena.apache.org: local policy) Received: from [74.208.4.195] (HELO mout.perfora.net) (74.208.4.195) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 02 Feb 2011 09:10:10 +0000 Received: from [192.168.1.4] (pool-173-56-3-140.nycmny.fios.verizon.net [173.56.3.140]) by mrelay.perfora.net (node=mrus3) with ESMTP (Nemesis) id 0LyVMk-1QHDxu1qkA-015xG9; Wed, 02 Feb 2011 04:09:48 -0500 Subject: Re: Checkout really slow in Windows with lots of files in one directory From: Nick To: Mark Phippard Cc: Neil Bird , users@subversion.apache.org In-Reply-To: References: <4D402FA4.4020607@jibbyjobby.co.uk> Content-Type: multipart/alternative; boundary="=-g77LR0lJr9ujfsLOrGH9" Date: Wed, 02 Feb 2011 04:09:43 -0500 Message-ID: <1296637783.6917.192.camel@nimble.325Bayport> Mime-Version: 1.0 X-Mailer: Evolution 2.30.3 X-Provags-ID: V02:K0:Xx1g0bCLuEvSqNuyz55eUUaDyejebwMrIXuHN0s09al rhiyNmVhfwh+HcQ4iJSsnv1riYOVM04Fgd7u7VGM7wfVqwrcp8 ynkskIIRkny2BZ6L+zwbbaLxH/xf3sa752hBBlJb6Ci0cNl95l xdrc7NnBQGuIpftlPRjjclYQBWLYEeCfhb+7jB5eI/J6Pg5bUC PuVKduHasW+P8iCrS7FZGVvSW/EndlNThJacwsepPA= --=-g77LR0lJr9ujfsLOrGH9 Content-Type: text/plain; charset="ISO-8859-1" Content-Transfer-Encoding: 7bit On Tue, 2011-02-01 at 13:00 -0500, Mark Phippard wrote: > On Wed, Jan 26, 2011 at 9:28 AM, Neil Bird wrote: > > > We have a graphics-oriented code-base that's auto-generated and has >5000 > > source files in one directory. While I can check this out OK on Linux, > > we're seeing an unusable slow-down on Windows XP (NTFS), both using Tortoise > > directly, and as a test on Linux with the Windows drive mapped over CIFS. > > I created a folder with 5001 files in it ... maybe that is not enough? > I just used small simple text files as I was only checking for the > general problem in managing the temp files and the WC metadata. > > Upon checkout (using 1.6.15 command line client) I did not notice any > slowdown. Windows checked out via HTTP across internet in about 49 > seconds as opposed to 33 from my Mac (which is a faster system). The > main thing is checkout did not seem to slow down. I did a similar test, using 5100 files in a single directory. Each file contained only the content "file XXXX" where XXXX was the number of the file (so tiny files). My linux system took 17 seconds, while Windows took a bit less than 2 min (but Windows is virtualized while linux is on the hardware). I also did not notice a slow-down as the checkout proceeded. Both systems used 1.6.15 and accessed the repo via https. I did, however, notice that the time to *add* the files (done via svn add *.txt) seemed to progressively slow down. But this was only observed by watching the files in the console as they were being added (it was relatively easy to see the rate because the each file name had a linear number at the end). I don't have any timings to back this up, though I'll collect some if anyone's interested. Nick --=-g77LR0lJr9ujfsLOrGH9 Content-Type: text/html; charset="utf-8" Content-Transfer-Encoding: 7bit On Tue, 2011-02-01 at 13:00 -0500, Mark Phippard wrote: 
On Wed, Jan 26, 2011 at 9:28 AM, Neil Bird <neil@jibbyjobby.co.uk> wrote:

>  We have a graphics-oriented code-base that's auto-generated and has >5000
> source files in one directory.  While I can check this out OK on Linux,
> we're seeing an unusable slow-down on Windows XP (NTFS), both using Tortoise
> directly, and as a test on Linux with the Windows drive mapped over CIFS.

I created a folder with 5001 files in it ... maybe that is not enough?
 I just used small simple text files as I was only checking for the
general problem in managing the temp files and the WC metadata.

Upon checkout (using 1.6.15 command line client) I did not notice any
slowdown.  Windows checked out via HTTP across internet in about 49
seconds as opposed to 33 from my Mac (which is a faster system).  The
main thing is checkout did not seem to slow down.
I did a similar test, using 5100 files in a single directory.  Each file contained only the content "file XXXX" where XXXX was the number of the file (so tiny files).  My linux system took 17 seconds, while Windows took a bit less than 2 min (but Windows is virtualized while linux is on the hardware).  I also did not notice a slow-down as the checkout proceeded.  Both systems used 1.6.15 and accessed the repo via https.

I did, however, notice that the time to *add* the files (done via svn add *.txt) seemed to progressively slow down.  But this was only observed by watching the files in the console as they were being added (it was relatively easy to see the rate because the each file name had a linear number at the end).  I don't have any timings to back this up, though I'll collect some if anyone's interested.

Nick

--=-g77LR0lJr9ujfsLOrGH9--