jackrabbit-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Brian Topping <topp...@codehaus.org>
Subject Problem importing more than 5K files over WebDAV
Date Tue, 14 Apr 2009 18:44:23 GMT
Greetings all,

I'm working to get my first Jackrabbit repository deployed and am  
having some pretty serious problems loading it via WebDAV.  It's  
generally working pretty well, but due to some brain damaged legacy, I  
have a directory with about 42K objects in the top-level directory and  
about 80K objects in the rest of the more evenly loaded tree.

The problem that I am seeing is that after importing a few thousand  
files, the speed that the import proceeds starts to slow  
exponentially.  Log messages start being generated from LRUNodeIdCache  
and BundleCache more rapidly, and intake that was running at about  
twenty files per second (and maybe one log message per 500 files)  
start slowing to one file every few seconds and the above log messages  
repeating multiple times per file.  I generally don't wait much longer  
than this, there are 73,000+ files left to go!

In attempts to resolve this, I've tried both the  
DerbyPersistenceManager and BundleFsPersistenceManager to no avail.   
Performance seems to improve if the SearchIndex is left unconfigured,  
but only to the extent that maybe 10% more items are imported before  
the performance falls off.

What I'd like to find from the experienced users here it is practical  
under any circumstances to have this many files in a directory, and if  
so, what I might tune to help the import along.

Kind regards,

Brian

Mime
View raw message