lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael Barry <mba...@cos.com>
Subject Indexing Tips and Hints
Date Mon, 24 Feb 2003 19:00:32 GMT
All,
   I'm in need of some pointers, hints or tips on indexing large collections
of data. I know I saw some tips on this list before but when I tried 
searching
the list, I came up blank.
   I have a large collection of XML files (336000 files around 5K 
apiece) that I'm
indexing and its taking quite a bit of time (27 hours). I've played 
around with the
mergeFactor, RAMDirectories and multiple threads (X number of threads 
indexing
a subset of the data and then merging the indexes at the end) but I 
cannot seem
to bring the time down. I'm probably not doing these things properly but 
from
what I read I believe I am.  Maybe this is the best I can do with this 
data but I
would be really grateful to hear how others have tackled this same issue.
   As always pointers to places in the mailing list archive or other 
places would be
appreciated.

Thanks, Mike.

---------------------------------------------------------------------
To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: lucene-user-help@jakarta.apache.org


Mime
View raw message