lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ivan Provalov <iprov...@yahoo.com>
Subject Lucene Partition Size
Date Thu, 08 Apr 2010 18:05:07 GMT
We are using Lucene for searching of 200+ mln documents (periodical publications).  Is there
any limitation on the size of the Lucene index (file size, number of docs, etc...)?  

We are partitioning the indexes at about 10 mln documents per partition (each partition is
on a separate box, some mounted volumes are shared among three-four partitions).  We have
index size around 10% of the content size.  The total content is around 4Tb and the index
around 400Gb.  This content is planned to be split into 20 partitions (10mln docs, 200Gb content
size, 20Gb index size).  We are using a memory mapped index directory implementation.  Our
testing is done with 600 concurrent users.  

We are seeing consistently high response times from the partitions (4-5 seconds).  Is there
a number of documents per partition limitation in Lucene for this particular scenario?

Thanks,

Ivan


      

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message