lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From sweety <sweetyshind...@yahoo.com>
Subject to reduce indexing time
Date Wed, 05 Mar 2014 14:47:56 GMT
Before indexing , this was the memory layout,

System Memory : 63.2% ,2.21 gb
JVM Memory : 8.3% , 81.60mb of 981.38mb

I have indexed 700 documents of total size 12MB.
Following are the results i get : 
Qtime: 8122, System time : 00:00:12.7318648
System Memory : 65.4% ,2.29 gb
JVM Memory : 15.3% , 148.32mb of 981.38mb

After indexing 7,000 documents,
Qtime: 51817, System time : 00:01:12.6028320
System Memory : 69.4% 2.43Gb
JVM Memoery : *26.5%* , 266.60mb

After indexing 70,000 documents of 1200mb size, this are the results :
Qtime: 511447, System time : 00:11:14.0398768
System memory : 82.7% , 2.89Gb
JVM memory :* 11.8%* , 118.46mb

Here the JVM usage decreases as compared to 7000 doc, why is it  so?? 

This is* solrconfig.xml *;
<updateHandler class="solr.DirectUpdateHandler2">
    <updateLog>
        <str name="dir">${solr.document.log.dir:}</str>
    </updateLog>
   <autoSoftCommit>
        <maxTime>1000</maxTime>
    </autoSoftCommit>
    <autoCommit>
        <maxTime>600000</maxTime> 
        <openSearcher>true</openSearcher>
    </autoCommit>
</updateHandler>
 
I am indexing through solrnet, indexing each document,  var res =
solr.Add(doc). // Doc doc = new Doc();

How do i reduce the time for indexing, as the size of data indexed is quite
less?? Will batch indexing reduce the indexing time?? But then, do i need to
make changes in solrconfig.xml
Also, i want the documents to be searched in 1sec of indexing.
Is it true that, if softcommit is done, then faceting cannot be done on the
data??



--
View this message in context: http://lucene.472066.n3.nabble.com/to-reduce-indexing-time-tp4121391.html
Sent from the Solr - User mailing list archive at Nabble.com.

Mime
View raw message