lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Aalap Parikh <alo...@yahoo.com>
Subject Re: Lucene bulk indexing
Date Fri, 22 Apr 2005 17:24:30 GMT
Hi,

> : the app using JProfiler and found out that 90% of
> time
> : is spent in the IndexWriter.addDocument call. As
> 
> what analyzer are you using?

I am using the StandardAnalyzer (tried using
SimpleAnalyzer too, but not much affect on
performance).

> : My machine: Pentium 4 CPU 2.40 GHz
> :             RAM 1 GB
> 
> what JVM args are you using? (in particular: how
> much ram are you telling
> the JVM to use) ... what else is running on this box
> while you index?

I am using -Xms32M -Xmx200M . I have tried giving more
but doesn't help much.

> The biggest help anyone an give you is if you post
> your code ... there may

Here is what my code looks like:

Document doc = null;
IndexWriter writer = new IndexWriter(indexDirPath, 
                         new StandardAnalyzer(),
true);

for (int i = 0; i < dataArrayToIndex.length; i++) {
    doc = new Document();
    doc.add(Field.UnIndexed("ID",  
        
String.valueOf(dataArrayToIndex[i].getId())));
    doc.add(Field.Keyword("claimCode", 
             dataArrayToIndex[i].getClaimCode()));
    doc.add(Field.Keyword("insuredName", 
             dataArrayToIndex[i].getInsuredName()));
    doc.add(Field.Keyword("claimantName", 
             dataArrayToIndex[i].getClaimantName()));
    ....
    //and so on with about 27 fields in total, most of
 
    //them indexed as Field.Keyword with text not more

    //than 10-20 character each.

    writer.addDocument(doc);
 } //end of for loop

 writer.optimize();
 writer.close();

Thanks,
Aalap.


--- Chris Hostetter <hossman_lucene@fucit.org> wrote:
> : the app using JProfiler and found out that 90% of
> time
> : is spent in the IndexWriter.addDocument call. As
> 
> what analyzer are you using?
> 
> : My machine: Pentium 4 CPU 2.40 GHz
> :             RAM 1 GB
> 
> what JVM args are you using? (in particular: how
> much ram are you telling
> the JVM to use) ... what else is running on this box
> while you index?
> 
> : Any help appreciated.
> 
> The biggest help anyone an give you is if you post
> your code ... there may
> be little things in the way you are using the
> IndexWriter that will jump
> out at people if they see exactly what you are
> doing.
> 
> 
> -Hoss
> 
> 
>
---------------------------------------------------------------------
> To unsubscribe, e-mail:
> java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail:
> java-user-help@lucene.apache.org
> 
> 

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message