lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "suman.holani" <suman.hol...@zapak.co.in>
Subject RE: performance issues in multivalued fields
Date Mon, 07 Mar 2011 12:41:41 GMT
Thanks for prompt reply.

I am not using compression or lazy loading in either clucene and lucene.
Since I need  to get the data from lucene for all searched docs for further
processing

If In clucene it takes 15 ms 
In lucene it takes 100ms+ for the same search :( 

Number of hits is around 1000 docs

One more query : which tends to give better search performance in lucene -
increasing number of fields per doc or increasing the number of docs (by
increasing redundancy of data)

regards,
Suman



-----Original Message-----
From: Erick Erickson [mailto:erickerickson@gmail.com] 
Sent: Monday, March 07, 2011 5:50 PM
To: java-user@lucene.apache.org
Subject: Re: performance issues in multivalued fields

You have to describe in detail what "taking a huge performance hit"
means, there's not much to go on here...

 But in general, adding N elements to a mutli-valued field isn't a
problem at all.

This bit of code:
Document D = searcher.doc(hits[i].doc);
is very suspicious. Does your cLucene version have
lazy loading enabled and your Java version? Compression?
How many hits are you cycling this way? How long does the
search take as opposed to the above loop? Details please.

Best
Erick


On Mon, Mar 7, 2011 at 6:41 AM, suman.holani <suman.holani@zapak.co.in>
wrote:
> Hello,
>
>
>
> I am facing an issue for multivalued fields in lucene
>
>
>
> I am generating lucene doc , where page is multivalued .
>
> So my doc will be like this having more than n fields( which can be more
> than 1500 also ..) per doc in case page attribute
>
>
>
>
>
>
>
> Example
>
>
> <doc>
>                <media>
>                                <id>12345</id>
>                                <title>A title</title>
>                                <description>My description</description>
>                                <page>
>                                                <!-- The page
element can
> contain up to 15000 entries!!!! -->
>
>                                                          
 page1
>
> Page 2                                                .
>                                               .
>               .n
>
>
>  </page>
>
>   </media>
>
>
>
>
>
> </doc>
>
>
>
>
>
>
> Will this structure can give a performance hit..?? as number of fields ar
> dynamic for every doc..and can be huge.
>
>
>
>
>
>
>
> Actually I am using same structure in clucene and its running awesome. Bt
> lucene , is taking huge performance hit
>
> Specially in . "Document D = searcher.doc(hits[i].doc); "
>
>
>
>
>
> Regards
>
> Suman
>
>
>
>
>
>
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org





---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message