lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Doug Cutting <cutt...@apache.org>
Subject Re: Sort: 1.4-rc3 vs. 1.4-final
Date Wed, 21 Jul 2004 17:56:22 GMT
The key in the WeakHashMap should be the IndexReader, not the Entry.  I 
think this should become a two-level cache, a WeakHashMap of HashMaps, 
the WeakHashMap keyed by IndexReader, the HashMap keyed by Entry.  I 
think the Entry class can also be changed to not include an IndexReader 
field.  Does this make sense?  Would someone like to construct a patch 
and submit it to the developer list?

Doug

Aviran wrote:
> I think I found the problem
> FieldCacheImpl uses WeakHashMap to store the cached objects, but since there
> is no other reference to this cache it is getting released.
> Switching to HashMap solves it.
> The only problem is that I don't see anywhere where the cached object will
> get released if you open a new IndexReader.
> 
> Aviran
> 
> -----Original Message-----
> From: Greg Gershman [mailto:greggersh@yahoo.com] 
> Sent: Wednesday, July 21, 2004 13:13 PM
> To: Lucene Users List
> Subject: RE: Sort: 1.4-rc3 vs. 1.4-final
> 
> 
> I've done a bit more snooping around; it seems that in
> FieldSortedHitQueue.getCachedComparator(line 153), calls to lookup a stored
> comparator in the cache always return null.  This occurs even for the
> built-in sort types (I tested it on integers and my code for longs).  The
> comparators don't even appear to be being stored in the HashMap to begin
> with.
> 
> Any ideas?
> 
> Greg
> 
>  
> 
> --- Aviran <amordo@infosciences.com> wrote:
> 
>>Since I had to implement sorting in lucene 1.2 I had
>>to write my own sorting
>>using something similar to a lucene's contribution
>>called SortField.
>>Yesterday I did some tests, trying to use lucene 1.4
>>Sort objects and I
>>realized that my old implementation works 40% faster
>>then Lucene's
>>implementation. My guess is that you are right and
>>there is a problem with
>>the cache although I couldn't find what that is yet.
>>
>>Aviran
>>
>>-----Original Message-----
>>From: Greg Gershman [mailto:greggersh@yahoo.com]
>>Sent: Wednesday, July 21, 2004 9:22 AM
>>To: lucene-user@jakarta.apache.org
>>Subject: Sort: 1.4-rc3 vs. 1.4-final
>>
>>
>>When rc3 came out, I modified the classes used for
>>Sorting to, in addition to Integer, Float and
>>String-based sort keys, use Long values.  All I did
>>was add extra statements in 2 classes (SortField and
>>FieldSortedHitQueue) that made a special case for
>>longs, and created a LongSortedHitQueue identical to
>>the IntegerSortedHitQueue, only using longs.
>>
>>This worked as expected; Long values converted to
>>strings and stored in Field.Keyword type fields
>>would
>>be sorted according to Long order.  The initial
>>query
>>would take a while, to build the sorted array, but
>>subsequent queries would take little to no time at
>>all.
>>
>>I went back to look at 1.4 final, and noticed the
>>Sort implementation has
>>changed quite a bit.  I tried the same type of
>>modifications to the existing
>>source files, but was unable to achieve similiar
>>results.
>>Each subsequent query seems to take a significant
>>amount of time, as if the Sorted array is being
>>rebuilt each time.  Also, I tried sorting on an
>>Integer fields and got similar results, which leads
>>me
>>to believe there might be a caching problem
>>somewhere.
>>
>>Has anyone else seen this in 1.4-final?  Also, I
>>would
>>like it if Long sorted fields could become a part of
>>the API; it makes sorting by date a breeze.
>>
>>Thanks!
>>
>>Greg Gershman
>>
>>
>>		
>>__________________________________
>>Do you Yahoo!?
>>New and Improved Yahoo! Mail - Send 10MB messages!
>>http://promotions.yahoo.com/new_mail
>>
>>
> 
> ---------------------------------------------------------------------
> 
>>To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
>>For additional commands, e-mail: lucene-user-help@jakarta.apache.org
>>
>>
>>
>>
>>
> 
> ---------------------------------------------------------------------
> 
>>To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
>>For additional commands, e-mail: lucene-user-help@jakarta.apache.org
>>
>>
> 
> 
> 
> 
> 	
> 		
> __________________________________
> Do you Yahoo!?
> Vote for the stars of Yahoo!'s next ad campaign!
> http://advision.webevents.yahoo.com/yahoo/votelifeengine/
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: lucene-user-help@jakarta.apache.org
> 
> 
> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: lucene-user-help@jakarta.apache.org
> 

---------------------------------------------------------------------
To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: lucene-user-help@jakarta.apache.org


Mime
View raw message