lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Aviran" <amo...@infosciences.com>
Subject RE: Sort: 1.4-rc3 vs. 1.4-final
Date Wed, 21 Jul 2004 17:42:01 GMT
I think I found the problem
FieldCacheImpl uses WeakHashMap to store the cached objects, but since there
is no other reference to this cache it is getting released.
Switching to HashMap solves it.
The only problem is that I don't see anywhere where the cached object will
get released if you open a new IndexReader.

Aviran

-----Original Message-----
From: Greg Gershman [mailto:greggersh@yahoo.com] 
Sent: Wednesday, July 21, 2004 13:13 PM
To: Lucene Users List
Subject: RE: Sort: 1.4-rc3 vs. 1.4-final


I've done a bit more snooping around; it seems that in
FieldSortedHitQueue.getCachedComparator(line 153), calls to lookup a stored
comparator in the cache always return null.  This occurs even for the
built-in sort types (I tested it on integers and my code for longs).  The
comparators don't even appear to be being stored in the HashMap to begin
with.

Any ideas?

Greg

 

--- Aviran <amordo@infosciences.com> wrote:
> Since I had to implement sorting in lucene 1.2 I had
> to write my own sorting
> using something similar to a lucene's contribution
> called SortField.
> Yesterday I did some tests, trying to use lucene 1.4
> Sort objects and I
> realized that my old implementation works 40% faster
> then Lucene's
> implementation. My guess is that you are right and
> there is a problem with
> the cache although I couldn't find what that is yet.
> 
> Aviran
> 
> -----Original Message-----
> From: Greg Gershman [mailto:greggersh@yahoo.com]
> Sent: Wednesday, July 21, 2004 9:22 AM
> To: lucene-user@jakarta.apache.org
> Subject: Sort: 1.4-rc3 vs. 1.4-final
> 
> 
> When rc3 came out, I modified the classes used for
> Sorting to, in addition to Integer, Float and
> String-based sort keys, use Long values.  All I did
> was add extra statements in 2 classes (SortField and
> FieldSortedHitQueue) that made a special case for
> longs, and created a LongSortedHitQueue identical to
> the IntegerSortedHitQueue, only using longs.
> 
> This worked as expected; Long values converted to
> strings and stored in Field.Keyword type fields
> would
> be sorted according to Long order.  The initial
> query
> would take a while, to build the sorted array, but
> subsequent queries would take little to no time at
> all.
> 
> I went back to look at 1.4 final, and noticed the
> Sort implementation has
> changed quite a bit.  I tried the same type of
> modifications to the existing
> source files, but was unable to achieve similiar
> results.
> Each subsequent query seems to take a significant
> amount of time, as if the Sorted array is being
> rebuilt each time.  Also, I tried sorting on an
> Integer fields and got similar results, which leads
> me
> to believe there might be a caching problem
> somewhere.
> 
> Has anyone else seen this in 1.4-final?  Also, I
> would
> like it if Long sorted fields could become a part of
> the API; it makes sorting by date a breeze.
> 
> Thanks!
> 
> Greg Gershman
> 
> 
> 		
> __________________________________
> Do you Yahoo!?
> New and Improved Yahoo! Mail - Send 10MB messages!
> http://promotions.yahoo.com/new_mail
> 
>
---------------------------------------------------------------------
> To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: lucene-user-help@jakarta.apache.org
> 
> 
> 
> 
>
---------------------------------------------------------------------
> To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: lucene-user-help@jakarta.apache.org
> 
> 



	
		
__________________________________
Do you Yahoo!?
Vote for the stars of Yahoo!'s next ad campaign!
http://advision.webevents.yahoo.com/yahoo/votelifeengine/

---------------------------------------------------------------------
To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: lucene-user-help@jakarta.apache.org




---------------------------------------------------------------------
To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: lucene-user-help@jakarta.apache.org


Mime
View raw message