lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Grant Ingersoll <gsing...@syr.edu>
Subject Re: Field compression too slow
Date Fri, 11 Aug 2006 12:39:00 GMT
SVN Head does.  Has not been released yet.  	

See http://issues.apache.org/jira/browse/LUCENE-545
and
http://issues.apache.org/jira/browse/LUCENE-609

for some of the issues with it.

On Aug 11, 2006, at 8:19 AM, Dragon Fly wrote:

> Mike, which version of Lucene supports lazy loading? Thanks.
>
>> From: Michael McCandless <lucene@mikemccandless.com>
>> Reply-To: java-user@lucene.apache.org
>> To: java-user@lucene.apache.org
>> Subject: Re: Field compression too slow
>> Date: Fri, 11 Aug 2006 06:59:58 -0400
>>
>>
>>> I can share the data.. but it would be quicker for you to just  
>>> pull out some
>>> random text from anywhere you like.
>>
>> OK, I hear you.  I'll pull together some test data ... thanks.
>>
>>> Also.. upon reflection I'm not certain using compression inside  
>>> the index is
>>> really a valuable process without lazy loading anyway.  The time- 
>>> cost of
>>> decompression when iterating hits reduces the overall  
>>> effectiveness of the
>>> index.  This is obviously solved by lazy loading (for searches)  
>>> and I am
>>> excited about this feature being added.  Obviously it depends on the
>>> use-case, but in mine I realised that storing large amounts of  
>>> data in the
>>> index is just not the right way to do things.  So I changed my  
>>> architecture
>>> so that the larger amounts of data are stored (and compressed)  
>>> elsewhere,
>>> then brought back in when I need to update a document.
>>>
>>> Of course all my problems would be solved if I had lazy loading  
>>> AND field
>>> updating :)
>>
>> Compleletely agreed!  Lazy loading & specific field selection on  
>> loading a doc, have been addressed ... but field updating in the  
>> presence of compressed fields hasn't yet been addressed (I  
>> think).  I'll raise this use case on the java-dev list.
>>
>> Mike
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>
>
> _________________________________________________________________
> Express yourself instantly with MSN Messenger! Download today -  
> it's FREE! http://messenger.msn.click-url.com/go/onm00200471ave/ 
> direct/01/
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>

--------------------------
Grant Ingersoll
Sr. Software Engineer
Center for Natural Language Processing
Syracuse University
335 Hinds Hall
Syracuse, NY 13244
http://www.cnlp.org

Voice: 315-443-5484
Skype: grant_ingersoll
Fax: 315-443-6886




---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message