lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael McCandless <luc...@mikemccandless.com>
Subject Re: Field compression too slow
Date Fri, 11 Aug 2006 10:59:58 GMT

> I can share the data.. but it would be quicker for you to just pull out 
> some
> random text from anywhere you like.

OK, I hear you.  I'll pull together some test data ... thanks.

> Also.. upon reflection I'm not certain using compression inside the 
> index is
> really a valuable process without lazy loading anyway.  The time-cost of
> decompression when iterating hits reduces the overall effectiveness of the
> index.  This is obviously solved by lazy loading (for searches) and I am
> excited about this feature being added.  Obviously it depends on the
> use-case, but in mine I realised that storing large amounts of data in the
> index is just not the right way to do things.  So I changed my architecture
> so that the larger amounts of data are stored (and compressed) elsewhere,
> then brought back in when I need to update a document.
> 
> Of course all my problems would be solved if I had lazy loading AND field
> updating :)

Compleletely agreed!  Lazy loading & specific field selection on loading 
a doc, have been addressed ... but field updating in the presence of 
compressed fields hasn't yet been addressed (I think).  I'll raise this 
use case on the java-dev list.

Mike

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message