lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Benson Margulies <bimargul...@gmail.com>
Subject Re: Liberating DirectPostingFormat from Codec
Date Mon, 27 Jan 2014 12:06:25 GMT
What do we have for a benchmark framework that is used to
justify/qualify speed-related things? One way forward would be to see
what a quantified measurement shows from the idea I have in mind, and
use that to facilitate deciding if this belongs in the tree.

On Sat, Jan 25, 2014 at 6:34 PM, Benson Margulies <bimargulies@gmail.com> wrote:
> Keeping things in memory and not re-reading them from disk is what
> really sang the song for us. Even if the initial read-in was more
> costly due to decompression, the long-term amortized benefit of not
> re-reading would still be a big winner.
>
>
> On Sat, Jan 25, 2014 at 5:37 PM, Robert Muir <rcmuir@gmail.com> wrote:
>> well the Directory layer likely isnt what probably makes DirectPF faster for
>> you. Its probably the fact it does no compression at all...
>>
>>
>> On Sat, Jan 25, 2014 at 5:34 PM, Benson Margulies <bimargulies@gmail.com>
>> wrote:
>>>
>>> On Sat, Jan 25, 2014 at 5:09 PM, Robert Muir <rcmuir@gmail.com> wrote:
>>> > That would be Directory :)
>>>
>>> Oh,  how embarrassing. I could have written a custom directory to begin
>>> with.
>>>
>>> Would a Directory class for this purpose be an interesting patch, in
>>> that case? I'm not discontented about building a Directory into our
>>> application, but it seems like I might not be the only person to find
>>> this useful.
>>>
>>> >
>>> >
>>> > On Sat, Jan 25, 2014 at 5:03 PM, Benson Margulies
>>> > <bimargulies@gmail.com>
>>> > wrote:
>>> >>
>>> >> I've had very gratifying results using the DirectPostingFormat to
>>> >> speed up queries when I had a read-only index with plenty of memory.
>>> >> The only downside was the need to specify it within the Codec, and
>>> >> thus write it into the index.
>>> >>
>>> >> Ever since, I've wondered if we could change things to introduce the
>>> >> same goodness without building it into the codec.
>>> >>
>>> >> Very roughly, I'm imagining an option in the IndexReader to provide
an
>>> >> object that can surround the codec that is called for in the stored
>>> >> format.
>>> >>
>>> >> Is this an old question? Is it worth sketching a patch?
>>> >>
>>> >> ---------------------------------------------------------------------
>>> >> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>>> >> For additional commands, e-mail: dev-help@lucene.apache.org
>>> >>
>>> >
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>>> For additional commands, e-mail: dev-help@lucene.apache.org
>>>
>>

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message