lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael McCandless <luc...@mikemccandless.com>
Subject Re: IndexingChain and TermHash
Date Thu, 07 Jan 2010 13:43:41 GMT
LUCENE-1410 has the PFor impl, that the PFor codec needs.

Mike

On Thu, Jan 7, 2010 at 7:46 AM, Renaud Delbru <renaud.delbru@deri.org> wrote:
> Hi Michael,
>
> I have started to look at the PFOR codec. However, when I include the codec
> files inside the flex_1458 branch, it misses the
> org.apache.lucene.util.pfor.PFor class which is the core of the codec. Where
> can I find this class ?
>
> Thanks,
> Regards
> --
> Renaud Delbru
>
> On 16/11/09 14:01, Michael McCandless wrote:
>>
>> Yes, the branch is here:
>>
>>     https://svn.apache.org/repos/asf/lucene/java/branches/flex_1458
>>
>> Mark (Miller) periodically re-sync's it to trunk.
>>
>> All tests should pass, and if you create a new Codec, please share the
>> experience!
>>
>> There are not yet many Codecs in existence... the branch has the
>> "standard" codec (closest to Lucene's current index format, but makes
>> some compelling improvements to the terms dict), a "pulsing" codec
>> (which inlines low-freq terms into the terms dict), an intblock codec
>> (an abstract base for building int-block codecs).  There's also the
>> PForDelta codec, attached to LUCENE-1410, which subclasses the
>> intblock codec and uses PForDelta encoding.  It's probably best to
>> peek at these example codecs for inspiration on how to build yours.
>>
>> Mike
>>
>> On Mon, Nov 16, 2009 at 7:28 AM, Renaud Delbru<renaud.delbru@deri.org>
>>  wrote:
>>
>>>
>>> Hi Michael,
>>>
>>> I see there is already a huge amount of work already done in LUCENE-1458.
>>> Is
>>> there a way to checkout the corresponding branch, and start to use it ?
>>> At
>>> least, to see if I can extend it and create my own Codec.
>>> I have started on my side to abstract the indexing chain of Lucene 2.9,
>>> in
>>> order to be able to plug my own chain, but I have the impression that
>>> you've
>>> done something similar already (with the codec abstraction). Would be a
>>> pity
>>> to lose my time doing something less convenient that your appraoch.
>>>
>>> Thanks.
>>> --
>>> Renaud Delbru
>>>
>>> On 14/11/09 13:22, Michael McCandless wrote:
>>>
>>>>
>>>> On Fri, Nov 6, 2009 at 1:34 PM, Renaud Delbru<renaud.delbru@deri.org>
>>>>  wrote:
>>>>
>>>>
>>>>>
>>>>> Hi Michael,
>>>>>
>>>>> Thanks for the quick fix. I have tested it (indexing multiple documents
>>>>> +
>>>>> searching), and it seems to work.
>>>>>
>>>>> On 06/11/09 18:09, Michael McCandless wrote:
>>>>>
>>>>>
>>>>>>
>>>>>> To be honest, you are sort of forging new territory here :)
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>> I think so too, not an easy task ;o). I have seen that you have tried
>>>>> to
>>>>> make modular the indexing chain of Lucene (DocumentsWriter). I still
>>>>> try
>>>>> to
>>>>> have a good understanding of the default indexing, but I would like to
>>>>> see
>>>>> how it is easy (or difficult) to modify the format of the postings.
>>>>> From
>>>>> my
>>>>> current understanding, it seems that only the consumer at the end of
>>>>> this
>>>>> chain (FreqProxTermsWriter and its consumer FormatPostingsFieldsWriter)
>>>>> has
>>>>> to be changed to a certain extend.
>>>>>
>>>>>
>>>>
>>>> Right, those two classes do the writing of the postings, currently.
>>>>
>>>> But with flexible indexing (LUCENE-1458), still in progress, we hope
>>>> to make it more easily pluggable, the codec that actually reads&
>>>> writes the postings.
>>>>
>>>> Mike
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>>>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>>>
>>>>
>>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>>
>>>
>>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message