lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jason Rutherglen" <jason.rutherg...@gmail.com>
Subject Re: Re[4]: Frequently updated fields
Date Wed, 17 Sep 2008 17:44:45 GMT
Hi Wojciech,

Integration with SOLR would be ideal.  However that would
take more time.  It depends on the exact features.  There is at least
one patch to IndexWriter.  The merging is the part that needs to be
synchronized and this is where I am hesitant because Ocean/realtime
search performs merges outside of the IndexWriter merging to make
it realtime.  However integrating with IndexWriter is the long term
way to go as it looks like IndexWriter will be realtime as well soon.


2008/9/17 Wojciech Strzałka <wstrzalka@gmail.com>:
>
>  I'll ask my boss. I don't expect it to be soon, but when SOLR will be
>  implemented, we will see how it works for us and maybe we will want
>  the feature, as we already have very good experience in sponsoring
>  Open Source features.
>  Can you tell me how big the project is? Are we talking about
>  hundreds, thousands, tenth of thousands $ ?
>  Do you think integration with SOLR would be also possible?
>
>> Hi Wojciech,
>
>> The code isn't ready, it is a major project and I am trying to also
>> complete the realtime indexing patches and look for a job.  I believe
>> that the tag indexing stuff is of interest to many people so if there
>> is someone who can pay to get it completed feel free to contact me as
>> I am available to do so.  The goal is to integrate tag index into
>> realtime indexing so that it is seamless and easy to use.  This will
>> not be easy however I gained a lot of knowledge on how to implement
>> realtime search design when writing the realtime indexing and serach
>> code.
>
>> Jason
>
>> On Tue, Sep 16, 2008 at 8:56 AM, Wojciech Strzałka <wstrzalka@gmail.com> wrote:
>>>
>>>  I saw your comments on JIRA. You mentioned about rework and I'm wondering if
the currently
>>>  available patch is production ready (functionally complete)?
>>>  Will the code after rework work with the index build with the current
>>>  version?
>>>
>>>  I'm quite new to SOLR/Lucene but I hope I could write custom
>>>  SearchComponent to SOLR based on yours Tag Index and this way get
>>>  exactly what I need.
>>>
>>>  And one more question:  I don't really get now how could I setup field to be
indexed by
>>>  TagIndexWriter instead of TagWriter. I know it's somehow handled by
>>>  ParallelIndexWriter but how can I setup it?
>>>
>>>
>>>> Yes Tag Index will work.  I have not had time to complete it however
>>>> if you are interested in working on it please feel free to contact me.
>>>
>>>> On Fri, Sep 12, 2008 at 3:48 PM, Mark Miller <markrmiller@gmail.com>
wrote:
>>>>> You might check out the tagindex issue in jira as well. Havn't looked
at it
>>>>> myself, but I believe its supposed to be an option for this.
>>>>>
>>>>> Gerardo Segura wrote:
>>>>>>
>>>>>> I think the important question is: in general how to cope with frequently
>>>>>> changing fields.
>>>>>>
>>>>>>
>>>>>> Karl Wettin wrote:
>>>>>>>
>>>>>>> Hi Wojciech,
>>>>>>>
>>>>>>> can you please give us a bit more specific information about
the meta
>>>>>>> data fields that will change? I would recommend you looking at
creating
>>>>>>> filters from your primary persistency for query clauses such
as unread/read,
>>>>>>> mailbox folders, et c.
>>>>>>>
>>>>>>>      karl
>>>>>>>
>>>>>>> 12 sep 2008 kl. 13.57 skrev Wojciech Strza?ka:
>>>>>>>
>>>>>>>> Hi.
>>>>>>>>
>>>>>>>>  I'm new to Lucene and I would like to get a few answers
(they can
>>>>>>>>  be lame)
>>>>>>>>
>>>>>>>>  I want to index large amount of emails using Lucene (maybe
SOLR), not
>>>>>>>> only
>>>>>>>>  the contents but also some metadata like state or flags.
The
>>>>>>>>  problem is that the metadata will change during mail lifecycle,
>>>>>>>>  although much smaller updating this information will require
>>>>>>>>  reindex the whole mail content which I see performance bottleneck.
>>>>>>>>
>>>>>>>>  I have the data in DB also so my first question is:
>>>>>>>>
>>>>>>>>  - are there any best practices to implement my needs (querying
both
>>>>>>>>  lucene & DB and then merging in memory?, close one eye
and re-index
>>>>>>>>  the whole content on every metadata change? others?)
>>>>>>>>
>>>>>>>>  - is at all Lucene good solution for my problem?
>>>>>>>>
>>>>>>>>  - are there any plans to implement field updates in more
efficient way
>>>>>>>> then
>>>>>>>>  delete/insert the whole document? if yes what's the time
horizon?
>>>>>>>>
>>>>>>>>
>>>>>>>>                                       Best regards
>>>>>>>>                                              Wojtek
>>>>>>>>
>>>>>>>>
>>>>>>>> ---------------------------------------------------------------------
>>>>>>>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>>>>>>>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> ---------------------------------------------------------------------
>>>>>>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>>>>>>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>> ---------------------------------------------------------------------
>>>>>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>>>>>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>>>>>
>>>>>
>>>>>
>>>>> ---------------------------------------------------------------------
>>>>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>>>>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>>>>
>>>>>
>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>>>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>>
>>>
>>>
>>> --
>>> Pozdrowienia,
>>>  Wojciech Strzalka
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>>
>>>
>
>
>
> --
> Pozdrowienia,
>  Wojciech Strzałka
>
>
Mime
View raw message