lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Shai Erera <ser...@gmail.com>
Subject Re: IndexReplication Client and IndexWriter
Date Wed, 16 Apr 2014 06:49:29 GMT
Hi Christoph,

Apologize for the delayed response, I'm on a holiday vacation. I will take
a look at your issues as soon as I can.

Shai


On Fri, Apr 11, 2014 at 12:02 PM, Christoph Kaser
<lucene_list@iconparc.de>wrote:

> Hello Shai and Mike,
>
> thank you for your answers!
>
> I created LUCENE-5597 for this feature. Unfortunately, I am not sure I
> will be able to provide patches: I don't need this feature at the moment
> (my interest was more academic) and unfortunately don't have the time to
> work on this.
>
> Additionally, I created LUCENE-5599, which provides a patch to fix a small
> performance issue I had with the replicator when replicating large indexes.
>
> Regards,
> Christoph Kaser
>
>
>
> Am 08.04.2014 12:45, schrieb Michael McCandless:
>
>> You might be able to use a class on the NRT replication branch
>> (LUCENE-5438), InfosRefCounts (weird name), whose purpose is to do
>> what IndexFileDeleter does for IndexWriter, ie keep track of which
>> files are still referenced, delete them when they are done, etc.  This
>> could used on the client side to hold a lease for another client.
>>
>> Mike McCandless
>>
>> http://blog.mikemccandless.com
>>
>>
>> On Tue, Apr 8, 2014 at 6:26 AM, Shai Erera <serera@gmail.com> wrote:
>>
>>> IndexRevision uses the IndexWriter for deleting unused files when the
>>> revision is released, as well as to obtain the SnapshotDeletionPolicy.
>>>
>>> I think that you will need to implement two things on the "client" side:
>>>
>>> * Revision, which doesn't use IndexWriter.
>>> * Replicator which keeps track of how many refs a file has (basically
>>> what
>>> IndexFileDeleter does)
>>>
>>> Then you could setup any node in the middle to be both a client and a
>>> server. Would be interesting to explore that. Would you like to open an
>>> issue? And maybe even try to come up w/ a patch?
>>>
>>> Shai
>>>
>>>
>>> On Tue, Apr 8, 2014 at 1:05 PM, Michael McCandless <
>>> lucene@mikemccandless.com> wrote:
>>>
>>>  It's not safe also opening an IndexWriter on the client side.
>>>>
>>>> But I agree, supporting tree topology would make sense; it seems like
>>>> we just need a way for the ReplicationClient to also be a Replicator.
>>>> It seems like it should be possible, since it's clearly aware of the
>>>> SessionToken it's pulled from the original Replicator.
>>>>
>>>> Mike McCandless
>>>>
>>>> http://blog.mikemccandless.com
>>>>
>>>>
>>>> On Tue, Apr 8, 2014 at 3:42 AM, Christoph Kaser <
>>>> lucene_list@iconparc.de>
>>>> wrote:
>>>>
>>>>> Hi all,
>>>>>
>>>>> I am trying out the (highly useful) index replicator module (with the
>>>>> HttpReplicator) and have stumbled upon a question:
>>>>> It seems, the IndexReplicationHandler is working directly on the index
>>>>> directory, without using an indexwriter. Could there be a problem if
I
>>>>>
>>>> open
>>>>
>>>>> an IndexWriter on the client side?
>>>>> Usually, this should not be needed, as only the master should be
>>>>> changed,
>>>>> however if I want to implement a "tree topology", I need an IndexWriter
>>>>>
>>>> on a
>>>>
>>>>> non-leaf client, because the IndexRevision that I need to publish needs
>>>>>
>>>> one.
>>>>
>>>>> Regards,
>>>>> Christoph
>>>>>
>>>>>
>>>>>  ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>>>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>>>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>>>
>>>>
>>>>  ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>
>>
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message