lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Christoph Kaser <lucene_l...@iconparc.de>
Subject Re: IndexReplication Client and IndexWriter
Date Fri, 11 Apr 2014 09:02:47 GMT
Hello Shai and Mike,

thank you for your answers!

I created LUCENE-5597 for this feature. Unfortunately, I am not sure I 
will be able to provide patches: I don't need this feature at the moment 
(my interest was more academic) and unfortunately don't have the time to 
work on this.

Additionally, I created LUCENE-5599, which provides a patch to fix a 
small performance issue I had with the replicator when replicating large 
indexes.

Regards,
Christoph Kaser



Am 08.04.2014 12:45, schrieb Michael McCandless:
> You might be able to use a class on the NRT replication branch
> (LUCENE-5438), InfosRefCounts (weird name), whose purpose is to do
> what IndexFileDeleter does for IndexWriter, ie keep track of which
> files are still referenced, delete them when they are done, etc.  This
> could used on the client side to hold a lease for another client.
>
> Mike McCandless
>
> http://blog.mikemccandless.com
>
>
> On Tue, Apr 8, 2014 at 6:26 AM, Shai Erera <serera@gmail.com> wrote:
>> IndexRevision uses the IndexWriter for deleting unused files when the
>> revision is released, as well as to obtain the SnapshotDeletionPolicy.
>>
>> I think that you will need to implement two things on the "client" side:
>>
>> * Revision, which doesn't use IndexWriter.
>> * Replicator which keeps track of how many refs a file has (basically what
>> IndexFileDeleter does)
>>
>> Then you could setup any node in the middle to be both a client and a
>> server. Would be interesting to explore that. Would you like to open an
>> issue? And maybe even try to come up w/ a patch?
>>
>> Shai
>>
>>
>> On Tue, Apr 8, 2014 at 1:05 PM, Michael McCandless <
>> lucene@mikemccandless.com> wrote:
>>
>>> It's not safe also opening an IndexWriter on the client side.
>>>
>>> But I agree, supporting tree topology would make sense; it seems like
>>> we just need a way for the ReplicationClient to also be a Replicator.
>>> It seems like it should be possible, since it's clearly aware of the
>>> SessionToken it's pulled from the original Replicator.
>>>
>>> Mike McCandless
>>>
>>> http://blog.mikemccandless.com
>>>
>>>
>>> On Tue, Apr 8, 2014 at 3:42 AM, Christoph Kaser <lucene_list@iconparc.de>
>>> wrote:
>>>> Hi all,
>>>>
>>>> I am trying out the (highly useful) index replicator module (with the
>>>> HttpReplicator) and have stumbled upon a question:
>>>> It seems, the IndexReplicationHandler is working directly on the index
>>>> directory, without using an indexwriter. Could there be a problem if I
>>> open
>>>> an IndexWriter on the client side?
>>>> Usually, this should not be needed, as only the master should be changed,
>>>> however if I want to implement a "tree topology", I need an IndexWriter
>>> on a
>>>> non-leaf client, because the IndexRevision that I need to publish needs
>>> one.
>>>> Regards,
>>>> Christoph
>>>>
>>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>>
>>>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>



  


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message