incubator-lucy-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From goran kent <gorank...@gmail.com>
Subject Re: [lucy-user] Couldn't completely remove 'seg_N'
Date Thu, 17 Nov 2011 06:42:05 GMT
On 11/17/11, Marvin Humphrey <marvin@rectangular.com> wrote:
> The fact that you
> are
> encountering repeated errors there is troubling; the same symptom can arise
> when two indexers from different machines collide when trying to write to
> the
> same index on a shared drive.  I recall you saying that you had your own
> locking mechanism, but as a precaution,

Yes, something is not working as expected with my locking, I'm sure.
I have a feeling things become brittle under heavy load.

> I suggest that you use this code to
> provide a defense in depth against locking problems:
>
>     use Sys::Hostname qw( hostname );
>     my $hostname = hostname() or die "Can't get unique hostname";
>     my $manager = Lucy::Index::IndexManager->new(
>         host => $hostname,
>     );
>     my $indexer = Lucy::Index::Indexer->new(
>         index => '/path/to/index',
>         manager => $manager,
>     );


Thanks, I've stuck this in, so let's see how it goes.  The docs for
Lucy::Index::IndexManager don't say much about what exactly the above
will do.  Presumably it's an extra lock/concurrency check or something
that will croak if a conflict is detected?

I see the hostname then goes into write.lock:host, is that merely to
provide a fingerprint of the last machine to touch the index?  Also, I
recall vaguely something to the effect that the lockfile will be
overwritten anyway, does IndexManager prevent that, or maybe I'm
misunderstanding the context here?

Mime
View raw message