Mailing-List: contact lucy-user-help@incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: lucy-user@incubator.apache.org
Received-SPF: pass (nike.apache.org: local policy)
Date: Wed, 16 Nov 2011 22:17:03 -0800
From: Marvin Humphrey <marvin@rectangular.com>
To: lucy-user@incubator.apache.org
Message-ID: <20111117061703.GA27162@rectangular.com>
References: 
 <CAKn6EF-o+aNM5icZg=7O2+7uVfSce_3xbMC7=Ym+jxnE4R9juw@mail.gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: 
 <CAKn6EF-o+aNM5icZg=7O2+7uVfSce_3xbMC7=Ym+jxnE4R9juw@mail.gmail.com>
User-Agent: Mutt/1.5.18 (2008-05-17)
Subject: Re: [lucy-user] Couldn't completely remove 'seg_N'

On Wed, Nov 16, 2011 at 09:28:25AM +0200, goran kent wrote:
> I've noticed this occasional error/failure during index merging:
> 
>   Couldn't completely remove 'seg_N'
> 
> Which comes from core/Lucy/Index/SegWriter.c, where the comment in
> SegWriter_prep_seg_dir() says:
> 
>   // Clear stale segment files from crashed indexing sessions.
> 
> Is that stale segment folder from a previous unrelated index/merge
> session, or is it from the current session which has crashed/failed
> and this is part of the cleanup procedure?  

> It seems to be the former, am I right?

That code is indeed supposed to clear files from discrete indexing sessions
which crashed earlier.

It generally works without calling attention to itself.  The fact that you are
encountering repeated errors there is troubling; the same symptom can arise
when two indexers from different machines collide when trying to write to the
same index on a shared drive.  I recall you saying that you had your own
locking mechanism, but as a precaution, I suggest that you use this code to
provide a defense in depth against locking problems:

    use Sys::Hostname qw( hostname );
    my $hostname = hostname() or die "Can't get unique hostname";
    my $manager = Lucy::Index::IndexManager->new( 
        host => $hostname,
    );
    my $indexer = Lucy::Index::Indexer->new(
        index => '/path/to/index',
        manager => $manager,
    );

> (grep -rl '^Folder_Delete_Tree' * failed to find anything,
> so I couldn't have a quick look to confirm that idea)

Folder_Delete_Tree is a method call.  The implementing function which gets
called in this case is Folder_delete_tree, in core/Lucy/Store/Folder.c.  

Marvin Humphrey