incubator-lucy-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Marvin Humphrey <mar...@rectangular.com>
Subject Re: [lucy-user] Couldn't completely remove 'seg_N'
Date Thu, 17 Nov 2011 06:17:03 GMT
On Wed, Nov 16, 2011 at 09:28:25AM +0200, goran kent wrote:
> I've noticed this occasional error/failure during index merging:
> 
>   Couldn't completely remove 'seg_N'
> 
> Which comes from core/Lucy/Index/SegWriter.c, where the comment in
> SegWriter_prep_seg_dir() says:
> 
>   // Clear stale segment files from crashed indexing sessions.
> 
> Is that stale segment folder from a previous unrelated index/merge
> session, or is it from the current session which has crashed/failed
> and this is part of the cleanup procedure?  

> It seems to be the former, am I right?

That code is indeed supposed to clear files from discrete indexing sessions
which crashed earlier.

It generally works without calling attention to itself.  The fact that you are
encountering repeated errors there is troubling; the same symptom can arise
when two indexers from different machines collide when trying to write to the
same index on a shared drive.  I recall you saying that you had your own
locking mechanism, but as a precaution, I suggest that you use this code to
provide a defense in depth against locking problems:

    use Sys::Hostname qw( hostname );
    my $hostname = hostname() or die "Can't get unique hostname";
    my $manager = Lucy::Index::IndexManager->new( 
        host => $hostname,
    );
    my $indexer = Lucy::Index::Indexer->new(
        index => '/path/to/index',
        manager => $manager,
    );

> (grep -rl '^Folder_Delete_Tree' * failed to find anything,
> so I couldn't have a quick look to confirm that idea)

Folder_Delete_Tree is a method call.  The implementing function which gets
called in this case is Folder_delete_tree, in core/Lucy/Store/Folder.c.  

Marvin Humphrey


Mime
View raw message