incubator-lucy-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Marvin Humphrey <>
Subject Re: [lucy-user] Index state during merges
Date Fri, 04 Nov 2011 00:25:24 GMT
On Thu, Nov 03, 2011 at 09:16:57AM +0200, goran kent wrote:
> On Wed, Nov 2, 2011 at 9:03 PM, goran kent <> wrote:
> > Is it ok to remove the seg_2 folder and lockfile and rename
> > snapshot_2.json to snapshot_1.json and likewise with schema_2.json ->
> > schema_1.json?

Here's the party line: Lucy's file format is not an official public API, and
thus hand editing of files is not supported.

Nevertheless, the human readable JSON format was deliberately selected to
facilitate troubleshooting and curious hacking, and it's gratifying to see
that you were able to able to intuit your way through.  :)

> Removed:
> locks/write.lock
> seg_2/

FWIW, that write.lock file (which is also in JSON format) would have been
zapped the next time you opened an Indexer.  The relevant method is


    Release all locks that meet the following three conditions: the lock name
    matches, the host id matches, and the process id that the lock was created
    under no longer identifies an active process.

With regards to removing seg_2 (the most recent segment), that worked just as
you surmised.  However, seg_2's data presumably existed at one time; it was
almost certainly wiped out when an Indexer session from one machine did not
realize that a write.lock file was owned by an active Indexer session running
on another machine.

> Renamed:
> schema_2.json -> schema_1.json
> snapshot_2.json -> snapshot_1.json

The number in the schema file name is not sync'd with the number in the
snapshot file name.  Changing the number in the schema file name achieved no
purpose, but it was harmless since you also edited the name in the snapshot
file which referenced it. 

Changing the number embedded in the snapshot file name had no effect since
there was only one snapshot file present in the index directory.

> At least, a search on the index returns results, that being the only way to
> check if the index is sane...  (lisc - lucy-index-system-check anyone?)

An "lisc" tool would certainly be useful.  I don't know of any other Lucy
community member who is working on something like that, and we don't have a
pressing need for it at my office (because we have me :P).  If you want to
work on it yourself, though, I will help -- my top priority, in the near-term
and medium-term, is to help other developers help themselves.

> btw, lucy designers, thank you for using human-readable-editable
> files, it is the UNIX way.

We are on the same page, and I'm sure I can speak for Peter and the others who
were involved in that design when I say: it's great that you were able to make
use of them.


Marvin Humphrey

View raw message