lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jason Rutherglen <>
Subject Re: Recreate segment infos
Date Tue, 05 Oct 2010 12:38:27 GMT
I'm not sure how it vanished, I think it was on a Solr rsync based
replication operation, and a lack of disk space.  Need to move to the
Java replication and get larger SSD drives, working on both, at least
they're SSDs, making some progress.  I was going to recover using the
IDs in the terms dict however there should be 130 mil and there were
only 16 mil.  So even if I had a way to recover, the index is far too
incomplete.  This is where re-indexing in Hadoop is coming in handy.

On Tue, Oct 5, 2010 at 3:26 AM, Michael McCandless
<> wrote:
> How did you lose your segments file...?
> This was discussed before but I don't think the idea ever turned into a tool.
> I think it should be possible.  You'd have to sort all files, deriving
> segment names from the prefixes.  Then, you have to reconstruct the
> metadata required for SegmentInfo.  EG open the fdx file to get
> numDocs, the .del file to get delCount, check for prx file to set
> .haxProx, etc.
> You'd have to carefully map segment -> doc store segment.  Multiple
> segments in a row may share the same docStore segment.  In this case
> the docStore segment is given the same name as the first segment that
> shares it.  However, unfortunately, because of merging, it's possible
> that this mapping is not easy (maybe not possible, depending on the
> merge policy...) to reconstruct.  I think this'll be the hardest part
> :)
> Mike
> On Mon, Oct 4, 2010 at 3:25 PM, Jason Rutherglen
> <> wrote:
>> Lets say the segment infos file is missing, and I'm aware of
>> CheckIndex, however is there a tool to recreate a segment infos file?
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail:
>> For additional commands, e-mail:
> ---------------------------------------------------------------------
> To unsubscribe, e-mail:
> For additional commands, e-mail:

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message