lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jason Rutherglen <jason.rutherg...@gmail.com>
Subject Re: Recreate segment infos
Date Tue, 05 Oct 2010 12:38:27 GMT
I'm not sure how it vanished, I think it was on a Solr rsync based
replication operation, and a lack of disk space.  Need to move to the
Java replication and get larger SSD drives, working on both, at least
they're SSDs, making some progress.  I was going to recover using the
IDs in the terms dict however there should be 130 mil and there were
only 16 mil.  So even if I had a way to recover, the index is far too
incomplete.  This is where re-indexing in Hadoop is coming in handy.

On Tue, Oct 5, 2010 at 3:26 AM, Michael McCandless
<lucene@mikemccandless.com> wrote:
> How did you lose your segments file...?
>
> This was discussed before but I don't think the idea ever turned into a tool.
>
> I think it should be possible.  You'd have to sort all files, deriving
> segment names from the prefixes.  Then, you have to reconstruct the
> metadata required for SegmentInfo.  EG open the fdx file to get
> numDocs, the .del file to get delCount, check for prx file to set
> .haxProx, etc.
>
> You'd have to carefully map segment -> doc store segment.  Multiple
> segments in a row may share the same docStore segment.  In this case
> the docStore segment is given the same name as the first segment that
> shares it.  However, unfortunately, because of merging, it's possible
> that this mapping is not easy (maybe not possible, depending on the
> merge policy...) to reconstruct.  I think this'll be the hardest part
> :)
>
> Mike
>
> On Mon, Oct 4, 2010 at 3:25 PM, Jason Rutherglen
> <jason.rutherglen@gmail.com> wrote:
>> Lets say the segment infos file is missing, and I'm aware of
>> CheckIndex, however is there a tool to recreate a segment infos file?
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message