lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Michael McCandless (JIRA)" <j...@apache.org>
Subject [jira] Commented: (LUCENE-702) Disk full during addIndexes(Directory[]) can corrupt index
Date Wed, 01 Nov 2006 21:02:17 GMT
    [ http://issues.apache.org/jira/browse/LUCENE-702?page=comments#action_12446376 ] 
            
Michael McCandless commented on LUCENE-702:
-------------------------------------------


That seems like a reasonable approach?  At least the index would be
consistent (ie, loadable).

Though, if you hit disk full part way through, then some of your
indexes were added and some where not.  How do you "recover" after you
free up your disk space?  If you just re-add all indexes then you have
duplicates in the index.

Is it possible instead to not reference (in the written segments info
file) those segments that were carried over from the input
readers/directories?  This would make the operation transactional, so
that if we crashed part way through, then the index rolls back to
where it was before the addIndexes call.  And user could free up disk
space and try again, without creating dups.

(One problem with this is that the orphan'd segments that had been
written would not actually get deleted automatically; lockless commits
would fix that though because it recomputes on instantiating a reader
which files are unreferenced and then deletes them).


> Disk full during addIndexes(Directory[]) can corrupt index
> ----------------------------------------------------------
>
>                 Key: LUCENE-702
>                 URL: http://issues.apache.org/jira/browse/LUCENE-702
>             Project: Lucene - Java
>          Issue Type: Bug
>          Components: Index
>    Affects Versions: 2.1
>            Reporter: Michael McCandless
>         Assigned To: Michael McCandless
>
> This is a spinoff of LUCENE-555
> If the disk fills up during this call then the committed segments file can reference
segments that were not written.  Then the whole index becomes unusable.
> Does anyone know of any other cases where disk full could corrupt the index?
> I think disk full should worse lose the documents that were "in flight" at the time.
 It shouldn't corrupt the index.

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org


Mime
View raw message