Mailing-List: contact java-dev-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: java-dev@lucene.apache.org
Received-SPF: pass (athena.apache.org: local policy)
Message-Id: <1194950219.29766.1221018253@webmail.messagingengine.com>
From: "Michael McCandless" <lucene@mikemccandless.com>
To: java-dev@lucene.apache.org
Content-Disposition: inline
Content-Transfer-Encoding: 7bit
Content-Type: text/plain; charset="ISO-8859-1"
MIME-Version: 1.0
References: <11500990.1194903350969.JavaMail.jira@brutus>
   <BFD76B8A-0726-4A53-9A1A-2897CC7702A4@ix.netcom.com>
   <c68e39170711121447w7619678fk35c02cc39b7f604b@mail.gmail.com>
   <AABA44C5-B4AE-4C05-A373-ABC2AADAE256@ix.netcom.com>
   <1194909297.25216.1220941175@webmail.messagingengine.com>
   <CFF97F18-805D-47FD-B15D-E5ED69A044E6@ix.netcom.com>
   <4738E4EB.9000102@gmail.com>
   <94FDD5BE-9A78-4D08-9374-177D88776798@ix.netcom.com>
   <4738E959.40306@gmail.com>
   <536D36A8-64C1-4AC4-9F75-43396759FF79@ix.netcom.com>
   <c68e39170711121721m1a3f05c8r40d083e9a0287a6f@mail.gmail.com>
   <079234A8-00A3-48D9-A942-0BE506A9C8F6@ix.netcom.com>
Subject: Re: [jira] Commented: (LUCENE-743) IndexReader.reopen()
In-Reply-To: <079234A8-00A3-48D9-A942-0BE506A9C8F6@ix.netcom.com>
Date: Tue, 13 Nov 2007 05:36:59 -0500


I've become lost in this thread.

It started with the unit test in the reopen patch, suddenly jumped to
NFS and lockless commits, then sync() got pulled in as well.

The retry logic (part of lockless commits) already handles this case
of a segments file being partially written & falling back to the
previous one.

Robert what problem are you trying to solve here?

Mike

"robert engels" <rengels@ix.netcom.com> wrote:
> True. It seems that the Lucene code might be a bit more resilient  
> here though, using the following:
> 
> 1. open the segments file exclusively (if this fails, updates are  
> prohibited, and an exception is thrown)
> 2. write new segments
> 3. write segments.new including segments hash & sync
> 4. update segments file including hash
> 5. delete segments that you can
> 
> Then if it crashes in step 4, it is easy to know segments is bad (out  
> of date) and use segments.new
> If it crashes in steps 3, then segments.new is easily detected as  
> being corrupt (hash does not match), so you know segments is valid.
> 
> if there are segments that cannot be deleted in 5, every open can  
> check if it can delete them...
> 
> A similar technique can be used if using lockless commits, just need  
> to make it segments.XXX.new, etc.
> 
> On Nov 12, 2007, at 7:21 PM, Yonik Seeley wrote:
> 
> > On Nov 12, 2007 7:19 PM, robert engels <rengels@ix.netcom.com> wrote:
> >> I would still argue that it is an incorrect setup - almost as bad as
> >> "not plugging the computer in".
> >
> > A user themselves could even go in and look at the index files (I've
> > done so myself)... as could a backup program or whatever.  It's a fact
> > of life on windows that a move or delete can fail.
> >
> > -Yonik
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
> > For additional commands, e-mail: java-dev-help@lucene.apache.org
> >
> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-dev-help@lucene.apache.org
> 

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org