From dev-return-82754-apmail-lucene-dev-archive=lucene.apache.org@lucene.apache.org Fri Nov 4 20:08:14 2011 Return-Path: X-Original-To: apmail-lucene-dev-archive@www.apache.org Delivered-To: apmail-lucene-dev-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 701359987 for ; Fri, 4 Nov 2011 20:08:14 +0000 (UTC) Received: (qmail 33504 invoked by uid 500); 4 Nov 2011 20:08:13 -0000 Delivered-To: apmail-lucene-dev-archive@lucene.apache.org Received: (qmail 33458 invoked by uid 500); 4 Nov 2011 20:08:13 -0000 Mailing-List: contact dev-help@lucene.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@lucene.apache.org Delivered-To: mailing list dev@lucene.apache.org Received: (qmail 33451 invoked by uid 99); 4 Nov 2011 20:08:13 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 04 Nov 2011 20:08:13 +0000 X-ASF-Spam-Status: No, hits=-2001.2 required=5.0 tests=ALL_TRUSTED,RP_MATCHES_RCVD X-Spam-Check-By: apache.org Received: from [140.211.11.116] (HELO hel.zones.apache.org) (140.211.11.116) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 04 Nov 2011 20:08:12 +0000 Received: from hel.zones.apache.org (hel.zones.apache.org [140.211.11.116]) by hel.zones.apache.org (Postfix) with ESMTP id E91AE30C473 for ; Fri, 4 Nov 2011 20:07:51 +0000 (UTC) Date: Fri, 4 Nov 2011 20:07:51 +0000 (UTC) From: "Simon Willnauer (Commented) (JIRA)" To: dev@lucene.apache.org Message-ID: <1752949157.899.1320437271956.JavaMail.tomcat@hel.zones.apache.org> In-Reply-To: <2023585747.47087.1320176492590.JavaMail.tomcat@hel.zones.apache.org> Subject: [jira] [Commented] (LUCENE-3551) Yet another race in IW#nrtIsCurrent MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/LUCENE-3551?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13144290#comment-13144290 ] Simon Willnauer commented on LUCENE-3551: ----------------------------------------- This seems to fix SOLR-2861 I went through the changes again double checking all conditions. It seems ready, I will commit soon if nobody objects. > Yet another race in IW#nrtIsCurrent > ----------------------------------- > > Key: LUCENE-3551 > URL: https://issues.apache.org/jira/browse/LUCENE-3551 > Project: Lucene - Java > Issue Type: Bug > Components: core/index > Affects Versions: 4.0 > Reporter: Simon Willnauer > Fix For: 4.0 > > Attachments: LUCENE-3551.patch, LUCENE-3551.patch, LUCENE-3551.patch, LUCENE-3551.patch > > > In IW#nrtIsCurrent looks like this: > {code} > synchronized boolean nrtIsCurrent(SegmentInfos infos) { > ensureOpen(); > return infos.version == segmentInfos.version && !docWriter.anyChanges() && !bufferedDeletesStream.any(); > } > {code} > * the version changes once we checkpoint the IW > * docWriter has changes if there are any docs in ram or any deletes in the delQueue > * bufferedDeletes contain all frozen del packages from the delQueue > yet, what happens is 1. we decrement the numDocsInRam in DWPT#doAfterFlush (which is executed during DWPT#flush) but before we checkpoint. 2. if we freeze deletes (empty the delQueue) we put them in the flushQueue to maintain the order. This means they are not yet in the bufferedDeleteStream. > Bottom line, there is a window where we could see IW#nrtIsCurrent returning true if we check within this particular window. Phew, I am not 100% sure if that is the reason for our latest failure in SOLR-2861 but from what the logs look like this could be what happens. If we randomly hit low values for maxBufferedDocs & maxBufferedDeleteTerms this is absolutely possible. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org For additional commands, e-mail: dev-help@lucene.apache.org