Mailing-List: contact dev-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@lucene.apache.org
Date: Tue, 7 Jun 2016 16:16:21 +0000 (UTC)
From: "Noble Paul (JIRA)" <jira@apache.org>
To: dev@lucene.apache.org
Message-ID: <JIRA.12976326.1465284124000.15840.1465316181116@Atlassian.JIRA>
In-Reply-To: <JIRA.12976326.1465284124000@Atlassian.JIRA>
References: <JIRA.12976326.1465284124000@Atlassian.JIRA> <JIRA.12976326.1465284124403@arcas>
Subject: [jira] [Commented] (SOLR-9191) OverseerTaskQueue.peekTopN() fatally
 flawed
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
archived-at: Tue, 07 Jun 2016 16:16:23 -0000


    [ https://issues.apache.org/jira/browse/SOLR-9191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15318782#comment-15318782 ] 

Noble Paul commented on SOLR-9191:
----------------------------------

Is there a way we can avoid reading the "already read"  items from ZK?

> OverseerTaskQueue.peekTopN() fatally flawed
> -------------------------------------------
>
>                 Key: SOLR-9191
>                 URL: https://issues.apache.org/jira/browse/SOLR-9191
>             Project: Solr
>          Issue Type: Bug
>    Affects Versions: 5.4, 5.4.1, 5.5, 5.5.1, 6.0, 6.0.1
>            Reporter: Scott Blum
>            Assignee: Scott Blum
>   Original Estimate: 24h
>  Remaining Estimate: 24h
>
> We rewrote DistributedQueue in SOLR-6760, to optimize its obvious use case as a FIFO.  But in doing so, we broke the assumptions in OverseerTaskQueue.peekTopN()..
> OverseerTaskQueue.peekTopN() involves filtering out items you're already working on, it's trying to peek for new items in the queue beyond what you already know about.  But DistributedQueue (being designed as a FIFO) doesn't know about the filtering; as long as it has any items in-memory it just keeps returning those over and over without ever pulling new data from ZK.  This is true even if the watcher has fired and marked the state as dirty.  So OverseerTaskQueue gets into a state where it can never read new items in ZK because DQ keeps returning the same items that it has marked as in-progress.


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org