Return-Path: X-Original-To: apmail-streams-dev-archive@minotaur.apache.org Delivered-To: apmail-streams-dev-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 24BE1118AB for ; Sat, 17 May 2014 02:31:19 +0000 (UTC) Received: (qmail 87804 invoked by uid 500); 17 May 2014 02:12:02 -0000 Delivered-To: apmail-streams-dev-archive@streams.apache.org Received: (qmail 86081 invoked by uid 500); 17 May 2014 01:57:01 -0000 Mailing-List: contact dev-help@streams.incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@streams.incubator.apache.org Delivered-To: mailing list dev@streams.incubator.apache.org Received: (qmail 84227 invoked by uid 99); 17 May 2014 01:54:08 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 17 May 2014 01:54:08 +0000 X-ASF-Spam-Status: No, hits=-0.0 required=5.0 tests=SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of mhager@w2odigital.com designates 74.201.97.205 as permitted sender) Received: from [74.201.97.205] (HELO mxout.myoutlookonline.com) (74.201.97.205) by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 17 May 2014 01:44:22 +0000 Received: from mxout.myoutlookonline.com (localhost [127.0.0.1]) by mxout.myoutlookonline.com (Postfix) with ESMTP id 1B27555758E for ; Fri, 16 May 2014 21:43:14 -0400 (EDT) X-Virus-Scanned: by SpamTitan at mail.lan Received: from HUB022.mail.lan (unknown [10.110.2.1]) by mxout.myoutlookonline.com (Postfix) with ESMTP id 7D834557587 for ; Fri, 16 May 2014 21:43:13 -0400 (EDT) Received: from CCRD014.mail.lan ([192.168.100.55]) by HUB022.mail.lan ([10.110.17.22]) with mapi; Fri, 16 May 2014 21:43:25 -0400 From: "Matthew Hager [W2O Digital]" To: "dev@streams.incubator.apache.org" Date: Fri, 16 May 2014 21:43:20 -0400 Subject: Re: Bug in StreamsPersistWriterTask and StreamsProcessorTask. BREAKS LOCAL RUNTIME Thread-Topic: Bug in StreamsPersistWriterTask and StreamsProcessorTask. BREAKS LOCAL RUNTIME Thread-Index: Ac9xcWSkgOtfLmeLS5OpLSFDqzIa0w== Message-ID: References: In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: user-agent: Microsoft-MacOutlook/14.4.1.140326 acceptlanguage: en-US Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-Virus-Checked: Checked by ClamAV on apache.org If it pleases the community, I have actually refactored a large portion of this for readability and added a series of tests to ensure that it performs under different conditions. I should have a PR for your review shortly. Thanks! Matthew On 5/15/14, 10:19 AM, "Ryan Ebanks" wrote: >Actually it should not be, !this.inQueue.isEmpty(). It should be, datum >!=3D >null > > >On Wed, May 14, 2014 at 5:19 PM, Ryan Ebanks wrote: > >> I can't seem to log into Jira at the moment create the bug ticket. But >> this bug breaks the local runtime. >> >> A recent commit seems to have introduced this. Its a simple fix, but >>the >> while statement on line 70 of StreamsProviderTask should read >> while(this.keepRunning.get() || !this.inQueue.isEmpty()) { >> >> And line 75 in StreamsPersist writer task should read >> while(this.keepRunning.get() || !this.inQueue.isEmpty()) { >> >> >> Both lines are missing the !this.inQueue.isEmpty() statement. This >>causes >> tasks to be shutdown before data is processed and data is orphaned in >>the >> stream. In the worst case only the provider runs. >> >> -Ryan Ebanks