From dev-return-25197-apmail-pig-dev-archive=pig.apache.org@pig.apache.org Tue Dec 6 22:01:02 2011 Return-Path: X-Original-To: apmail-pig-dev-archive@www.apache.org Delivered-To: apmail-pig-dev-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 6F42E9C2C for ; Tue, 6 Dec 2011 22:01:02 +0000 (UTC) Received: (qmail 81238 invoked by uid 500); 6 Dec 2011 22:01:02 -0000 Delivered-To: apmail-pig-dev-archive@pig.apache.org Received: (qmail 81084 invoked by uid 500); 6 Dec 2011 22:01:02 -0000 Mailing-List: contact dev-help@pig.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@pig.apache.org Delivered-To: mailing list dev@pig.apache.org Received: (qmail 81070 invoked by uid 500); 6 Dec 2011 22:01:02 -0000 Delivered-To: apmail-hadoop-pig-dev@hadoop.apache.org Received: (qmail 81063 invoked by uid 99); 6 Dec 2011 22:01:02 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 06 Dec 2011 22:01:02 +0000 X-ASF-Spam-Status: No, hits=-2001.2 required=5.0 tests=ALL_TRUSTED,RP_MATCHES_RCVD X-Spam-Check-By: apache.org Received: from [140.211.11.116] (HELO hel.zones.apache.org) (140.211.11.116) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 06 Dec 2011 22:01:01 +0000 Received: from hel.zones.apache.org (hel.zones.apache.org [140.211.11.116]) by hel.zones.apache.org (Postfix) with ESMTP id E4E841031ED for ; Tue, 6 Dec 2011 22:00:40 +0000 (UTC) Date: Tue, 6 Dec 2011 22:00:40 +0000 (UTC) From: "Daniel Dai (Updated) (JIRA)" To: pig-dev@hadoop.apache.org Message-ID: <1476267453.47078.1323208840939.JavaMail.tomcat@hel.zones.apache.org> In-Reply-To: <781717203.9238.1322126680134.JavaMail.tomcat@hel.zones.apache.org> Subject: [jira] [Updated] (PIG-2387) BinStorageRecordReader causes negative progress MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/PIG-2387?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Dai updated PIG-2387: ---------------------------- Resolution: Fixed Fix Version/s: 0.11 Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Patch looks good. Unit tests pass. test-patch: [exec] -1 overall. [exec] [exec] +1 @author. The patch does not contain any @author tags. [exec] [exec] -1 tests included. The patch doesn't appear to include any new or modified tests. [exec] Please justify why no tests are needed for this patch. [exec] [exec] +1 javadoc. The javadoc tool did not generate any warning messages. [exec] [exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings. [exec] [exec] +1 findbugs. The patch does not introduce any new Findbugs warnings. [exec] [exec] -1 release audit. The applied patch generated 463 release audit warnings (more than the trunk's current 456 warnings). No tests included, Xuting tested it manually. No new files added, ignore release audit warnings. Patch committed to trunk/0.10/0.9 Thanks Xuting! > BinStorageRecordReader causes negative progress > ----------------------------------------------- > > Key: PIG-2387 > URL: https://issues.apache.org/jira/browse/PIG-2387 > Project: Pig > Issue Type: Bug > Affects Versions: 0.9.0 > Reporter: Anitha Raju > Assignee: xuting zhao > Fix For: 0.10, 0.9.2, 0.11 > > Attachments: PIG-2387.patch > > > Hi, > When an input file of size greater than default split size is loaded using BinStorage() and some processing is done, the task returns negative progress > Script > {code} > A = load 'input' using BinStorage() as (a:chararray); > B = filter A by (a matches '.*blinds.*'); > store B into 'op'; > {code} > Looking at the code, BinStorage which uses BinStorageRecordReader, has getProgress() > {code} > public float getProgress() { > if (start == end) { > return 0.0f; > } else { > return Math.min(1.0f, (pos - start) / (float)(end - start)); > } > } > {code} > In BinStorageRecordReader, pos is always 0 and not getting updated at any point. > So when the input file of size greater than default split size is loaded and processed, the getProgress() method returns negative value, thus showing negative progress. > Regards, > Anitha -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira