pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Daniel Dai (Updated) (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (PIG-2387) BinStorageRecordReader causes negative progress
Date Tue, 06 Dec 2011 22:00:40 GMT

     [ https://issues.apache.org/jira/browse/PIG-2387?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Daniel Dai updated PIG-2387:
----------------------------

       Resolution: Fixed
    Fix Version/s: 0.11
     Hadoop Flags: Reviewed
           Status: Resolved  (was: Patch Available)

Patch looks good. 

Unit tests pass. test-patch: 
     [exec] -1 overall.  
     [exec] 
     [exec]     +1 @author.  The patch does not contain any @author tags.
     [exec] 
     [exec]     -1 tests included.  The patch doesn't appear to include any new or modified
tests.
     [exec]                         Please justify why no tests are needed for this patch.
     [exec] 
     [exec]     +1 javadoc.  The javadoc tool did not generate any warning messages.
     [exec] 
     [exec]     +1 javac.  The applied patch does not increase the total number of javac compiler
warnings.
     [exec] 
     [exec]     +1 findbugs.  The patch does not introduce any new Findbugs warnings.
     [exec] 
     [exec]     -1 release audit.  The applied patch generated 463 release audit warnings
(more than the trunk's current 456 warnings).

No tests included, Xuting tested it manually. No new files added, ignore release audit warnings.

Patch committed to trunk/0.10/0.9

Thanks Xuting!
                
> BinStorageRecordReader causes negative progress
> -----------------------------------------------
>
>                 Key: PIG-2387
>                 URL: https://issues.apache.org/jira/browse/PIG-2387
>             Project: Pig
>          Issue Type: Bug
>    Affects Versions: 0.9.0
>            Reporter: Anitha Raju
>            Assignee: xuting zhao
>             Fix For: 0.10, 0.9.2, 0.11
>
>         Attachments: PIG-2387.patch
>
>
> Hi,
> When an input file of size greater than default split size is loaded using BinStorage()
and some processing is done, the task returns negative progress
> Script
> {code}
> A = load 'input' using BinStorage() as (a:chararray);
> B = filter A by (a matches '.*blinds.*');
> store B into 'op';
> {code}
> Looking at the code, BinStorage which uses BinStorageRecordReader, has getProgress()
> {code}
> public float getProgress() {
>     if (start == end) {
>       return 0.0f;
>     } else {
>           return Math.min(1.0f, (pos - start) / (float)(end - start));
>     }
>   }
> {code}
> In BinStorageRecordReader, pos is always 0 and not getting updated at any point.
> So when the input file of size greater than default split size is loaded and processed,
the getProgress() method returns negative value, thus showing negative progress.
> Regards,
> Anitha 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message