hadoop-pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alan Gates (JIRA)" <j...@apache.org>
Subject [jira] Commented: (PIG-831) Records and bytes written reported by pig are wrong in a multi-store program
Date Tue, 02 Jun 2009 20:23:07 GMT

    [ https://issues.apache.org/jira/browse/PIG-831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12715657#action_12715657
] 

Alan Gates commented on PIG-831:
--------------------------------

There are a couple of issues going on here.

One, PigStats looks through the plan until it finds the first root and then stops.  So for
multi-store scripts that have multiple roots in their plans, this does not work.

Two, Hadoop does not return accurate numbers for records written in many cases.  I do not
know if this is a bug in hadoop or a bug in the output format pig uses when doing multiple
stores in one job.

> Records and bytes written reported by pig are wrong in a multi-store program
> ----------------------------------------------------------------------------
>
>                 Key: PIG-831
>                 URL: https://issues.apache.org/jira/browse/PIG-831
>             Project: Pig
>          Issue Type: Bug
>          Components: impl
>    Affects Versions: 0.3.0
>            Reporter: Alan Gates
>            Assignee: Alan Gates
>            Priority: Minor
>
> The stats features checked in as part of PIG-626 (reporting the number of records and
bytes written at the end of the query) print wrong values (often but not always 0) when the
pig script being run contains more than 1 store.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message