pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mohit Sabharwal (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (PIG-4788) the value BytesRead metric info always returns 0 even the length of input file is not 0 in spark engine
Date Tue, 01 Mar 2016 03:56:18 GMT

    [ https://issues.apache.org/jira/browse/PIG-4788?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15173194#comment-15173194
] 

Mohit Sabharwal commented on PIG-4788:
--------------------------------------

Custom LoadFunc for example can be written to return any inputFormat , no ? https://pig.apache.org/docs/r0.12.0/api/org/apache/pig/LoadFunc.html

> the value BytesRead metric info always returns 0 even the length of input file is not
0 in spark engine
> -------------------------------------------------------------------------------------------------------
>
>                 Key: PIG-4788
>                 URL: https://issues.apache.org/jira/browse/PIG-4788
>             Project: Pig
>          Issue Type: Sub-task
>          Components: spark
>            Reporter: liyunzhang_intel
>            Assignee: liyunzhang_intel
>             Fix For: spark-branch
>
>         Attachments: PIG-4788.patch
>
>
> In [JobMetricsLinstener#onTaskEnd|https://github.com/apache/pig/blob/spark/src/org/apache/pig/tools/pigstats/spark/SparkJobStats.java#L140],
taskMetrics.inputMetrics().get().bytesRead() always returns 0 even the length of input file
is not zero.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message