pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Aniket Mokashi (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (PIG-3754) InputSizeReducerEstimator.getTotalInputFileSize reports incorrect size
Date Thu, 06 Mar 2014 21:23:44 GMT

     [ https://issues.apache.org/jira/browse/PIG-3754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Aniket Mokashi updated PIG-3754:
--------------------------------

    Description: 
If you have more than one input, InputSizeReducerEstimator.getTotalInputFileSize can return
incorrect value if one of the loader returns \-1 and is not file based (eg- hbase). This causes
incorrect reducer estimation and problems in auto.local mode.

If size of input is not found in for any of the inputs, we should bail out with return value
of -1.

  was:
If you have more than one input, InputSizeReducerEstimator.getTotalInputFileSize can return
incorrect value if one of the loader returns -1 and is not file based (eg- hbase). This causes
incorrect reducer estimation and problems in auto.local mode.

If size of input is not found in for any of the inputs, we should bail out with return value
of -1.


> InputSizeReducerEstimator.getTotalInputFileSize reports incorrect size
> ----------------------------------------------------------------------
>
>                 Key: PIG-3754
>                 URL: https://issues.apache.org/jira/browse/PIG-3754
>             Project: Pig
>          Issue Type: Bug
>    Affects Versions: 0.13.0
>            Reporter: Aniket Mokashi
>            Assignee: Aniket Mokashi
>            Priority: Trivial
>             Fix For: 0.13.0
>
>
> If you have more than one input, InputSizeReducerEstimator.getTotalInputFileSize can
return incorrect value if one of the loader returns \-1 and is not file based (eg- hbase).
This causes incorrect reducer estimation and problems in auto.local mode.
> If size of input is not found in for any of the inputs, we should bail out with return
value of -1.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message