pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Julien Le Dem (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (PIG-2693) LoadFunc.setLocation should be called before LoadMetadata.getStatistics
Date Wed, 16 May 2012 23:41:07 GMT

    [ https://issues.apache.org/jira/browse/PIG-2693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13277267#comment-13277267
] 

Julien Le Dem commented on PIG-2693:
------------------------------------

+1 this looks good to me.
Dmitriy said he will re run the check for the reducer estimator to make sure there's no unexpected
side effect.

                
> LoadFunc.setLocation should be called before LoadMetadata.getStatistics 
> ------------------------------------------------------------------------
>
>                 Key: PIG-2693
>                 URL: https://issues.apache.org/jira/browse/PIG-2693
>             Project: Pig
>          Issue Type: Bug
>            Reporter: Bill Graham
>            Assignee: Bill Graham
>         Attachments: PIG-2693.1.patch, PIG-2693.2.patch
>
>
> We ran into a bug with Pig/HCatalog integration on the trunk. The issue is that {{JobControlCompiler}}
calls the {{adjustNumReducers}} method just before it calls {{setLocation}} on all of the
{{LoadFuncs}}. This causes problems, since some loaders (i.e.,  {{HCatLoader}}) need {{setLocation}}
to be called before it can respond to {{getStatistics}} with it's data size.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message