pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Bill Graham (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (PIG-2870) pigServer.openIterator fails for jobs with no input splits
Date Mon, 13 Aug 2012 22:52:38 GMT

    [ https://issues.apache.org/jira/browse/PIG-2870?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13433697#comment-13433697
] 

Bill Graham commented on PIG-2870:
----------------------------------

There are at least two use cases that need to be met as I see it:

1. A user needs to set a field on the Config object that will be seen when the MR job(s) run.
Care must be taken that one instance doesn't clobber the other with two different setting.
We could wrap the conf object and throw an exception or log a warning in this case.

2. Multiple instances of store StoreFuncs need to be initialized differently and pass the
state from the client StoreFunc to their respective server StoreFunc. This seems like UDFContext
should be used in this case (and fixed as needed if it doesn't work currently). 
                
> pigServer.openIterator fails for jobs with no input splits
> ----------------------------------------------------------
>
>                 Key: PIG-2870
>                 URL: https://issues.apache.org/jira/browse/PIG-2870
>             Project: Pig
>          Issue Type: Bug
>    Affects Versions: 0.11
>            Reporter: Bill Graham
>            Assignee: Bill Graham
>         Attachments: PIG-2870.1.patch
>
>
> Jobs that have valid input data, but 0 input splits (this is the case where indexing
implemented in the {{InputFormat}} might return 0 splits for an aggressive filter) fail when
{{pigServer.openIterator}} is called. This is because {{mapred.output.dir}} isn't set, so
the job succeeds without creating the empty output directory. The {{ReadToEndLoader}} then
fails due to the null input directory.
> It seems PIG-2578 introduced this issue. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message