pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vivek Padmanabhan (Updated) (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (PIG-2576) Change in behavior for UDFContext.getUDFContext().getJobConf() in front-end
Date Fri, 09 Mar 2012 09:23:57 GMT

     [ https://issues.apache.org/jira/browse/PIG-2576?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Vivek Padmanabhan updated PIG-2576:

    Attachment: PIG-2576_Script_UDF.txt

Attaching a sample script, udf and exception trace to illustrate the issue.
Looks like the change came in as part of PIG-2532 in which Pig MR Launcher will set the current
UDFContext before the MR jobs are
Hence after the job launch , the UDFContext.getUDFContext().getJobConf() will
always be a non null value.
> Change in behavior for UDFContext.getUDFContext().getJobConf() in front-end
> ---------------------------------------------------------------------------
>                 Key: PIG-2576
>                 URL: https://issues.apache.org/jira/browse/PIG-2576
>             Project: Pig
>          Issue Type: Bug
>    Affects Versions: 0.9.3
>            Reporter: Vivek Padmanabhan
>         Attachments: PIG-2576_Script_UDF.txt
> We read a file in the UDF constructor. (The file is transferred to the compute nodes
via distache)
> To avoid this case in the front-end while the script is in the compile stage,
> we differentiate between front end and back end execution depending upon a condition
( UDFContext.getUDFContext().getJobConf() == null )
> This was working till Pig 0.9.1, in the current Pig 0.9 version this is breaking.
> ie, If I have any 'fs' commands after the STORE statement, the GruntParser invokes the
udf constructor again and the above condition check returns false causing errors.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message