pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vivek Padmanabhan (Updated) (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (PIG-2576) Change in behavior for UDFContext.getUDFContext().getJobConf() in front-end
Date Fri, 09 Mar 2012 09:23:57 GMT

     [ https://issues.apache.org/jira/browse/PIG-2576?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Vivek Padmanabhan updated PIG-2576:
-----------------------------------

    Attachment: PIG-2576_Script_UDF.txt

Attaching a sample script, udf and exception trace to illustrate the issue.
Looks like the change came in as part of PIG-2532 in which Pig MR Launcher will set the current
UDFContext before the MR jobs are
launched. 
Hence after the job launch , the UDFContext.getUDFContext().getJobConf() will
always be a non null value.
                
> Change in behavior for UDFContext.getUDFContext().getJobConf() in front-end
> ---------------------------------------------------------------------------
>
>                 Key: PIG-2576
>                 URL: https://issues.apache.org/jira/browse/PIG-2576
>             Project: Pig
>          Issue Type: Bug
>    Affects Versions: 0.9.3
>            Reporter: Vivek Padmanabhan
>         Attachments: PIG-2576_Script_UDF.txt
>
>
> We read a file in the UDF constructor. (The file is transferred to the compute nodes
via distache)
> To avoid this case in the front-end while the script is in the compile stage,
> we differentiate between front end and back end execution depending upon a condition
( UDFContext.getUDFContext().getJobConf() == null )
> This was working till Pig 0.9.1, in the current Pig 0.9 version this is breaking.
> ie, If I have any 'fs' commands after the STORE statement, the GruntParser invokes the
udf constructor again and the above condition check returns false causing errors.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message