pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vivek Padmanabhan (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (PIG-2404) NullPointerException when I have multiple python udfs
Date Wed, 07 Dec 2011 12:20:40 GMT

    [ https://issues.apache.org/jira/browse/PIG-2404?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13164350#comment-13164350
] 

Vivek Padmanabhan commented on PIG-2404:
----------------------------------------

I guess, this issue is happening since PythonInterpreter returns all the functions from all
the registered python files. 
But the ScriptEngine registers all of them under the last namespace received to it.
This is causing the duplicate log, and also leads to incorrect function definitions or incorrect
namespace to function name mapping.

Since this is a very common scenario, it would be nice to have a fix for this in 0.9.
                
> NullPointerException when I have multiple python udfs
> -----------------------------------------------------
>
>                 Key: PIG-2404
>                 URL: https://issues.apache.org/jira/browse/PIG-2404
>             Project: Pig
>          Issue Type: Bug
>    Affects Versions: 0.8.1, 0.9.1
>            Reporter: Vivek Padmanabhan
>
> When I have multiple python udfs registered, the script fails at compile phase while
trying to get the udf ouputschema.
> {code}
> register 'a.py' using org.apache.pig.scripting.jython.JythonScriptEngine as a_func;
> register 'b.py' using org.apache.pig.scripting.jython.JythonScriptEngine as b_func;
> a = load 'i1' as (f1:chararray);
> b = foreach a generate a_func.helloworld(), b_func.square(3);
> dump b;
> {code}
> a.py 
> {code}
> @outputSchema("word:chararray")
> def helloworld():  
>   return 'Hello, World'
> {code}
> b.py 
> {code}
> @outputSchemaFunction("squareSchema")
> def square(num):
>   return ((num)*(num))
> {code}
> Moreover , in the log we can see duplicate and incorrect registration of udfs which I
believe the cause for the script failure.
> INFO  org.apache.pig.scripting.jython.JythonScriptEngine - Register scripting UDF: a_func.helloworld
> INFO  org.apache.pig.scripting.jython.JythonScriptEngine - Register scripting UDF: b_func.square
> INFO  org.apache.pig.scripting.jython.JythonScriptEngine - Register scripting UDF: b_func.helloworld
> This issue is observed in 0.9,0.8 and  in trunk also.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message