hadoop-pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Santhosh Srinivasan (JIRA)" <j...@apache.org>
Subject [jira] Commented: (PIG-552) UDF defined with argument causes class instantiation exception
Date Wed, 03 Dec 2008 19:27:44 GMT

    [ https://issues.apache.org/jira/browse/PIG-552?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12652915#action_12652915

Santhosh Srinivasan commented on PIG-552:

Question related to the test case reported in the bug report. Can you post the UDF? If not,
can you confirm if the UDF is missing a default constructor?

Review comments:

The patch ignores the problem and tries to proceed. This will lead to runtime issues as the
class will not be instantiated in the backend. This is not what the user wants.

Its probably a bug in the parser where the user defined alias is not getting picked up.

> UDF defined with argument causes class instantiation exception
> --------------------------------------------------------------
>                 Key: PIG-552
>                 URL: https://issues.apache.org/jira/browse/PIG-552
>             Project: Pig
>          Issue Type: Bug
>    Affects Versions: types_branch
>            Reporter: Christopher Olston
>         Attachments: pig.patch
> I'm doing:
> define myFunc myFunc('blah');
> b = foreach a generate myFunc(*);
> Pig parses it, but fails when it tries to run it on hadoop (I'm using "local" mode).
It tries to invoke the class loader on "myFunc('blah')" instead of on "myFunc", which causes
an exception.
> The bug seems to stem from this part of JobControlCompiler.getJobConf():
>                 if(mro.UDFs.size()==1){
>                     String compFuncSpec = mro.UDFs.get(0);
>                     Class comparator = PigContext.resolveClassName(compFuncSpec);
>                     if(ComparisonFunc.class.isAssignableFrom(comparator)) {
>                         jobConf.setMapperClass(PigMapReduce.MapWithComparator.class);
>                         jobConf.setReducerClass(PigMapReduce.ReduceWithComparator.class);
>                         jobConf.set("pig.reduce.package", ObjectSerializer.serialize(pack));
>                         jobConf.set("pig.usercomparator", "true");
>                         jobConf.setOutputKeyClass(NullableTuple.class);
>                         jobConf.setOutputKeyComparatorClass(comparator);
>                     }
>                 } else {
>                     jobConf.set("pig.sortOrder",
>                         ObjectSerializer.serialize(mro.getSortOrder()));
>                 }

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message