pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Santhosh Srinivasan (JIRA)" <j...@apache.org>
Subject [jira] Commented: (PIG-552) UDF defined with argument causes class instantiation exception
Date Thu, 04 Dec 2008 19:38:44 GMT

    [ https://issues.apache.org/jira/browse/PIG-552?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12653429#action_12653429
] 

Santhosh Srinivasan commented on PIG-552:
-----------------------------------------

Sorry about the reference to ComparisonFunc, I was looking at your issue - PIG-549 and had
that in mind.

I was not able to reproduce your scenario. I tried the following script and it worked.

{code}

grunt> define mapUdf MapUDF('world');                                                 
         
grunt> RAW_LOGS = load '/user/sms/data/mydata.txt' as (url:chararray, numvisits:int); 
         
grunt> b = foreach RAW_LOGS generate mapUdf(*);         

{code}

> UDF defined with argument causes class instantiation exception
> --------------------------------------------------------------
>
>                 Key: PIG-552
>                 URL: https://issues.apache.org/jira/browse/PIG-552
>             Project: Pig
>          Issue Type: Bug
>    Affects Versions: types_branch
>            Reporter: Christopher Olston
>         Attachments: pig.patch
>
>
> I'm doing:
> define myFunc myFunc('blah');
> b = foreach a generate myFunc(*);
> Pig parses it, but fails when it tries to run it on hadoop (I'm using "local" mode).
It tries to invoke the class loader on "myFunc('blah')" instead of on "myFunc", which causes
an exception.
> The bug seems to stem from this part of JobControlCompiler.getJobConf():
>                 if(mro.UDFs.size()==1){
>                     String compFuncSpec = mro.UDFs.get(0);
>                     Class comparator = PigContext.resolveClassName(compFuncSpec);
>                     if(ComparisonFunc.class.isAssignableFrom(comparator)) {
>                         jobConf.setMapperClass(PigMapReduce.MapWithComparator.class);
>                         jobConf.setReducerClass(PigMapReduce.ReduceWithComparator.class);
>                         jobConf.set("pig.reduce.package", ObjectSerializer.serialize(pack));
>                         jobConf.set("pig.usercomparator", "true");
>                         jobConf.setOutputKeyClass(NullableTuple.class);
>                         jobConf.setOutputKeyComparatorClass(comparator);
>                     }
>                 } else {
>                     jobConf.set("pig.sortOrder",
>                         ObjectSerializer.serialize(mro.getSortOrder()));
>                 }

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message