hadoop-pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Christopher Olston (JIRA)" <j...@apache.org>
Subject [jira] Created: (PIG-552) UDF defined with argument causes class instantiation exception
Date Tue, 02 Dec 2008 00:49:44 GMT
UDF defined with argument causes class instantiation exception
--------------------------------------------------------------

                 Key: PIG-552
                 URL: https://issues.apache.org/jira/browse/PIG-552
             Project: Pig
          Issue Type: Bug
    Affects Versions: types_branch
            Reporter: Christopher Olston


I'm doing:

define myFunc myFunc('blah');
b = foreach a generate myFunc(*);

Pig parses it, but fails when it tries to run it on hadoop (I'm using "local" mode). It tries
to invoke the class loader on "myFunc('blah')" instead of on "myFunc", which causes an exception.

The bug seems to stem from this part of JobControlCompiler.getJobConf():

                if(mro.UDFs.size()==1){
                    String compFuncSpec = mro.UDFs.get(0);
                    Class comparator = PigContext.resolveClassName(compFuncSpec);
                    if(ComparisonFunc.class.isAssignableFrom(comparator)) {
                        jobConf.setMapperClass(PigMapReduce.MapWithComparator.class);
                        jobConf.setReducerClass(PigMapReduce.ReduceWithComparator.class);
                        jobConf.set("pig.reduce.package", ObjectSerializer.serialize(pack));
                        jobConf.set("pig.usercomparator", "true");
                        jobConf.setOutputKeyClass(NullableTuple.class);
                        jobConf.setOutputKeyComparatorClass(comparator);
                    }
                } else {
                    jobConf.set("pig.sortOrder",
                        ObjectSerializer.serialize(mro.getSortOrder()));
                }


-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message