hadoop-pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Shravan Matthur Narayanamurthy (JIRA)" <j...@apache.org>
Subject [jira] Updated: (PIG-369) Filter does not allow udf as the filter operator and only allows ComparisonOperators
Date Wed, 13 Aug 2008 12:48:44 GMT

     [ https://issues.apache.org/jira/browse/PIG-369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Shravan Matthur Narayanamurthy updated PIG-369:
-----------------------------------------------

    Comment: was deleted

> Filter does not allow udf as the filter operator and only allows ComparisonOperators
> ------------------------------------------------------------------------------------
>
>                 Key: PIG-369
>                 URL: https://issues.apache.org/jira/browse/PIG-369
>             Project: Pig
>          Issue Type: Bug
>    Affects Versions: types_branch
>            Reporter: Pradeep Kamath
>            Assignee: Shravan Matthur Narayanamurthy
>             Fix For: types_branch
>
>
> The following pig script does not work:
> {code}
> register util.jar;
> define MyFilterSet util.FilterUdf('filter.txt');
> A = load 'simpletest' using PigStorage() as ( x, y );
> B = filter A by MyFilterSet(x);
> dump B;
> {code}
> The following error is seen:
> {noformat}
> java -cp pig.jar:$localc org.apache.pig.Main filter.pig 
> 2008-08-07 17:59:37,663 [main] INFO  org.apache.pig.backend.hadoop.executionengine.HExecutionEngine
- Connecting to hadoop file system at: localhost:9000
> 2008-08-07 17:59:37,748 [main] WARN  org.apache.hadoop.fs.FileSystem - "localhost:9000"
is a deprecated filesystem name. Use "hdfs://localhost:9000/" instead.
> 2008-08-07 17:59:38,035 [main] INFO  org.apache.pig.backend.hadoop.executionengine.HExecutionEngine
- Connecting to map-reduce job tracker at: localhost:9001
> 2008-08-07 17:59:38,166 [main] WARN  org.apache.hadoop.fs.FileSystem - "localhost:9000"
is a deprecated filesystem name. Use "hdfs://localhost:9000/" instead.
> java.io.IOException: Unable to open iterator for alias: B [org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc]
>         at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POFilter.setPlan(POFilter.java:179)
>         at org.apache.pig.backend.hadoop.executionengine.physicalLayer.LogToPhyTranslationVisitor.visit(LogToPhyTranslationVisitor.java:592)
>         at org.apache.pig.impl.logicalLayer.LOFilter.visit(LOFilter.java:102)
>         at org.apache.pig.impl.logicalLayer.LOFilter.visit(LOFilter.java:31)
>         at org.apache.pig.impl.plan.DependencyOrderWalker.walk(DependencyOrderWalker.java:68)
>         at org.apache.pig.impl.plan.PlanVisitor.visit(PlanVisitor.java:51)
>         at org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.compile(HExecutionEngine.java:245)
>         at org.apache.pig.PigServer.compilePp(PigServer.java:590)
>         at org.apache.pig.PigServer.execute(PigServer.java:516)
>         at org.apache.pig.PigServer.openIterator(PigServer.java:307)
>         at org.apache.pig.tools.grunt.GruntParser.processDump(GruntParser.java:258)
>         at org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:175)
>         at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:82)
>         at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:64)
>         at org.apache.pig.Main.main(Main.java:302)
> Caused by: java.lang.ClassCastException: org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc
>         ... 15 more
> {noformat}
> I looked further and the issue seems to be in POFilter which only thinks of the filter
operator as a ComparisonOperator and doesn't allow a UDF for filtering:
> {code}
> public void setPlan(PhysicalPlan plan) {
>         this.plan = plan;
>         comOp = (ComparisonOperator) (plan.getLeaves()).get(0);
>         compOperandType = comOp.getOperandType();
>     }
> {code}

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message