hadoop-pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Richard Ding (JIRA)" <j...@apache.org>
Subject [jira] Commented: (PIG-502) Limit and Illustrate do not work together
Date Wed, 06 Jan 2010 19:26:54 GMT

    [ https://issues.apache.org/jira/browse/PIG-502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12797260#action_12797260
] 

Richard Ding commented on PIG-502:
----------------------------------

As of now, illustrate also doesn't support operators _CROSS_, _DISTINCT_, and _STREAM_. We
need to look into these (and above) operators and see if we can make illustrate to work with
them.

> Limit and Illustrate do not work together
> -----------------------------------------
>
>                 Key: PIG-502
>                 URL: https://issues.apache.org/jira/browse/PIG-502
>             Project: Pig
>          Issue Type: Improvement
>          Components: tools
>    Affects Versions: 0.2.0
>         Environment: Hadoop 18
>            Reporter: Viraj Bhat
>
> Suppose a user wants to do an illustrate command after limiting his data to a certain
number of records, it does not seem to work..
> --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
> {code}
> MYDATA = load 'testfilelarge.txt' as (f1, f2, f3, f4, f5);
> MYDATA  = limit MYDATA 10;
> describe MYDATA;
> illustrate MYDATA;
> {code}
> --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
> Running this script produces the following output and error
> --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
> MYDATA: {f1: bytearray,f2: bytearray,f3: bytearray,f4: bytearray,f5: bytearray}
> 2008-10-18 02:14:26,900 [main] INFO  org.apache.pig.backend.hadoop.executionengine.HExecutionEngine
- Connecting to hadoop fil
> e system at: hdfs://localhost:9000
> 2008-10-18 02:14:27,013 [main] INFO  org.apache.pig.backend.hadoop.executionengine.HExecutionEngine
- Connecting to map-reduce
>  job tracker at: localhost:9001
> java.lang.RuntimeException: Unrecognized logical operator.
>         at org.apache.pig.pen.EquivalenceClasses.GetEquivalenceClasses(EquivalenceClasses.java:60)
>         at org.apache.pig.pen.DerivedDataVisitor.evaluateOperator(DerivedDataVisitor.java:368)
>         at org.apache.pig.pen.DerivedDataVisitor.visit(DerivedDataVisitor.java:273)
>         at org.apache.pig.impl.logicalLayer.LOLimit.visit(LOLimit.java:71)
>         at org.apache.pig.impl.logicalLayer.LOLimit.visit(LOLimit.java:10)
>         at org.apache.pig.impl.plan.DependencyOrderWalker.walk(DependencyOrderWalker.java:68)
>         at org.apache.pig.impl.plan.PlanVisitor.visit(PlanVisitor.java:51)
>         at org.apache.pig.pen.LineageTrimmingVisitor.init(LineageTrimmingVisitor.java:98)
>         at org.apache.pig.pen.LineageTrimmingVisitor.<init>(LineageTrimmingVisitor.java:90)
>         at org.apache.pig.pen.ExampleGenerator.getExamples(ExampleGenerator.java:106)
>         at org.apache.pig.PigServer.getExamples(PigServer.java:630)
>         at org.apache.pig.tools.grunt.GruntParser.processIllustrate(GruntParser.java:279)
>         at org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:183)
>         at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:84)
>         at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:64)
>         at org.apache.pig.Main.main(Main.java:306)
> --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
> If I remove the illustrate and replace it with "dump MYDATA;"  it works..
> --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
> {code}
> MYDATA = load 'testfilelarge.txt' as (f1, f2, f3, f4, f5);
> MYDATA  = limit MYDATA 10;
> describe MYDATA;
> -- illustrate MYDATA;
> dump MYDATA;
> {code}
> --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message