hadoop-pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Pradeep Kamath (JIRA)" <j...@apache.org>
Subject [jira] Commented: (PIG-428) TypeCastInserter does not replace projects in inner plans correctly
Date Mon, 02 Feb 2009 18:59:59 GMT

    [ https://issues.apache.org/jira/browse/PIG-428?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12669687#action_12669687
] 

Pradeep Kamath commented on PIG-428:
------------------------------------

Have you tried your query with top of trunk? The original issue fixed in this issue was when
the TypeCastInserter was involved in the query. That is the case only when the load statement
has a schema like " a = load 'bla' as (x:int, y:float);". In your query in the previous comment
the load statement does not have a schema. I am wondering if the issues is somewhere else
in that query but the error message is the same. 

> TypeCastInserter does not replace projects in inner plans correctly
> -------------------------------------------------------------------
>
>                 Key: PIG-428
>                 URL: https://issues.apache.org/jira/browse/PIG-428
>             Project: Pig
>          Issue Type: Bug
>    Affects Versions: types_branch
>            Reporter: Pradeep Kamath
>             Fix For: types_branch
>
>         Attachments: PIG-428.patch
>
>
> The TypeCastInserter tries to replace the Project's input operator in inner plans with
the new foreach operator it adds. However it should replace only those Projects' input where
the new Foreach has been added after the operator which was earlier the input to Project.
> Here is a query which fails due to this:
> {code}
> a = load 'st10k' as (name:chararray,age:int, gpa:double);
> another = load 'st10k';
> c = foreach another generate $0, $1+ 10, $2 + 10;
> d = join a by $0, c by $0;
> dump d;
> {code}
> Here is the error:
> {noformat}
> 2008-09-11 23:34:28,169 [main] ERROR org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.Launcher
- Error message from task (map) tip_200809051428_0045_m_000000java.io.IOException: Type mismatch
in key from map: expected org.apache.pig.impl.io.NullableText, recieved org.apache.pig.impl.io.NullableBytesWritable
>         at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.collect(MapTask.java:419)
>         at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Map.collect(PigMapReduce.java:83)
>         at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.runPipeline(PigMapBase.java:172)
>         at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:158)
>         at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Map.map(PigMapReduce.java:75)
>         at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
>         at org.apache.hadoop.mapred.MapTask.run(MapTask.java:219)
>         at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2124)
> {noformat}

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message