hadoop-pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alan Gates (JIRA)" <j...@apache.org>
Subject [jira] Commented: (PIG-919) Type mismatch in key from map: expected org.apache.pig.impl.io.NullableBytesWritable, recieved org.apache.pig.impl.io.NullableText when doing simple group
Date Thu, 10 Dec 2009 23:24:18 GMT

    [ https://issues.apache.org/jira/browse/PIG-919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12789016#action_12789016
] 

Alan Gates commented on PIG-919:
--------------------------------

This was closed Fixed when it should have been closed Won't Fix or Later.  No patch was checked
in for it.  Where are you seeing this error?  In general Pig Latin does not allow users to
declare the type of map values, so you have to explicitly cast map values if you want to group
by them so that Pig knows how to set the key in the MR job.

> Type mismatch in key from map: expected org.apache.pig.impl.io.NullableBytesWritable,
recieved org.apache.pig.impl.io.NullableText when doing simple group
> ----------------------------------------------------------------------------------------------------------------------------------------------------------
>
>                 Key: PIG-919
>                 URL: https://issues.apache.org/jira/browse/PIG-919
>             Project: Pig
>          Issue Type: Bug
>          Components: impl
>    Affects Versions: 0.3.0
>            Reporter: Viraj Bhat
>            Assignee: Viraj Bhat
>             Fix For: 0.3.0
>
>         Attachments: GenHashList.java, mapscript.pig, mymapudf.jar
>
>
> I have a Pig script, which takes in a student file and generates a bag of maps.  I later
want to group on the value of the key "name0" which corresponds to the first name of the student.
> {code}
> register mymapudf.jar;
> data = LOAD '/user/viraj/studenttab10k' AS (somename:chararray,age:long,marks:float);
> genmap = foreach data generate flatten(mymapudf.GenHashList(somename,' ')) as bp:map[],
age, marks;
> getfirstnames = foreach genmap generate bp#'name0' as firstname, age, marks;
> filternonnullfirstnames = filter getfirstnames by firstname is not null;
> groupgenmap = group filternonnullfirstnames by firstname;
> dump groupgenmap;
> {code}
> When I execute this code, I get an error in the Map Phase:
> ===========================================================================================================
> java.io.IOException: Type mismatch in key from map: expected org.apache.pig.impl.io.NullableBytesWritable,
recieved org.apache.pig.impl.io.NullableText
> 	at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.collect(MapTask.java:415)
> 	at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Map.collect(PigMapReduce.java:108)
> 	at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.runPipeline(PigMapBase.java:253)
> 	at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:242)
> 	at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Map.map(PigMapReduce.java:93)
> 	at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
> 	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:227)
> 	at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2209)
> ===========================================================================================================

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message