hadoop-pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Olga Natkovich (JIRA)" <j...@apache.org>
Subject [jira] Resolved: (PIG-919) Type mismatch in key from map: expected org.apache.pig.impl.io.NullableBytesWritable, recieved org.apache.pig.impl.io.NullableText when doing simple group
Date Thu, 27 Aug 2009 23:36:59 GMT

     [ https://issues.apache.org/jira/browse/PIG-919?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Olga Natkovich resolved PIG-919.
--------------------------------

      Resolution: Fixed
    Release Note: Jeff is right - the issue is in how maps are interpreted by Pig. I agree
we need to figure out how best to work with maps but it is a bigger issue than the scope of
this bug and I would like to deal with it separately. (In fact it is on the roadmap to address.)

> Type mismatch in key from map: expected org.apache.pig.impl.io.NullableBytesWritable,
recieved org.apache.pig.impl.io.NullableText when doing simple group
> ----------------------------------------------------------------------------------------------------------------------------------------------------------
>
>                 Key: PIG-919
>                 URL: https://issues.apache.org/jira/browse/PIG-919
>             Project: Pig
>          Issue Type: Bug
>          Components: impl
>    Affects Versions: 0.3.0
>            Reporter: Viraj Bhat
>             Fix For: 0.3.0
>
>         Attachments: GenHashList.java, mapscript.pig, mymapudf.jar
>
>
> I have a Pig script, which takes in a student file and generates a bag of maps.  I later
want to group on the value of the key "name0" which corresponds to the first name of the student.
> {code}
> register mymapudf.jar;
> data = LOAD '/user/viraj/studenttab10k' AS (somename:chararray,age:long,marks:float);
> genmap = foreach data generate flatten(mymapudf.GenHashList(somename,' ')) as bp:map[],
age, marks;
> getfirstnames = foreach genmap generate bp#'name0' as firstname, age, marks;
> filternonnullfirstnames = filter getfirstnames by firstname is not null;
> groupgenmap = group filternonnullfirstnames by firstname;
> dump groupgenmap;
> {code}
> When I execute this code, I get an error in the Map Phase:
> ===========================================================================================================
> java.io.IOException: Type mismatch in key from map: expected org.apache.pig.impl.io.NullableBytesWritable,
recieved org.apache.pig.impl.io.NullableText
> 	at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.collect(MapTask.java:415)
> 	at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Map.collect(PigMapReduce.java:108)
> 	at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.runPipeline(PigMapBase.java:253)
> 	at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:242)
> 	at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Map.map(PigMapReduce.java:93)
> 	at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
> 	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:227)
> 	at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2209)
> ===========================================================================================================

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message