hadoop-pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Viraj Bhat (JIRA)" <j...@apache.org>
Subject [jira] Created: (PIG-1278) Type mismatch in key from map: expected org.apache.pig.impl.io.NullableFloatWritable, recieved org.apache.pig.impl.io.NullableText
Date Fri, 05 Mar 2010 22:47:27 GMT
Type mismatch in key from map: expected org.apache.pig.impl.io.NullableFloatWritable, recieved
org.apache.pig.impl.io.NullableText 
-----------------------------------------------------------------------------------------------------------------------------------

                 Key: PIG-1278
                 URL: https://issues.apache.org/jira/browse/PIG-1278
             Project: Pig
          Issue Type: Bug
    Affects Versions: 0.6.0
            Reporter: Viraj Bhat
             Fix For: 0.7.0


I have a script which uses Map data, and runs a UDF, which creates random numbers and then
orders the data by these random numbers.

{code}
REGISTER myloader.jar;
--jar produced from the source code listed below
REGISTER math.jar;

DEFINE generator math.Random();

inputdata = LOAD '/user/viraj/mymapdata'   USING MyMapLoader()    AS (s:map[], m:map[], l:map[]);

queries = FILTER inputdata   BY m#'key'#'query' IS NOT null;

queries_rand = FOREACH queries  GENERATE generator('') AS rand_num, (CHARARRAY) m#'key'#'query'
AS query_string;

queries_sorted = ORDER queries_rand  BY rand_num  PARALLEL 10;

queries_limit = LIMIT queries_sorted 1000;

rand_queries = FOREACH queries_limit  GENERATE query_string;

STORE rand_queries INTO 'finalresult';

{code}

UDF source for Random.java
{code}
package math;

import java.io.IOException;

/*
* Implements a random float [0,1) generator.
*/

public class Random extends EvalFunc<Float>
{
        private final Random m_rand = new Random();

       public Float exec(Tuple input) throws IOException
        {
               return new Float(m_rand.nextFloat());
        }

        public Schema outputSchema(Schema input)
        {
               final String name = getSchemaName(getClass().getName(), input);
               return new Schema(new Schema.FieldSchema(name, DataType.FLOAT));
        }
}
{code}

Running this script returns the following error in the Mapper
=============================================
java.io.IOException: Type mismatch in key from map: expected org.apache.pig.impl.io.NullableFloatWritable,
recieved org.apache.pig.impl.io.NullableText at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.collect(MapTask.java:845)
at org.apache.hadoop.mapred.MapTask$OldOutputCollector.collect(MapTask.java:466) at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Map.collect(PigMapReduce.java:109)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.runPipeline(PigMapBase.java:255)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:244)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Map.map(PigMapReduce.java:94)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50) at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:358)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307) at org.apache.hadoop.mapred.Child.main(Child.java:159)

=============================================

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message