hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mark Kerzner <markkerz...@gmail.com>
Subject Re: Type mismatch
Date Fri, 04 Feb 2011 06:57:37 GMT
I am on 0.89 from CDH3
I tried IdentityTableReducer, but get the same error
I will try 0.90. Should I include the HBase code, so that I can step through
it?

On Fri, Feb 4, 2011 at 12:38 AM, Stack <stack@duboce.net> wrote:

> I'm not sure whats up w/ your sample above.  Here's some observations
> that might help.
>
> Here is the code.  Our line numbers differ.  You are not on 0.90.0?
> Thats not important.  You are in this method it seems:
>
> http://hbase.apache.org/xref/org/apache/hadoop/hbase/mapreduce/TableOutputFormat.html#124
>  See the message on the end.  You should submit a patch where we add
> to the IOException message a toString on the value passed so we have a
> better clue as to where we are off here -- so you can see class of
> object submitted (debugging, I'd add this to the log message).
>
> Looking at how you declare TOF, it doesn't look right (This helps with
> that:
> http://hbase.apache.org/xref/org/apache/hadoop/hbase/mapreduce/TableReducer.html
> ).
>  It seems like the declaration should be <KEYIN, VALUEIN, KEYOUT>  but
> you are outputting a Text for KEYOUT, not the declared Put.  This is
> probably not your prob. though.
>
> Looking at IdentityTableReducer, it just passes Writables with the
> value a Delete or Put.
>
> St.Ack
>
>
>
>
>
> On Thu, Feb 3, 2011 at 10:00 PM, Mark Kerzner <markkerzner@gmail.com>
> wrote:
> > Thank you, St.Ack, it is very nice of you to keep helping me. Here is the
> > stack :) trace, but as you can see, it is the internal Hadoop code. I see
> > this code and I see the message - I am not passing it the right object -
> but
> > how DO I pass the right object?
> >
> > M
> >
> >
> >        at
> >
> org.apache.hadoop.hbase.mapreduce.TableOutputFormat$TableRecordWriter.write(TableOutputFormat.java:106)
> >        at
> >
> org.apache.hadoop.hbase.mapreduce.TableOutputFormat$TableRecordWriter.write(TableOutputFormat.java:65)
> >        at
> >
> org.apache.hadoop.mapred.ReduceTask$NewTrackingRecordWriter.write(ReduceTask.java:512)
> >        at
> >
> org.apache.hadoop.mapreduce.TaskInputOutputContext.write(TaskInputOutputContext.java:80)
> >        at org.apache.hadoop.mapreduce.Reducer.reduce(Reducer.java:156)
> >        at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:176)
> >        at
> > org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:570)
> >        at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:412)
> >        at
> > org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:258)
> >
> > On Thu, Feb 3, 2011 at 11:52 PM, Stack <stack@duboce.net> wrote:
> >
> >> Look at the stack trace.  See where its being thrown.  Look at that
> >> src code at that line offset.  Should give you a clue.
> >> St.Ack
> >>
> >> On Thu, Feb 3, 2011 at 9:36 PM, Mark Kerzner <markkerzner@gmail.com>
> >> wrote:
> >> > Thank you, that helped, but now I get this error on trying to write
> back
> >> to
> >> > HBase:
> >> >
> >> > java.io.IOException: Pass a Delete or a Put
> >> >
> >> > Here is a fragment on my code. Again, thanks a bunch!
> >> >
> >> >    public static class RowCounterReducer
> >> >            extends TableReducer <Text, IntWritable, Put>
> >> >    {
> >> >        public void reduce(Text key,
> >> >                Iterable<IntWritable> values,
> >> >                Reducer.Context context)
> >> >                throws IOException,
> >> >                InterruptedException {
> >> >            Iterator <IntWritable> iterator = values.iterator();
> >> >            while (iterator.hasNext()) {
> >> >                IntWritable value = iterator.next();
> >> >                Put put = new Put();
> >> >                context.write(key, put);
> >> >            }
> >> >        }
> >> >    }
> >> >
> >> >
> >> > On Thu, Feb 3, 2011 at 2:50 PM, Stack <stack@duboce.net> wrote:
> >> >
> >> >> You are emitting a Text type.  Try just passing 'row' to the context,
> >> >> the one passed in to your map.
> >> >> St.Ack
> >> >>
> >> >> On Thu, Feb 3, 2011 at 12:23 PM, Mark Kerzner <markkerzner@gmail.com
> >
> >> >> wrote:
> >> >> > Hi,
> >> >> >
> >> >> > I have this code to read and write to HBase from MR, and it works
> fine
> >> >> with
> >> >> > 0 reducers, but it gives a type mismatch error when with 1 reducer.
> >> What
> >> >> > should I look at? *Thank you!*
> >> >> >
> >> >> > *Code:*
> >> >> >
> >> >> >    static class RowCounterMapper
> >> >> >            extends TableMapper<Text, IntWritable> {
> >> >> >
> >> >> >        private static enum Counters {
> >> >> >
> >> >> >            ROWS
> >> >> >        }
> >> >> >
> >> >> >        @Override
> >> >> >        public void map(ImmutableBytesWritable row, Result values,
> >> Context
> >> >> > context)
> >> >> >                throws IOException, InterruptedException {
> >> >> >            for (KeyValue value : values.list()) {
> >> >> >                if (value.getValue().length > 0) {
> >> >> >                    Text key = new Text(value.getValue());
> >> >> >                    context.write(key, ONE);
> >> >> >                }
> >> >> >            }
> >> >> >        }
> >> >> >    }
> >> >> >
> >> >> > *Error: *
> >> >> >
> >> >> > java.io.IOException: Type mismatch in key from map: expected
> >> >> > org.apache.hadoop.hbase.io.ImmutableBytesWritable, recieved
> >> >> > org.apache.hadoop.io.Text
> >> >> >
> >> >>
> >> >
> >>
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message