hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jim Klucar <klu...@gmail.com>
Subject Re: Intermittent TableInputFormat error
Date Mon, 19 May 2014 22:19:09 GMT
So why would intBytes be zero? Is this a normal thing that can happen? If
so, we can inspect the underlying byte array size and bail out ourselves. I
just don't want to mask an underlying problem.

Want me to file a JIRA? I could probably supply a patch too. I've been
around that block with Accumulo before.


On Mon, May 19, 2014 at 6:13 PM, Ted Yu <yuzhihong@gmail.com> wrote:

> Looking at the exception message, bytes field of the KeyValue had 0 length.
> Among the places in KeyValue where bytes field is assigned, I find this
> around line 2419 (trunk code):
>
>     // TODO: perhaps some sanity check is needed here.
>
>     byte [] bytes = new byte[Bytes.toInt(intBytes)];
>
> If bytes has 0 length at time of KeyValue, we should report / bail out
> early.
>
> Cheers
>
>
> On Mon, May 19, 2014 at 2:52 PM, Jim Klucar <klucar@gmail.com> wrote:
>
> > Ted,
> >
> > Here are the mapper Exceptions. The first thing the mapper does is try to
> > create an "EventKey" object and emit it with the value. Code snippet is
> in
> > pastebin.
> >
> > http://pastebin.com/yNyBxsx3
> >
> > One thing we're doing that may be different is that we're continuously
> > bulk-loading data into HBase.
> >
> >
> >
> >
> > On Thu, May 15, 2014 at 11:19 AM, Jim Klucar <klucar@gmail.com> wrote:
> >
> > > Hello,
> > >
> > > We are running map/reduce over HBase 0.96.2-hadoop1 with
> TableInputFormat
> > > and get an intermittent error that we can't track down. It shows up in
> > > different jobs and we can reprocess the same data with no problems. Our
> > > theory is that HBase may be migrating a region to a different server
> > while
> > > the scanner is reading it. Perhaps there is some setting we can enable
> to
> > > prevent this?
> > >
> > > Here's the stack trace we get:
> > > http://pastebin.com/Wh3ZBC0J
> > >
> > > Any insight is appreciated, let me know if you would like me to gather
> > any
> > > other log data.
> > >
> > > Thanks,
> > > Jim
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message