hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Amit Sela <am...@infolinks.com>
Subject Re: Added a key not lexically larger than previous key
Date Sun, 25 May 2014 11:21:13 GMT
The files for bulk load are generated with MR job where output format is
sequence file, key is ImmutableBytesWritable and value is Put.
Bulk load MR map only writes out the key value as they are read by the
Mapper.
Don't have the files anymore... if it'll happen again I'll try to read them
with the hfile tool.
Thanks.


On Sun, May 25, 2014 at 1:41 PM, Stack <stack@duboce.net> wrote:

> On Sun, May 25, 2014 at 2:02 AM, Amit Sela <amits@infolinks.com> wrote:
>
> > I'm running HBase 0.94.12 (Hadoop 1.0.4).
> > I have a daily bulk load job that runs for a long time now, suddenly I
> got
> > this exception:
> >
> > java.io.IOException: Added a key not lexically larger than previous
> >
> >
> key=\x00U20140523]org.cinetux\x09/2013/11/ver-pelicula-charlie-countryman-online-gratis-2013.html\x03gen\x03WMQ\x00\x00\x00\x05\x02MX\x0210\x010\
> > x11www.infolinks.com\x08tagcloud\x00\x00\x01F-\x83\xB0\x88\x04,
> >
> >
> lastkey=\x00R20140523_org.cinetux\x09/2013/11/ver-pelicula-caza-al-asesino-online-gratis-2013.html\x03gen\x03WMQ\x00\x00\x00\x05\x02VE\x03901\x010\x0Cfacebook.com\x08tagcloud\x00\x00\x01F-\x83\xB0\x88\x04
> > at
> >
> >
> org.apache.hadoop.hbase.io.hfile.AbstractHFileWriter.checkKey(AbstractHFileWriter.java:207)
> > ..........
> >
> > I'm running with configureIncrementalLoad() and using
> > ImmutableBytesWritable as key and Put as value (leads to using
> > PutSortReducer).
> >
> > The weird thing is after I re-executed the same job, it worked just fine,
> > no exceptions...
> >
> > While I'm happy it works now, I really don't understand what could go
> wrong
> > there ?
> >
>
> It does indeed look as though keys in the file are not properly ordered.
>  How are you generating the files you are bulk loading?  Do you have the
> file still?  IIRC, reading the file with the hfile tool will fail if it is
> not sorted properly: http://hbase.apache.org/book.html#hfile_tool2
>
> St.Ack
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message