hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jeff Kolesky <j...@opower.com>
Subject Re: Problem with HFile lexical comparison
Date Thu, 20 Jun 2013 00:23:39 GMT
Last time I wrote directly to an HFile, I instantiated an HFile.Writer
using this statement:

        HFile.Writer writer = HFile.getWriterFactory(config)
            .createWriter(fs, hfilePath,
                    (bytesPerBlock * 1024),
                    Compression.Algorithm.GZ,
                    KeyValue.KEY_COMPARATOR);

Perhaps you need the declaration of the comparator in the create statement
for the writer.

Jeff



On Wed, Jun 19, 2013 at 5:11 PM, Rohit Kelkar <rohitkelkar@gmail.com> wrote:

> Thanks for the replies, I tried the KeyValue.KVComparator but still no
> luck. So I commented the comparator and played around with the sequence of
> writing the qualifiers to the HFile. (see code here:
> https://gist.github.com/anonymous/5819254)
>
> If I set the variable String[] starr = new String[]{"a", "d", "dt", "dth"}
> then the code breaks while writing the qualifier "dt" to the HFile.
> If I set the variable String[] starr = new String[]{"a", "dth", "dt", "d"}
> then the code runs successfully.
> If I set the variable String[] starr = new String[]{"dth", "dt", "d", "a"}
> then the code breaks while writing "a" to the HFile
>
> Does this mean that if the qualifiers start with the same character then
> the longest qualifier should be written first? Else the usual lexical order
> is honoured?
>
> The code throws following stack trace
> Added a key not lexically larger than previous
>
> key=\x00\x1B10011-2-0000000000000000703\x02sddt\x7F\xFF\xFF\xFF\xFF\xFF\xFF\xFF\x04,
>
> lastkey=\x00\x1B10011-2-0000000000000000703\x02sdd\x7F\xFF\xFF\xFF\xFF\xFF\xFF\xFF\x04
>  at
>
> org.apache.hadoop.hbase.io.hfile.AbstractHFileWriter.checkKey(AbstractHFileWriter.java:207)
> at
>
> org.apache.hadoop.hbase.io.hfile.HFileWriterV2.append(HFileWriterV2.java:317)
>  at
>
> org.apache.hadoop.hbase.io.hfile.HFileWriterV2.append(HFileWriterV2.java:282)
> at com.mycompany.hbase.process.myprocess.myFunction(MyClass.java:1492)
>
> I am using hbase-0.94.2
>
> - Rohit Kelkar
>
>
> On Wed, Jun 19, 2013 at 1:15 PM, Jeff Kolesky <jeff@opower.com> wrote:
>
> > I believe you need to use the KVComparator:
> >
> >
> >
> https://github.com/apache/hbase/blob/trunk/hbase-common/src/main/java/org/apache/hadoop/hbase/KeyValue.java#L88
> >
> > Jeff
> >
> >
> > On Wed, Jun 19, 2013 at 10:32 AM, Rohit Kelkar <rohitkelkar@gmail.com
> > >wrote:
> >
> > > Here is the code - https://gist.github.com/anonymous/5816180
> > >
> > > I guess the issue is with my use of the comparator function.
> > >
> > > - R
> > >
> >
> >
> >
> > --
> > *Jeff Kolesky*
> > Chief Software Architect
> > *Opower*
> >
>



-- 
*Jeff Kolesky*
Chief Software Architect
*Opower*

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message