Mailing-List: contact hbase-dev-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: hbase-dev@hadoop.apache.org
Message-ID: <75145308.1244665028054.JavaMail.jira@brutus>
Date: Wed, 10 Jun 2009 13:17:08 -0700 (PDT)
From: "Lars George (JIRA)" <jira@apache.org>
To: hbase-dev@hadoop.apache.org
Subject: [jira] Commented: (HBASE-1500) KeyValue$KeyComparator array overrun
In-Reply-To: <773638980.1244502127346.JavaMail.jira@brutus>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/HBASE-1500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12718183#action_12718183 ] 

Lars George commented on HBASE-1500:
------------------------------------

It looks like it happens here from the above

{code}
    public int compare(byte[] left, int loffset, int llength, byte[] right,
        int roffset, int rlength) {
      // Compare row
      short lrowlength = Bytes.toShort(left, loffset);
      short rrowlength = Bytes.toShort(right, roffset);
      int compare = compareRows(left, loffset + Bytes.SIZEOF_SHORT,
          lrowlength,
          right, roffset + Bytes.SIZEOF_SHORT, rrowlength);
      if (compare != 0) {
        return compare;
      }
{code}

It reads the lengths from the key byte array assuming they are correct and then applies those lengths to the compareRow() call, which then fails.

This indicates a borked length from a key array? Could you instrument the code above and log the lrowlength, rrowlength and the the real left.length and right.length?

If that mismatches, the question is why?

> KeyValue$KeyComparator array overrun
> ------------------------------------
>
>                 Key: HBASE-1500
>                 URL: https://issues.apache.org/jira/browse/HBASE-1500
>             Project: Hadoop HBase
>          Issue Type: Bug
>    Affects Versions: 0.20.0
>            Reporter: Andrew Purtell
>            Priority: Blocker
>             Fix For: 0.20.0
>
>
> {code}
> 09/06/08 22:58:47 INFO zookeeper.ZooKeeper: Initiating client connection, host=B
> OA03:2181,BOA02:2181,BOA01:2181,BOA04:2181 sessionTimeout=10000 watcher=org.apac
> he.hadoop.hbase.zookeeper.WatcherWrapper@518bf072
> 09/06/08 22:58:47 INFO zookeeper.ClientCnxn: zookeeper.disableAutoWatchReset is
> false
> 09/06/08 22:58:47 INFO zookeeper.ClientCnxn: Attempting connection to server BOA
> 04/172.20.3.231:2181
> 09/06/08 22:58:47 INFO zookeeper.ClientCnxn: Priming connection to java.nio.chan
> nels.SocketChannel[connected local=/172.20.3.232:40296 remote=BOA04/172.20.3.231
> :2181]
> 09/06/08 22:58:47 INFO zookeeper.ClientCnxn: Server connection successful
> 09/06/08 22:58:47 WARN mapred.JobClient: Use GenericOptionsParser for parsing th
> e arguments. Applications should implement Tool for the same.
> 09/06/08 22:58:47 WARN mapred.JobClient: No job jar file set.  User classes may
> not be found. See JobConf(Class) or JobConf#setJar(String).
> 09/06/08 22:58:47 INFO zookeeper.ZooKeeper: Initiating client connection, host=B
> OA03:2181,BOA02:2181,BOA01:2181,BOA04:2181 sessionTimeout=10000 watcher=org.apac
> he.hadoop.hbase.zookeeper.WatcherWrapper@362f0d54
> 09/06/08 22:58:47 INFO zookeeper.ClientCnxn: Attempting connection to server BOA
> 03/172.20.3.230:2181
> 09/06/08 22:58:47 INFO zookeeper.ClientCnxn: Priming connection to java.nio.chan
> nels.SocketChannel[connected local=/172.20.3.232:42792 remote=BOA03/172.20.3.230
> :2181]
> 09/06/08 22:58:47 INFO zookeeper.ClientCnxn: Server connection successful
> 09/06/08 22:58:48 INFO mapred.TableInputFormatBase: split: 0->BOA04.trendmicro.c
> om:,01e33c601a7a9dd0ddb5c8427438f2f1
> Exception in thread "main" java.lang.ArrayIndexOutOfBoundsException: 32
>         at org.apache.hadoop.hbase.util.Bytes.compareTo(Bytes.java:798)
>         at org.apache.hadoop.hbase.KeyValue$KeyComparator.compareRows(KeyValue.j
> ava:1760)
>         at org.apache.hadoop.hbase.KeyValue$KeyComparator.compare(KeyValue.java:
> 1696)
>         at org.apache.hadoop.hbase.KeyValue$KeyComparator.compare(KeyValue.java:
> 1755)
>         at org.apache.hadoop.hbase.KeyValue$KeyComparator.compare(KeyValue.java:
> 1687)
>         at org.apache.hadoop.hbase.client.HConnectionManager$TableServers.getCac
> hedLocation(HConnectionManager.java:697)
>         at org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locate
> RegionInMeta(HConnectionManager.java:541)
>         at org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locate
> Region(HConnectionManager.java:525)
>         at org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locate
> Region(HConnectionManager.java:488)
>         at org.apache.hadoop.hbase.client.HConnectionManager$TableServers.getReg
> ionLocation(HConnectionManager.java:342)
>         at org.apache.hadoop.hbase.client.HTable.getRegionLocation(HTable.java:1
> 91)
>         at org.apache.hadoop.hbase.mapred.TableInputFormatBase.getSplits(TableIn
> putFormatBase.java:296)
>         at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:742)
>         at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:1026)
>         at net.iridiant.crawler.mapred.DocumentParser.main(Unknown Source)
> {code}

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.