hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Andrew Purtell (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HBASE-1500) KeyValue$KeyComparator array overrun
Date Fri, 12 Jun 2009 19:23:07 GMT

    [ https://issues.apache.org/jira/browse/HBASE-1500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12718949#action_12718949
] 

Andrew Purtell commented on HBASE-1500:
---------------------------------------

Some detail I added up on IRC:

<apurtell>	i was wondering if it was just me...
<apurtell>	yeah would be great if someone else can confirm the repro works for them...
<apurtell>	what's odd is the shell's count of all the rows comes back before it even
tries to advance the scanner to the next split. some kind of META sort problem, parent region
coming back before daughter? i'd like to dig in but this code is all new to me
<apurtell>	in the repro case, the split is triggered and confirmed, so there is more
than one region...
<apurtell>	this is the same basic problem as with TableSplit, iterating over META is
going off the rails, giving bad results, triggering the out of bounds exception
<apurtell>	at least this is what it looks like to me
<apurtell>	started immediately after commit of 1304
<apurtell>	noticed it first while testing stargate scanners, scanners hang/crash/fail
on tables with more than one region



> KeyValue$KeyComparator array overrun
> ------------------------------------
>
>                 Key: HBASE-1500
>                 URL: https://issues.apache.org/jira/browse/HBASE-1500
>             Project: Hadoop HBase
>          Issue Type: Bug
>    Affects Versions: 0.20.0
>            Reporter: Andrew Purtell
>            Assignee: ryan rawson
>            Priority: Blocker
>             Fix For: 0.20.0
>
>         Attachments: hbase.log.gz, test.log.gz, test.rb
>
>
> {code}
> 09/06/08 22:58:47 INFO zookeeper.ZooKeeper: Initiating client connection, host=B
> OA03:2181,BOA02:2181,BOA01:2181,BOA04:2181 sessionTimeout=10000 watcher=org.apac
> he.hadoop.hbase.zookeeper.WatcherWrapper@518bf072
> 09/06/08 22:58:47 INFO zookeeper.ClientCnxn: zookeeper.disableAutoWatchReset is
> false
> 09/06/08 22:58:47 INFO zookeeper.ClientCnxn: Attempting connection to server BOA
> 04/172.20.3.231:2181
> 09/06/08 22:58:47 INFO zookeeper.ClientCnxn: Priming connection to java.nio.chan
> nels.SocketChannel[connected local=/172.20.3.232:40296 remote=BOA04/172.20.3.231
> :2181]
> 09/06/08 22:58:47 INFO zookeeper.ClientCnxn: Server connection successful
> 09/06/08 22:58:47 WARN mapred.JobClient: Use GenericOptionsParser for parsing th
> e arguments. Applications should implement Tool for the same.
> 09/06/08 22:58:47 WARN mapred.JobClient: No job jar file set.  User classes may
> not be found. See JobConf(Class) or JobConf#setJar(String).
> 09/06/08 22:58:47 INFO zookeeper.ZooKeeper: Initiating client connection, host=B
> OA03:2181,BOA02:2181,BOA01:2181,BOA04:2181 sessionTimeout=10000 watcher=org.apac
> he.hadoop.hbase.zookeeper.WatcherWrapper@362f0d54
> 09/06/08 22:58:47 INFO zookeeper.ClientCnxn: Attempting connection to server BOA
> 03/172.20.3.230:2181
> 09/06/08 22:58:47 INFO zookeeper.ClientCnxn: Priming connection to java.nio.chan
> nels.SocketChannel[connected local=/172.20.3.232:42792 remote=BOA03/172.20.3.230
> :2181]
> 09/06/08 22:58:47 INFO zookeeper.ClientCnxn: Server connection successful
> 09/06/08 22:58:48 INFO mapred.TableInputFormatBase: split: 0->BOA04.trendmicro.c
> om:,01e33c601a7a9dd0ddb5c8427438f2f1
> Exception in thread "main" java.lang.ArrayIndexOutOfBoundsException: 32
>         at org.apache.hadoop.hbase.util.Bytes.compareTo(Bytes.java:798)
>         at org.apache.hadoop.hbase.KeyValue$KeyComparator.compareRows(KeyValue.j
> ava:1760)
>         at org.apache.hadoop.hbase.KeyValue$KeyComparator.compare(KeyValue.java:
> 1696)
>         at org.apache.hadoop.hbase.KeyValue$KeyComparator.compare(KeyValue.java:
> 1755)
>         at org.apache.hadoop.hbase.KeyValue$KeyComparator.compare(KeyValue.java:
> 1687)
>         at org.apache.hadoop.hbase.client.HConnectionManager$TableServers.getCac
> hedLocation(HConnectionManager.java:697)
>         at org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locate
> RegionInMeta(HConnectionManager.java:541)
>         at org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locate
> Region(HConnectionManager.java:525)
>         at org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locate
> Region(HConnectionManager.java:488)
>         at org.apache.hadoop.hbase.client.HConnectionManager$TableServers.getReg
> ionLocation(HConnectionManager.java:342)
>         at org.apache.hadoop.hbase.client.HTable.getRegionLocation(HTable.java:1
> 91)
>         at org.apache.hadoop.hbase.mapred.TableInputFormatBase.getSplits(TableIn
> putFormatBase.java:296)
>         at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:742)
>         at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:1026)
>         at net.iridiant.crawler.mapred.DocumentParser.main(Unknown Source)
> {code}

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message