hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jim Kellerman (POWERSET)" <Jim.Keller...@microsoft.com>
Subject RE: [jira] Commented: (HBASE-1500) KeyValue$KeyComparator array overrun
Date Thu, 11 Jun 2009 07:26:13 GMT
Or attach it to the Jira issue?

---
Jim Kellerman, Powerset (Live Search, Microsoft Corporation)


> -----Original Message-----
> From: Ryan Rawson [mailto:ryanobjc@gmail.com]
> Sent: Thursday, June 11, 2009 12:24 AM
> To: hbase-dev@hadoop.apache.org
> Subject: Re: [jira] Commented: (HBASE-1500) KeyValue$KeyComparator
> array overrun
>
> Can you send me the source to the test program?
>
> -ryan
>
> On Wed, Jun 10, 2009 at 10:43 PM, Andrew Purtell (JIRA)
> <jira@apache.org>wrote:
>
> >
> >    [
> > https://issues.apache.org/jira/browse/HBASE-
> 1500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-
> tabpanel&focusedCommentId=12718307#action_12718307]
> >
> > Andrew Purtell commented on HBASE-1500:
> > ---------------------------------------
> >
> > This is pretty basic breakage. I inserted a few 1000 rows of data
> keyed
> > with MD5 sums, then tried to run a count from the shell:
> >
> > {code}
> > count 'content'
> > 09/06/10 22:36:15 DEBUG client.HConnectionManager$TableServers:
> Cache hit
> > for ro
> > w <> in tableName content: location server 192.168.56.1:39961,
> location
> > region n
> > ame content,,1244694091453
> > 09/06/10 22:36:15 DEBUG client.HConnectionManager$TableServers:
> Cache hit
> > for ro
> > w <> in tableName .META.: location server 192.168.56.1:39961,
> location
> > region na
> > me .META.,,1
> > 09/06/10 22:36:15 DEBUG client.HTable$ClientScanner: Creating
> scanner over
> > conte
> > nt starting at key ''
> > 09/06/10 22:36:15 DEBUG client.HTable$ClientScanner: Advancing
> internal
> > scanner
> > to startKey at ''
> > 09/06/10 22:36:15 DEBUG client.HConnectionManager$TableServers:
> Cache hit
> > for ro
> > w <> in tableName content: location server 192.168.56.1:39961,
> location
> > region n
> > ame content,,1244694091453
> > 09/06/10 22:36:17 DEBUG client.HConnectionManager$TableServers:
> address:
> > 192.168
> > .56.1:39961, regioninfo: REGION => {NAME =>
> 'content,,1244698495875',
> > STARTKEY =
> > > '', ENDKEY => '7f47a51d4f7a8c882288d86a4c5cdc3d', ENCODED =>
> 1221125875,
> > TABLE
> >  => {{NAME => 'content', FAMILIES => [{NAME => 'content', VERSIONS
> => '1',
> > COMPR
> > ESSION => 'GZ', TTL => '2147483647', BLOCKSIZE => '65536',
> IN_MEMORY =>
> > 'false',
> >  BLOCKCACHE => 'false'}, {NAME => 'info', VERSIONS => '1',
> COMPRESSION =>
> > 'NONE'
> > , TTL => '2147483647', BLOCKSIZE => '65536', IN_MEMORY => 'false',
> > BLOCKCACHE =>
> >  'false'}, {NAME => 'url', VERSIONS => '1', COMPRESSION => 'NONE',
> TTL =>
> > '21474
> > 83647', BLOCKSIZE => '65536', IN_MEMORY => 'false', BLOCKCACHE =>
> > 'false'}]}}
> > Current count: 1000, row: 0a917eadf27f4917f1a823f085d1e4bd
> >
> > Current count: 2000, row: 156d46ad656a85f47e4e8d50c23167e9
> >
> > Current count: 3000, row: 1fef03d4744e5252dee5f44c36f74a00
> >
> > Current count: 4000, row: 2a8376e2718993eab5a7aa3cfc769e42
> >
> > Current count: 5000, row: 34e566f2cd7eb35ea2d7ce6bc2312638
> >
> > Current count: 6000, row: 3f68045f0aae18f432e0de665909f202
> >
> > Current count: 7000, row: 49c514099448cfd9609445b37f5b161f
> >
> > Current count: 8000, row: 5476db74043ccf5e40cb7974e7c07c83
> >
> > Current count: 9000, row: 5f3998658a477b7cbf806f4082643f2a
> >
> > Current count: 10000, row: 69fd32fcf76f64116602def18248938b
> >
> > Current count: 11000, row: 74be3000056b58b8b487c5243aadf02b
> >
> > Current count: 12000, row: 7ef67ca4c45a28d90847288300f05d4a
> >
> > Current count: 13000, row: 89783606fa73bcbf10a7b850bfa57877
> >
> > Current count: 14000, row: 945c830a5eecd63e948e736deada42f3
> >
> > Current count: 15000, row: 9f41e1454905fd7416f89aa4380a65e1
> >
> > Current count: 16000, row: aa54517d0450127c90e4dd4631eaa010
> >
> > Current count: 17000, row: b57b55373bf1302135dcefb044a58528
> >
> > Current count: 18000, row: c06d66ca04ab273cc96f050ded843160
> >
> > Current count: 19000, row: cb36f29c8d12098e25f007d7f658f495
> >
> > Current count: 20000, row: d59d9e495fd2f983b302e67d94aa5962
> >
> > Current count: 21000, row: e10430e1fda5ad363c72bb5ab7e7f8dd
> >
> > Current count: 22000, row: ec1d2d9cc89cd8ec67ffa1db1f3a7474
> >
> > Current count: 23000, row: f73d7cb9759f556b94aad178e6a6e0d5
> >
> > 09/06/10 22:37:24 DEBUG client.HTable$ClientScanner: Advancing
> forward from
> > regi
> > on REGION => {NAME => 'content,,1244698495875', STARTKEY => '',
> ENDKEY =>
> > '7f47a
> > 51d4f7a8c882288d86a4c5cdc3d', ENCODED => 1221125875, TABLE =>
> {{NAME =>
> > 'content
> > ', FAMILIES => [{NAME => 'content', VERSIONS => '1', COMPRESSION
> => 'GZ',
> > TTL =>
> >  '2147483647', BLOCKSIZE => '65536', IN_MEMORY => 'false',
> BLOCKCACHE =>
> > 'false'
> > }, {NAME => 'info', VERSIONS => '1', COMPRESSION => 'NONE', TTL =>
> > '2147483647',
> >  BLOCKSIZE => '65536', IN_MEMORY => 'false', BLOCKCACHE =>
> 'false'}, {NAME
> > => 'u
> > rl', VERSIONS => '1', COMPRESSION => 'NONE', TTL => '2147483647',
> BLOCKSIZE
> > => '
> > 65536', IN_MEMORY => 'false', BLOCKCACHE => 'false'}]}}
> > 09/06/10 22:37:24 DEBUG client.HTable$ClientScanner: Advancing
> internal
> > scanner
> > to startKey at '7f47a51d4f7a8c882288d86a4c5cdc3d'
> > NativeException: java.lang.RuntimeException:
> > org.apache.hadoop.hbase.client.Retr
> > iesExhaustedException: Trying to contact region server null for
> region ,
> > row '7f
> > 47a51d4f7a8c882288d86a4c5cdc3d', but failed after 5 attempts.
> > Exceptions:
> > java.lang.ArrayIndexOutOfBoundsException: 32
> > java.lang.ArrayIndexOutOfBoundsException: 32
> > java.lang.ArrayIndexOutOfBoundsException: 32
> > java.lang.ArrayIndexOutOfBoundsException: 32
> > java.lang.ArrayIndexOutOfBoundsException: 32
> >        from org/apache/hadoop/hbase/client/HTable.java:2002:in
> `hasNext'
> >        from sun.reflect.GeneratedMethodAccessor2:-1:in `invoke'
> >        from sun/reflect/DelegatingMethodAccessorImpl.java:25:in
> `invoke'
> >        from java/lang/reflect/Method.java:597:in `invoke'
> >        from org/jruby/javasupport/JavaMethod.java:298:in
> > `invokeWithExceptionHandling'
> >        from org/jruby/javasupport/JavaMethod.java:259:in `invoke'
> >        from
> org/jruby/java/invokers/InstanceMethodInvoker.java:36:in `call'
> >        from org/jruby/runtime/callsite/CachingCallSite.java:70:in
> `call'
> >        from org/jruby/ast/CallNoArgNode.java:61:in `interpret'
> >        from org/jruby/ast/WhileNode.java:127:in `interpret'
> >        from org/jruby/ast/NewlineNode.java:104:in `interpret'
> >        from org/jruby/ast/BlockNode.java:71:in `interpret'
> >        from
> > org/jruby/internal/runtime/methods/InterpretedMethod.java:163:in
> `call'
> >        from
> org/jruby/internal/runtime/methods/DefaultMethod.java:144:in
> > `call'
> >        from org/jruby/runtime/callsite/CachingCallSite.java:110:in
> `call'
> >        from
> C_3a_/cygwin/opt/HBASE_minus_7e_1_dot_3/bin/hirb.rb:369:in
> > `method_
> > _25$RUBY$count'
> > ... 106 levels...
> >        from
> C_3a_/cygwin/opt/HBASE_minus_7e_1_dot_3/bin/hirb#start:-1:in
> > `call'
> >        from
> org/jruby/internal/runtime/methods/DynamicMethod.java:226:in
> > `call'
> >        from
> org/jruby/internal/runtime/methods/CompiledMethod.java:211:in
> > `call'
> >        from
> org/jruby/internal/runtime/methods/CompiledMethod.java:71:in
> > `call'
> >        from org/jruby/runtime/callsite/CachingCallSite.java:253:in
> > `cacheAndCall'
> >        from org/jruby/runtime/callsite/CachingCallSite.java:72:in
> `call'
> >        from
> C_3a_/cygwin/opt/HBASE_minus_7e_1_dot_3/bin/hirb.rb:450:in
> > `__file__'
> >        from C_3a_/cygwin/opt/HBASE_minus_7e_1_dot_3/bin/hirb.rb:-
> 1:in
> > `load'
> >        from org/jruby/Ruby.java:577:in `runScript'
> >        from org/jruby/Ruby.java:480:in `runNormally'
> >        from org/jruby/Ruby.java:354:in `runFromMain'
> >        from org/jruby/Main.java:229:in `run'
> >        from org/jruby/Main.java:110:in `run'
> >        from org/jruby/Main.java:94:in `main'
> >        from C:\cygwin\opt\HBASE-~1.3\/bin/hirb.rb:369:in `count'
> >        from (hbase):4
> > {code}
> >
> > > KeyValue$KeyComparator array overrun
> > > ------------------------------------
> > >
> > >                 Key: HBASE-1500
> > >                 URL:
> https://issues.apache.org/jira/browse/HBASE-1500
> > >             Project: Hadoop HBase
> > >          Issue Type: Bug
> > >    Affects Versions: 0.20.0
> > >            Reporter: Andrew Purtell
> > >            Priority: Blocker
> > >             Fix For: 0.20.0
> > >
> > >
> > > {code}
> > > 09/06/08 22:58:47 INFO zookeeper.ZooKeeper: Initiating client
> connection,
> > host=B
> > > OA03:2181,BOA02:2181,BOA01:2181,BOA04:2181 sessionTimeout=10000
> > watcher=org.apac
> > > he.hadoop.hbase.zookeeper.WatcherWrapper@518bf072
> > > 09/06/08 22:58:47 INFO zookeeper.ClientCnxn:
> > zookeeper.disableAutoWatchReset is
> > > false
> > > 09/06/08 22:58:47 INFO zookeeper.ClientCnxn: Attempting
> connection to
> > server BOA
> > > 04/172.20.3.231:2181
> > > 09/06/08 22:58:47 INFO zookeeper.ClientCnxn: Priming connection
> to
> > java.nio.chan
> > > nels.SocketChannel[connected local=/172.20.3.232:40296
> remote=BOA04/
> > 172.20.3.231
> > > :2181]
> > > 09/06/08 22:58:47 INFO zookeeper.ClientCnxn: Server connection
> successful
> > > 09/06/08 22:58:47 WARN mapred.JobClient: Use
> GenericOptionsParser for
> > parsing th
> > > e arguments. Applications should implement Tool for the same.
> > > 09/06/08 22:58:47 WARN mapred.JobClient: No job jar file set.
> User
> > classes may
> > > not be found. See JobConf(Class) or JobConf#setJar(String).
> > > 09/06/08 22:58:47 INFO zookeeper.ZooKeeper: Initiating client
> connection,
> > host=B
> > > OA03:2181,BOA02:2181,BOA01:2181,BOA04:2181 sessionTimeout=10000
> > watcher=org.apac
> > > he.hadoop.hbase.zookeeper.WatcherWrapper@362f0d54
> > > 09/06/08 22:58:47 INFO zookeeper.ClientCnxn: Attempting
> connection to
> > server BOA
> > > 03/172.20.3.230:2181
> > > 09/06/08 22:58:47 INFO zookeeper.ClientCnxn: Priming connection
> to
> > java.nio.chan
> > > nels.SocketChannel[connected local=/172.20.3.232:42792
> remote=BOA03/
> > 172.20.3.230
> > > :2181]
> > > 09/06/08 22:58:47 INFO zookeeper.ClientCnxn: Server connection
> successful
> > > 09/06/08 22:58:48 INFO mapred.TableInputFormatBase: split:
> > 0->BOA04.trendmicro.c
> > > om:,01e33c601a7a9dd0ddb5c8427438f2f1
> > > Exception in thread "main"
> java.lang.ArrayIndexOutOfBoundsException: 32
> > >         at
> org.apache.hadoop.hbase.util.Bytes.compareTo(Bytes.java:798)
> > >         at
> >
> org.apache.hadoop.hbase.KeyValue$KeyComparator.compareRows(KeyValue.
> j
> > > ava:1760)
> > >         at
> >
> org.apache.hadoop.hbase.KeyValue$KeyComparator.compare(KeyValue.java
> :
> > > 1696)
> > >         at
> >
> org.apache.hadoop.hbase.KeyValue$KeyComparator.compare(KeyValue.java
> :
> > > 1755)
> > >         at
> >
> org.apache.hadoop.hbase.KeyValue$KeyComparator.compare(KeyValue.java
> :
> > > 1687)
> > >         at
> >
> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.getCa
> c
> > > hedLocation(HConnectionManager.java:697)
> > >         at
> >
> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locat
> e
> > > RegionInMeta(HConnectionManager.java:541)
> > >         at
> >
> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locat
> e
> > > Region(HConnectionManager.java:525)
> > >         at
> >
> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locat
> e
> > > Region(HConnectionManager.java:488)
> > >         at
> >
> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.getRe
> g
> > > ionLocation(HConnectionManager.java:342)
> > >         at
> >
> org.apache.hadoop.hbase.client.HTable.getRegionLocation(HTable.java:
> 1
> > > 91)
> > >         at
> >
> org.apache.hadoop.hbase.mapred.TableInputFormatBase.getSplits(TableI
> n
> > > putFormatBase.java:296)
> > >         at
> > org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:742)
> > >         at
> org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:1026)
> > >         at
> net.iridiant.crawler.mapred.DocumentParser.main(Unknown
> > Source)
> > > {code}
> >
> > --
> > This message is automatically generated by JIRA.
> > -
> > You can reply to this email to add a comment to the issue online.
> >
> >

Mime
View raw message