hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alex Newman (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HBASE-1359) After a large truncating table HBase becomes unresponsive
Date Wed, 15 Jul 2009 15:44:14 GMT

    [ https://issues.apache.org/jira/browse/HBASE-1359?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12731517#action_12731517
] 

Alex Newman commented on HBASE-1359:
------------------------------------

Disable and drop now work so I would say this is not a blocking issue. This is a rather large
table maybe 40/50 regions.


$date; echo "disable 't5'"| .../hbase shell ;date
Wed Jul 15 11:36:21 EDT 2009
HBase Shell; enter 'help<RETURN>' for list of supported commands.
Version: 0.20.0-dev, r794282, Wed Jul 15 10:29:12 EDT 2009
disable 't5'
09/07/15 11:36:24 DEBUG client.HConnectionManager: Created new HBASE_INSTANCES
09/07/15 11:36:24 DEBUG client.HConnectionManager$TableServers: Got ZooKeeper event, state:
SyncConnected, type: None, path: null
09/07/15 11:36:24 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/master got **************:60000
09/07/15 11:36:49 DEBUG client.HConnectionManager: Created new HBASE_INSTANCES
09/07/15 11:36:49 DEBUG client.HConnectionManager$TableServers: Got ZooKeeper event, state:
SyncConnected, type: None, path: null
09/07/15 11:36:49 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got
**************:60020
09/07/15 11:36:49 DEBUG client.HConnectionManager$TableServers: Found ROOT at **************:60020
09/07/15 11:36:50 DEBUG client.HConnectionManager$TableServers: Cached location address: **************:60020,
regioninfo: REGION => {NAME => '.META.,,1', STARTKEY => '', ENDKEY => '', ENCODED
=> 1028785192, TABLE => {{NAME => '.META.', IS_META => 'true', MEMSTORE_FLUSHSIZE
=> '16384', FAMILIES => [{NAME => 'historian', COMPRESSION => 'NONE', VERSIONS
=> '2147483647', TTL => '604800', BLOCKSIZE => '8192', IN_MEMORY => 'false', BLOCKCACHE
=> 'false'}, {NAME => 'info', COMPRESSION => 'NONE', VERSIONS => '10', TTL =>
'2147483647', BLOCKSIZE => '8192', IN_MEMORY => 'false', BLOCKCACHE => 'true'}]}}


:$date; echo "drop 't5'"| /home/fds/ts/hadoop/hbase/bin/hbase shell ;date Wed Jul 15 11:37:39
EDT 2009
HBase Shell; enter 'help<RETURN>' for list of supported commands.
Version: 0.20.0-dev, r794282, Wed Jul 15 10:29:12 EDT 2009
drop't5'
09/07/15 11:37:42 DEBUG client.HConnectionManager: Created new HBASE_INSTANCES
09/07/15 11:37:42 DEBUG client.HConnectionManager$TableServers: Got ZooKeeper event, state:
SyncConnected, type: None, path: null
09/07/15 11:37:42 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/master got **************:60000
09/07/15 11:37:43 DEBUG client.HConnectionManager: Created new HBASE_INSTANCES
09/07/15 11:37:43 DEBUG client.HConnectionManager$TableServers: Got ZooKeeper event, state:
SyncConnected, type: None, path: null
09/07/15 11:37:43 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got
**************:60020
09/07/15 11:37:43 DEBUG client.HConnectionManager$TableServers: Found ROOT at **************:60020
09/07/15 11:37:44 DEBUG client.HConnectionManager$TableServers: Cached location address: **************:60020,
regioninfo: REGION => {NAME => '.META.,,1', STARTKEY => '', ENDKEY => '', ENCODED
=> 1028785192, TABLE => {{NAME => '.META.', IS_META => 'true', MEMSTORE_FLUSHSIZE
=> '16384', FAMILIES => [{NAME => 'historian', COMPRESSION => 'NONE', VERSIONS
=> '2147483647', TTL => '604800', BLOCKSIZE => '8192', IN_MEMORY => 'false', BLOCKCACHE
=> 'false'}, {NAME => 'info', COMPRESSION => 'NONE', VERSIONS => '10', TTL =>
'2147483647', BLOCKSIZE => '8192', IN_MEMORY => 'false', BLOCKCACHE => 'true'}]}}
09/07/15 11:37:47 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got
**************:60020
09/07/15 11:37:47 DEBUG client.HConnectionManager$TableServers: Found ROOT at **************:60020
09/07/15 11:37:47 DEBUG client.HConnectionManager$TableServers: Cached location address: **************:60020,
regioninfo: REGION => {NAME => '.META.,,1', STARTKEY => '', ENDKEY => '', ENCODED
=> 1028785192, TABLE => {{NAME => '.META.', IS_META => 'true', MEMSTORE_FLUSHSIZE
=> '16384', FAMILIES => [{NAME => 'historian', COMPRESSION => 'NONE', VERSIONS
=> '2147483647', TTL => '604800', BLOCKSIZE => '8192', IN_MEMORY => 'false', BLOCKCACHE
=> 'false'}, {NAME => 'info', COMPRESSION => 'NONE', VERSIONS => '10', TTL =>
'2147483647', BLOCKSIZE => '8192', IN_MEMORY => 'false', BLOCKCACHE => 'true'}]}}
09/07/15 11:39:37 DEBUG zookeeper.ZooKeeperWrapper: Closed connection with ZooKeeper
09/07/15 11:39:37 INFO client.HBaseAdmin: Deleted t5
0 row(s) in 3.3580 seconds
0 row(s) in 118.0930 seconds
Wed Jul 15 11:39:42 EDT 2009

09/07/15 11:36:53 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode /hbase/root-region-server got
**************:60020
09/07/15 11:36:54 DEBUG client.HConnectionManager$TableServers: Found ROOT at **************:60020
09/07/15 11:36:54 DEBUG client.HConnectionManager$TableServers: Cached location address: **************:60020,
regioninfo: REGION => {NAME => '.META.,,1', STARTKEY => '', ENDKEY => '', ENCODED
=> 1028785192, TABLE => {{NAME => '.META.', IS_META => 'true', MEMSTORE_FLUSHSIZE
=> '16384', FAMILIES => [{NAME => 'historian', COMPRESSION => 'NONE', VERSIONS
=> '2147483647', TTL => '604800', BLOCKSIZE => '8192', IN_MEMORY => 'false', BLOCKCACHE
=> 'false'}, {NAME => 'info', COMPRESSION => 'NONE', VERSIONS => '10', TTL =>
'2147483647', BLOCKSIZE => '8192', IN_MEMORY => 'false', BLOCKCACHE => 'true'}]}}
09/07/15 11:37:04 DEBUG client.HBaseAdmin: Sleep. Waiting for all regions to be disabled from
t5
09/07/15 11:37:06 DEBUG client.HBaseAdmin: Wake. Waiting for all regions to be disabled from
t5
09/07/15 11:37:06 DEBUG client.HConnectionManager$TableServers: Cache hit for row <>
in tableName .META.: location server **************:60020, location region name .META.,,1
09/07/15 11:37:09 DEBUG client.HConnectionManager$TableServers: Cache hit for row <>
in tableName .META.: location server **************60020, location region name .META.,,1
09/07/15 11:37:16 INFO client.HBaseAdmin: Disabled t5
0 row(s) in 51.0380 seconds
Wed Jul 15 11:37:16 EDT 2009





> After  a large truncating table HBase becomes unresponsive
> ----------------------------------------------------------
>
>                 Key: HBASE-1359
>                 URL: https://issues.apache.org/jira/browse/HBASE-1359
>             Project: Hadoop HBase
>          Issue Type: Bug
>         Environment: linux  r769665
>            Reporter: Alex Newman
>            Assignee: Alex Newman
>            Priority: Critical
>             Fix For: 0.20.0
>
>
> If you see **** I removed and ip or something for security reasons
> Once I truncate the table, hbase freaks out for about 10 seconds and all the thrift servers
die.
> Thrift server log:
> 2009-04-02 12:09:08,971 INFO org.apache.hadoop.ipc.HBaseClass: Retrying connect 
> to server: /*****:60020. Already tried 0 time(s).
> You see this a bunch of times and then it times out
> The hbase shell
> nhbase(main):001:0> truncate 't2'
> 09/04/30 13:01:08 INFO zookeeper.ZooKeeperWrapper: Quorum servers: 
> ****
> Truncating t2; it may take a while
> Disabling table...
> 09/04/30 13:01:19 INFO client.HBaseAdmin: Disabled t2
> 0 row(s) in 10.3417 seconds
> Dropping table...
> 09/04/30 13:01:19 INFO client.HBaseAdmin: Deleted t2
> 0 row(s) in 0.1592 seconds
> Creating table...
> 0 row(s) in 14.7567 seconds
> hbase(main):002:0> lsit
> NameError: undefined local variable or method `lsit' for #<Object:0x3bbe9a50>
>         from (hbase):3
> hbase(main):003:0> lsit
> NameError: undefined local variable or method `lsit' for #<Object:0x3bbe9a50>
>         from (hbase):4
> hbase(main):004:0> list
> NativeException: java.lang.NullPointerException: null
>         from org/apache/hadoop/hbase/client/HConnectionManager.java:344:in `proc
> essRow' 
>       from org/apache/hadoop/hbase/client/MetaScanner.java:64:in `metaScan'
>         from org/apache/hadoop/hbase/client/MetaScanner.java:29:in `metaScan'
>         from org/apache/hadoop/hbase/client/HConnectionManager.java:351:in `list
> Tables' 
>         from org/apache/hadoop/hbase/client/HBaseAdmin.java:121:in `listTables'
>         from sun/reflect/NativeMethodAccessorImpl.java:-2:in `invoke0'
>         from sun/reflect/NativeMethodAccessorImpl.java:39:in `invoke'
>         from sun/reflect/DelegatingMethodAccessorImpl.java:25:in `invoke'
>         from java/lang/reflect/Method.java:597:in `invoke'
>         from org/jruby/javasupport/JavaMethod.java:298:in `invokeWithExceptionHa
> ndling' 
>         from org/jruby/javasupport/JavaMethod.java:259:in `invoke'
>         from org/jruby/java/invokers/InstanceMethodInvoker.java:36:in `call'
>         from org/jruby/runtime/callsite/CachingCallSite.java:260:in `cacheAndCal
> l'
>         from org/jruby/runtime/callsite/CachingCallSite.java:75:in `call'
>         from org/jruby/ast/CallNoArgNode.java:61:in `interpret'
>         from org/jruby/ast/ForNode.java:101:in `interpret'
> ... 113 levels...
>         from org/jruby/internal/runtime/methods/DynamicMethod.java:226:in `call'
>         from org/jruby/internal/runtime/methods/CompiledMethod.java:216:in `call
> '
>         from org/jruby/internal/runtime/methods/CompiledMethod.java:71:in `call'
>         from org/jruby/runtime/callsite/CachingCallSite.java:260:in `cacheAndCal
> l'
>         from org/jruby/runtime/callsite/CachingCallSite.java:75:in `call'
>         from home/fds/ts/hadoop/hbase/bin/$_dot_dot_/bin/hirb.rb:441:in `__file_
> _'
>         from home/fds/ts/hadoop/hbase/bin/$_dot_dot_/bin/hirb.rb:-1:in `__file__
> '
>         from home/fds/ts/hadoop/hbase/bin/$_dot_dot_/bin/hirb.rb:-1:in `load'
>         from org/jruby/Ruby.java:564:in `runScript'
>         from org/jruby/Ruby.java:467:in `runNormally'
>         from org/jruby/Ruby.java:340:in `runFromMain'
>         from org/jruby/Main.java:214:in `run'
>         from org/jruby/Main.java:100:in `run'
>         from org/jruby/Main.java:84:in `main'
>         from /home/fds/ts/hadoop/hbase/bin/../bin/hirb.rb:300:in `list'
>         from (hbase):5hbase(main):005:0>
> hbase(main):006:0* 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message