hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Andrew Purtell (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (HBASE-12142) Truncate command does not preserve ACLs table
Date Tue, 04 Nov 2014 02:43:36 GMT

    [ https://issues.apache.org/jira/browse/HBASE-12142?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14195604#comment-14195604
] 

Andrew Purtell edited comment on HBASE-12142 at 11/4/14 2:43 AM:
-----------------------------------------------------------------

Truncation is not working correctly. 

To reproduce:
# ./bin/hbase ltt -init_only
# ./bin/hbase shell
> truncate 'cluster_test'

TruncateTableHandler runs and claims to create table info, but it might not be moving the
file into the right place:
{noformat}
2014-11-04 02:30:23,653 DEBUG [FifoRpcScheduler.handler1-thread-18] handler.TruncateTableHandler:
Table 'cluster_test' archived!
2014-11-04 02:30:23,679 DEBUG [FifoRpcScheduler.handler1-thread-18] util.FSTableDescriptors:
Wrote descriptor into: hdfs://ip-10-33-129-9.us-west-2.compute.internal:8020/hbase/.tmp/.tabledesc/.tableinfo.0000000001
2014-11-04 02:30:23,680 INFO  [FifoRpcScheduler.handler1-thread-18] handler.TruncateTableHandler:
Truncate will not preserve the regions
2014-11-04 02:30:23,681 INFO  [RegionOpenAndInitThread-cluster_test-1] regionserver.HRegion:
creating HRegion cluster_test HTD == 'cluster_test', {TABLE_ATTRIBUTES => {DURABILITY =>
'USE_DEFAULT'}, {NAME => 'test_cf', DATA_BLOCK_ENCODING => 'NONE', BLOOMFILTER =>
'ROW', REPLICATION_SCOPE => '0', COMPRESSION => 'NONE', VERSIONS => '1', TTL =>
'FOREVER', MIN_VERSIONS => '0', KEEP_DELETED_CELLS => 'false', BLOCKSIZE => '65536',
IN_MEMORY => 'false', BLOCKCACHE => 'true'} RootDir = hdfs://ip-10-33-129-9.us-west-2.compute.internal:8020/hbase/.tmp
Table name == cluster_test
2014-11-04 02:30:23,703 DEBUG [RegionOpenAndInitThread-cluster_test-1] regionserver.HRegion:
Instantiated cluster_test,,1415068223680.c23e76525f0f025d51f74006db7903e2.
2014-11-04 02:30:23,703 DEBUG [RegionOpenAndInitThread-cluster_test-1] regionserver.HRegion:
Closing cluster_test,,1415068223680.c23e76525f0f025d51f74006db7903e2.: disabling compactions
& flushes
2014-11-04 02:30:23,703 DEBUG [RegionOpenAndInitThread-cluster_test-1] regionserver.HRegion:
Updates disabled for region cluster_test,,1415068223680.c23e76525f0f025d51f74006db7903e2.
2014-11-04 02:30:23,703 INFO  [RegionOpenAndInitThread-cluster_test-1] regionserver.HRegion:
Closed cluster_test,,1415068223680.c23e76525f0f025d51f74006db7903e2.
2014-11-04 02:30:23,707 INFO  [FifoRpcScheduler.handler1-thread-18] catalog.MetaEditor: Added
1
2014-11-04 02:30:23,708 DEBUG [FifoRpcScheduler.handler1-thread-18] master.AssignmentManager:
Assigning 1 region(s) to ip-10-204-3-67.us-west-2.compute.internal,8120,1415068107305
2014-11-04 02:30:23,708 DEBUG [FifoRpcScheduler.handler1-thread-18] zookeeper.ZKAssign: master:8100-0x149780b006a010a,
quorum=ip-10-33-129-9.us-west-2.compute.internal:2181, baseZNode=/hbase Async create of unassigned
node c23e76525f0f025d51f74006db7903e2 with OFFLINE state
{noformat}

But the region will fail to open
{noformat}
2014-11-04 02:30:23,738 DEBUG [main-EventThread] zookeeper.ZooKeeperWatcher: master:8100-0x149780b006a010a,
quorum=ip-10-33-129-9.us-west-2.compute.internal:2181, baseZNode=/hbase Received ZooKeeper
Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/region-in-transition/c23e76525f0f025d51f74006db7903e2
2014-11-04 02:30:23,738 DEBUG [AM.ZK.Worker-pool2-t14] master.AssignmentManager: Handling
RS_ZK_REGION_FAILED_OPEN, server=ip-10-204-3-67.us-west-2.compute.internal,8120,1415068107305,
region=c23e76525f0f025d51f74006db7903e2, current_state={c23e76525f0f025d51f74006db7903e2 state=OPENING,
ts=1415068223734, server=ip-10-204-3-67.us-west-2.compute.internal,8120,1415068107305}
2014-11-04 02:30:23,738 INFO  [AM.ZK.Worker-pool2-t14] master.RegionStates: Transition {c23e76525f0f025d51f74006db7903e2
state=OPENING, ts=1415068223734, server=ip-10-204-3-67.us-west-2.compute.internal,8120,1415068107305}
to {c23e76525f0f025d51f74006db7903e2 state=CLOSED, ts=1415068223738, server=ip-10-204-3-67.us-west-2.compute.internal,8120,1415068107305}
2014-11-04 02:30:23,738 WARN  [AM.ZK.Worker-pool2-t14] master.AssignmentManager: Can't move
c23e76525f0f025d51f74006db7903e2, there is no destination server available.
2014-11-04 02:30:23,739 DEBUG [AM.ZK.Worker-pool2-t14] handler.ClosedRegionHandler: Handling
CLOSED event for c23e76525f0f025d51f74006db7903e2
{noformat}

Because this is happening on the RegionServer side:
{noformat}
2014-11-04 02:30:23,704 INFO  [PriorityRpcServer.handler=16,queue=0,port=8120] regionserver.HRegionServer:
Open cluster_test,,1415068223680.c23e76525f0f025d51f74006db7903e2.
2014-11-04 02:30:23,717 DEBUG [PriorityRpcServer.handler=16,queue=0,port=8120] util.FSTableDescriptors:
Exception during readTableDecriptor. Current table name = cluster_test
org.apache.hadoop.hbase.TableInfoMissingException: No table descriptor file under hdfs://ip-10-33-129-9.us-west-2.compute.internal:8020/hbase/data/default/cluster_test
	at org.apache.hadoop.hbase.util.FSTableDescriptors.getTableDescriptorFromFs(FSTableDescriptors.java:510)
	at org.apache.hadoop.hbase.util.FSTableDescriptors.getTableDescriptorFromFs(FSTableDescriptors.java:487)
	at org.apache.hadoop.hbase.util.FSTableDescriptors.get(FSTableDescriptors.java:173)
	at org.apache.hadoop.hbase.regionserver.HRegionServer.openRegion(HRegionServer.java:3826)
	at org.apache.hadoop.hbase.protobuf.generated.AdminProtos$AdminService$2.callBlockingMethod(AdminProtos.java:20164)
	at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2027)
	at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:108)
	at org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:110)
	at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:90)
	at java.lang.Thread.run(Thread.java:745)
2014-11-04 02:30:23,719 DEBUG [RS_OPEN_REGION-ip-10-204-3-67:8120-2] zookeeper.ZKAssign: regionserver:8120-0x149780b006a0105,
quorum=ec2-54-71-67-141.us-west-2.compute.amazonaws.com:2181, baseZNode=/hbase Transitioning
c23e76525f0f025d51f74006db7903e2 from M_ZK_REGION_OFFLINE to RS_ZK_REGION_OPENING
2014-11-04 02:30:23,723 DEBUG [RS_OPEN_REGION-ip-10-204-3-67:8120-2] zookeeper.ZKAssign: regionserver:8120-0x149780b006a0105,
quorum=ec2-54-71-67-141.us-west-2.compute.amazonaws.com:2181, baseZNode=/hbase Transitioned
node c23e76525f0f025d51f74006db7903e2 from M_ZK_REGION_OFFLINE to RS_ZK_REGION_OPENING
2014-11-04 02:30:23,723 DEBUG [RS_OPEN_REGION-ip-10-204-3-67:8120-2] regionserver.HRegion:
Opening region: {ENCODED => c23e76525f0f025d51f74006db7903e2, NAME => 'cluster_test,,1415068223680.c23e76525f0f025d51f74006db7903e2.',
STARTKEY => '', ENDKEY => ''}
2014-11-04 02:30:23,723 ERROR [RS_OPEN_REGION-ip-10-204-3-67:8120-2] handler.OpenRegionHandler:
Failed open of region=cluster_test,,1415068223680.c23e76525f0f025d51f74006db7903e2., starting
to roll back the global memstore size.
java.lang.IllegalStateException: Could not instantiate a region instance.
	at org.apache.hadoop.hbase.regionserver.HRegion.newHRegion(HRegion.java:4266)
	at org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:4574)
	at org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:4547)
	at org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:4503)
	at org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:4454)
	at org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.openRegion(OpenRegionHandler.java:482)
	at org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.process(OpenRegionHandler.java:145)
	at org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:128)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.reflect.InvocationTargetException
	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
	at org.apache.hadoop.hbase.regionserver.HRegion.newHRegion(HRegion.java:4263)
	... 10 more
Caused by: java.lang.IllegalArgumentException: Need table descriptor
	at org.apache.hadoop.hbase.regionserver.HRegion.<init>(HRegion.java:556)
	at org.apache.hadoop.hbase.regionserver.HRegion.<init>(HRegion.java:532)
	... 15 more
2014-11-04 02:30:23,724 INFO  [RS_OPEN_REGION-ip-10-204-3-67:8120-2] handler.OpenRegionHandler:
Opening of region {ENCODED => c23e76525f0f025d51f74006db7903e2, NAME => 'cluster_test,,1415068223680.c23e76525f0f025d51f74006db7903e2.',
STARTKEY => '', ENDKEY => ''} failed, transitioning from OPENING to FAILED_OPEN in ZK,
expecting version 1
{noformat}



was (Author: apurtell):
Truncation is not working correctly. 

To reproduce:
# ./bin/hbase ltt -init_only
# ./bin/hbase shell
> truncate 'cluster_test'

TruncateTableHandler runs and claims to create region info:
{noformat}
2014-11-04 02:30:23,653 DEBUG [FifoRpcScheduler.handler1-thread-18] handler.TruncateTableHandler:
Table 'cluster_test' archived!
2014-11-04 02:30:23,679 DEBUG [FifoRpcScheduler.handler1-thread-18] util.FSTableDescriptors:
Wrote descriptor into: hdfs://ip-10-33-129-9.us-west-2.compute.internal:8020/hbase/.tmp/.tabledesc/.tableinfo.0000000001
2014-11-04 02:30:23,680 INFO  [FifoRpcScheduler.handler1-thread-18] handler.TruncateTableHandler:
Truncate will not preserve the regions
2014-11-04 02:30:23,681 INFO  [RegionOpenAndInitThread-cluster_test-1] regionserver.HRegion:
creating HRegion cluster_test HTD == 'cluster_test', {TABLE_ATTRIBUTES => {DURABILITY =>
'USE_DEFAULT'}, {NAME => 'test_cf', DATA_BLOCK_ENCODING => 'NONE', BLOOMFILTER =>
'ROW', REPLICATION_SCOPE => '0', COMPRESSION => 'NONE', VERSIONS => '1', TTL =>
'FOREVER', MIN_VERSIONS => '0', KEEP_DELETED_CELLS => 'false', BLOCKSIZE => '65536',
IN_MEMORY => 'false', BLOCKCACHE => 'true'} RootDir = hdfs://ip-10-33-129-9.us-west-2.compute.internal:8020/hbase/.tmp
Table name == cluster_test
2014-11-04 02:30:23,703 DEBUG [RegionOpenAndInitThread-cluster_test-1] regionserver.HRegion:
Instantiated cluster_test,,1415068223680.c23e76525f0f025d51f74006db7903e2.
2014-11-04 02:30:23,703 DEBUG [RegionOpenAndInitThread-cluster_test-1] regionserver.HRegion:
Closing cluster_test,,1415068223680.c23e76525f0f025d51f74006db7903e2.: disabling compactions
& flushes
2014-11-04 02:30:23,703 DEBUG [RegionOpenAndInitThread-cluster_test-1] regionserver.HRegion:
Updates disabled for region cluster_test,,1415068223680.c23e76525f0f025d51f74006db7903e2.
2014-11-04 02:30:23,703 INFO  [RegionOpenAndInitThread-cluster_test-1] regionserver.HRegion:
Closed cluster_test,,1415068223680.c23e76525f0f025d51f74006db7903e2.
2014-11-04 02:30:23,707 INFO  [FifoRpcScheduler.handler1-thread-18] catalog.MetaEditor: Added
1
2014-11-04 02:30:23,708 DEBUG [FifoRpcScheduler.handler1-thread-18] master.AssignmentManager:
Assigning 1 region(s) to ip-10-204-3-67.us-west-2.compute.internal,8120,1415068107305
2014-11-04 02:30:23,708 DEBUG [FifoRpcScheduler.handler1-thread-18] zookeeper.ZKAssign: master:8100-0x149780b006a010a,
quorum=ip-10-33-129-9.us-west-2.compute.internal:2181, baseZNode=/hbase Async create of unassigned
node c23e76525f0f025d51f74006db7903e2 with OFFLINE state
{noformat}

But the region will fail to open
{noformat}
2014-11-04 02:30:23,738 DEBUG [main-EventThread] zookeeper.ZooKeeperWatcher: master:8100-0x149780b006a010a,
quorum=ip-10-33-129-9.us-west-2.compute.internal:2181, baseZNode=/hbase Received ZooKeeper
Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/region-in-transition/c23e76525f0f025d51f74006db7903e2
2014-11-04 02:30:23,738 DEBUG [AM.ZK.Worker-pool2-t14] master.AssignmentManager: Handling
RS_ZK_REGION_FAILED_OPEN, server=ip-10-204-3-67.us-west-2.compute.internal,8120,1415068107305,
region=c23e76525f0f025d51f74006db7903e2, current_state={c23e76525f0f025d51f74006db7903e2 state=OPENING,
ts=1415068223734, server=ip-10-204-3-67.us-west-2.compute.internal,8120,1415068107305}
2014-11-04 02:30:23,738 INFO  [AM.ZK.Worker-pool2-t14] master.RegionStates: Transition {c23e76525f0f025d51f74006db7903e2
state=OPENING, ts=1415068223734, server=ip-10-204-3-67.us-west-2.compute.internal,8120,1415068107305}
to {c23e76525f0f025d51f74006db7903e2 state=CLOSED, ts=1415068223738, server=ip-10-204-3-67.us-west-2.compute.internal,8120,1415068107305}
2014-11-04 02:30:23,738 WARN  [AM.ZK.Worker-pool2-t14] master.AssignmentManager: Can't move
c23e76525f0f025d51f74006db7903e2, there is no destination server available.
2014-11-04 02:30:23,739 DEBUG [AM.ZK.Worker-pool2-t14] handler.ClosedRegionHandler: Handling
CLOSED event for c23e76525f0f025d51f74006db7903e2
{noformat}

Because this is happening on the RegionServer side:
{noformat}
2014-11-04 02:30:23,704 INFO  [PriorityRpcServer.handler=16,queue=0,port=8120] regionserver.HRegionServer:
Open cluster_test,,1415068223680.c23e76525f0f025d51f74006db7903e2.
2014-11-04 02:30:23,717 DEBUG [PriorityRpcServer.handler=16,queue=0,port=8120] util.FSTableDescriptors:
Exception during readTableDecriptor. Current table name = cluster_test
org.apache.hadoop.hbase.TableInfoMissingException: No table descriptor file under hdfs://ip-10-33-129-9.us-west-2.compute.internal:8020/hbase/data/default/cluster_test
	at org.apache.hadoop.hbase.util.FSTableDescriptors.getTableDescriptorFromFs(FSTableDescriptors.java:510)
	at org.apache.hadoop.hbase.util.FSTableDescriptors.getTableDescriptorFromFs(FSTableDescriptors.java:487)
	at org.apache.hadoop.hbase.util.FSTableDescriptors.get(FSTableDescriptors.java:173)
	at org.apache.hadoop.hbase.regionserver.HRegionServer.openRegion(HRegionServer.java:3826)
	at org.apache.hadoop.hbase.protobuf.generated.AdminProtos$AdminService$2.callBlockingMethod(AdminProtos.java:20164)
	at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2027)
	at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:108)
	at org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:110)
	at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:90)
	at java.lang.Thread.run(Thread.java:745)
2014-11-04 02:30:23,719 DEBUG [RS_OPEN_REGION-ip-10-204-3-67:8120-2] zookeeper.ZKAssign: regionserver:8120-0x149780b006a0105,
quorum=ec2-54-71-67-141.us-west-2.compute.amazonaws.com:2181, baseZNode=/hbase Transitioning
c23e76525f0f025d51f74006db7903e2 from M_ZK_REGION_OFFLINE to RS_ZK_REGION_OPENING
2014-11-04 02:30:23,723 DEBUG [RS_OPEN_REGION-ip-10-204-3-67:8120-2] zookeeper.ZKAssign: regionserver:8120-0x149780b006a0105,
quorum=ec2-54-71-67-141.us-west-2.compute.amazonaws.com:2181, baseZNode=/hbase Transitioned
node c23e76525f0f025d51f74006db7903e2 from M_ZK_REGION_OFFLINE to RS_ZK_REGION_OPENING
2014-11-04 02:30:23,723 DEBUG [RS_OPEN_REGION-ip-10-204-3-67:8120-2] regionserver.HRegion:
Opening region: {ENCODED => c23e76525f0f025d51f74006db7903e2, NAME => 'cluster_test,,1415068223680.c23e76525f0f025d51f74006db7903e2.',
STARTKEY => '', ENDKEY => ''}
2014-11-04 02:30:23,723 ERROR [RS_OPEN_REGION-ip-10-204-3-67:8120-2] handler.OpenRegionHandler:
Failed open of region=cluster_test,,1415068223680.c23e76525f0f025d51f74006db7903e2., starting
to roll back the global memstore size.
java.lang.IllegalStateException: Could not instantiate a region instance.
	at org.apache.hadoop.hbase.regionserver.HRegion.newHRegion(HRegion.java:4266)
	at org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:4574)
	at org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:4547)
	at org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:4503)
	at org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:4454)
	at org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.openRegion(OpenRegionHandler.java:482)
	at org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.process(OpenRegionHandler.java:145)
	at org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:128)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.reflect.InvocationTargetException
	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
	at org.apache.hadoop.hbase.regionserver.HRegion.newHRegion(HRegion.java:4263)
	... 10 more
Caused by: java.lang.IllegalArgumentException: Need table descriptor
	at org.apache.hadoop.hbase.regionserver.HRegion.<init>(HRegion.java:556)
	at org.apache.hadoop.hbase.regionserver.HRegion.<init>(HRegion.java:532)
	... 15 more
2014-11-04 02:30:23,724 INFO  [RS_OPEN_REGION-ip-10-204-3-67:8120-2] handler.OpenRegionHandler:
Opening of region {ENCODED => c23e76525f0f025d51f74006db7903e2, NAME => 'cluster_test,,1415068223680.c23e76525f0f025d51f74006db7903e2.',
STARTKEY => '', ENDKEY => ''} failed, transitioning from OPENING to FAILED_OPEN in ZK,
expecting version 1
{noformat}


> Truncate command does not preserve ACLs table
> ---------------------------------------------
>
>                 Key: HBASE-12142
>                 URL: https://issues.apache.org/jira/browse/HBASE-12142
>             Project: HBase
>          Issue Type: Bug
>    Affects Versions: 0.98.6
>            Reporter: Vandana Ayyalasomayajula
>            Assignee: Vandana Ayyalasomayajula
>            Priority: Minor
>              Labels: security
>             Fix For: 2.0.0, 0.98.8, 0.99.2
>
>         Attachments: HBASE-12142_0.patch, HBASE-12142_1.patch, HBASE-12142_2.patch, HBASE-12142_98.patch,
HBASE-12142_98_2.patch, HBASE-12142_branch_1.patch, HBASE-12142_branch_1.patch, HBASE-12142_master_addendum.patch
>
>
> The current truncate command does not preserve acls on a table. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message