hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Daniel Dai (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HIVE-18530) Replication should skip MM table (for now)
Date Wed, 24 Jan 2018 23:22:00 GMT

     [ https://issues.apache.org/jira/browse/HIVE-18530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Daniel Dai updated HIVE-18530:
------------------------------
    Attachment: HIVE-18530.1.patch

> Replication should skip MM table (for now)
> ------------------------------------------
>
>                 Key: HIVE-18530
>                 URL: https://issues.apache.org/jira/browse/HIVE-18530
>             Project: Hive
>          Issue Type: Bug
>          Components: repl
>            Reporter: Daniel Dai
>            Assignee: Daniel Dai
>            Priority: Major
>         Attachments: HIVE-18530.1.patch
>
>
> Currently replication cannot handle transactional table (including MM table) until HIVE-18320.
HIVE-17504 skips table with transactional=true explicitly. HIVE-18352 changes the logic to
use AcidUtils.isAcidTable for the same purpose. However, isAcidTable returns false for mm
table, thus Hive still dump mm table during replication.
> Here is an error message during dump mm table:
> {code}
> ERROR : FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.repl.ReplDumpTask.
java.io.FileNotFoundException: Path is not a file: /apps/hive/warehouse/testrepldb5.db/test1/delta_0000261_0000261_0000
> 	at org.apache.hadoop.hdfs.server.namenode.INodeFile.valueOf(INodeFile.java:89)
> 	at org.apache.hadoop.hdfs.server.namenode.INodeFile.valueOf(INodeFile.java:75)
> 	at org.apache.hadoop.hdfs.server.namenode.FSDirStatAndListingOp.getBlockLocations(FSDirStatAndListingOp.java:154)
> 	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocations(FSNamesystem.java:1920)
> 	at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.getBlockLocations(NameNodeRpcServer.java:731)
> 	at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.getBlockLocations(ClientNamenodeProtocolServerSideTranslatorPB.java:424)
> 	at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
> 	at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:523)
> 	at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:991)
> 	at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:869)
> 	at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:815)
> 	at java.security.AccessController.doPrivileged(Native Method)
> 	at javax.security.auth.Subject.doAs(Subject.java:422)
> 	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1965)
> 	at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2675)
> INFO  : Completed executing command(queryId=hive_20180119203438_293813df-7630-47fa-bc30-5ef7cbb42842);
Time taken: 1.119 seconds
> Error: Error while processing statement: FAILED: Execution Error, return code 1 from
org.apache.hadoop.hive.ql.exec.repl.ReplDumpTask. java.io.FileNotFoundException: Path is not
a file: /apps/hive/warehouse/testrepldb5.db/test1/delta_0000261_0000261_0000
> 	at org.apache.hadoop.hdfs.server.namenode.INodeFile.valueOf(INodeFile.java:89)
> 	at org.apache.hadoop.hdfs.server.namenode.INodeFile.valueOf(INodeFile.java:75)
> 	at org.apache.hadoop.hdfs.server.namenode.FSDirStatAndListingOp.getBlockLocations(FSDirStatAndListingOp.java:154)
> 	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocations(FSNamesystem.java:1920)
> 	at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.getBlockLocations(NameNodeRpcServer.java:731)
> 	at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.getBlockLocations(ClientNamenodeProtocolServerSideTranslatorPB.java:424)
> 	at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
> 	at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:523)
> 	at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:991)
> 	at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:869)
> 	at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:815)
> 	at java.security.AccessController.doPrivileged(Native Method)
> 	at javax.security.auth.Subject.doAs(Subject.java:422)
> 	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1965)
> 	at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2675) (state=08S01,code=1)
> 0: jdbc:hive2://ctr-e137-1514896590304-25219-> Closing: 0: jdbc:hive2://ctr-e137-1514896590304-25219-02-000005.hwx.site:2181,ctr-e137-1514896590304-25219-02-000012.hwx.site:2181,ctr-e137-1514896590304-25219-02-000009.hwx.site:2181,ctr-e137-1514896590304-25219-02-000004.hwx.site:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2;principal=hive/_HOST@EXAMPLE.COM
> {code}
> We shall switch to use AcidUtils.isTransactionalTable.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message