carbondata-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Venkata Ramana G (JIRA)" <j...@apache.org>
Subject [jira] [Resolved] (CARBONDATA-1277) Dictionary generation failure if there is failure in closing output stream in HDFS
Date Tue, 11 Jul 2017 12:34:00 GMT

     [ https://issues.apache.org/jira/browse/CARBONDATA-1277?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Venkata Ramana G resolved CARBONDATA-1277.
------------------------------------------
    Resolution: Fixed

> Dictionary generation failure if there is failure in closing output stream in HDFS
> ----------------------------------------------------------------------------------
>
>                 Key: CARBONDATA-1277
>                 URL: https://issues.apache.org/jira/browse/CARBONDATA-1277
>             Project: CarbonData
>          Issue Type: Bug
>            Reporter: Manish Gupta
>            Assignee: Manish Gupta
>             Fix For: 1.2.0
>
>          Time Spent: 2h 40m
>  Remaining Estimate: 0h
>
> If there is any failure while closing the output stream of dictionary file in HDFS then
on next data load, update or insert into operation dictionary generation fails. This is because
we open the dictionary file in append mode and when we try to get the output stream for that
file HDFS throws an exception that Lease is already acquired by some other client. Exception
trace as below.
> java.io.IOException: Failed to APPEND_FILE /user/hive/warehouse/carbon.store/test/t12/Metadata/010497af-1833-4804-a7ec-849ab7b9bf10.dictmeta
for DFSClient_NONMAPREDUCE_-1904161438_159 on 172.168.100.212 because lease recovery is in
progress. Try again later.
> at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.recoverLeaseInternal(FSNamesystem.java:2901)
> at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.appendFileInternal(FSNamesystem.java:2655)
> at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.appendFileInt(FSNamesystem.java:2968)
> at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.appendFile(FSNamesystem.java:2937)
> at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.append(NameNodeRpcServer.java:776)
> at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.append(ClientNamenodeProtocolServerSideTranslatorPB.java:458)
> at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
> at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:616)



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message