flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From static-max <flasha...@googlemail.com>
Subject Re: PathIsNotEmptyDirectoryException in Namenode HDFS log when using Jobmanager HA in YARN
Date Mon, 21 Nov 2016 20:51:08 GMT
Hi Stephan,

it's not a problem, but makes finding other errors on my NameNode
complicated as I have this error message every minute.
Can't we just delete the directory recursively?

Regards,
Max

2016-10-11 17:59 GMT+02:00 Stephan Ewen <sewen@apache.org>:

> Hi!
>
> I think to some extend this is expected. There is some cleanup code that
> deletes files and then  issues parent directory remove requests. It relies
> on the fact that the parent directory is only removed if it is empty (after
> the last file was deleted).
>
> Is this a problem right now, or just a confusing behavior?
>
> Greetings,
> Stephan
>
>
> On Tue, Oct 11, 2016 at 5:25 PM, static-max <flashacid@googlemail.com>
> wrote:
>
>> Hi,
>>
>> I get many (multiple times per minute) errors in my Namenode HDFS logfile:
>>
>> 2016-10-11 17:17:07,596 INFO  ipc.Server (Server.java:logException(2401))
>> - IPC Server handler 295 on 8020, call org.apache.hadoop.hdfs.protocol.ClientProtocol.delete
>> from datanode1:34872 Call#2361 Retry#0
>> org.apache.hadoop.fs.PathIsNotEmptyDirectoryException: `/flink/recovery
>> is non empty': Directory is not empty
>>         at org.apache.hadoop.hdfs.server.namenode.FSDirDeleteOp.delete(
>> FSDirDeleteOp.java:89)
>>         at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.delete(F
>> SNamesystem.java:3829)
>>         at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.del
>> ete(NameNodeRpcServer.java:1071)
>>         at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServ
>> erSideTranslatorPB.delete(ClientNamenodeProtocolServerSideTr
>> anslatorPB.java:619)
>>         at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocol
>> Protos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNam
>> enodeProtocolProtos.java)
>>         at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcIn
>> voker.call(ProtobufRpcEngine.java:640)
>>         at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:982)
>>         at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2313)
>>         at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2309)
>>         at java.security.AccessController.doPrivileged(Native Method)
>>         at javax.security.auth.Subject.doAs(Subject.java:422)
>>         at org.apache.hadoop.security.UserGroupInformation.doAs(UserGro
>> upInformation.java:1724)
>>         at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2307)
>>
>> That is the directory I configured for Jobmanager HA. I deleted it before
>> starting the YARN session but that did not help. The folder gets created by
>> Flink without problems.
>>
>> I'm using latest Flink Master (Commit: 6731ec1) and build it for Hadoop
>> 2.7.3.
>>
>> Any idea is highly appreciated. Thanks a lot!
>>
>
>

Mime
View raw message