hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From 陈 建平Chen Jianping <chenjianp...@agora.io>
Subject YARN process with status D
Date Mon, 20 Mar 2017 14:00:08 GMT
Hi group,

I met with a problem that sometimes when I running my MapReduce job, some YARN process will
go to status D, which means uninterruptable status. In this case this YARN process cannot
be killed, and only reboot server can recover. Since in this situation, some command such
as ps, reboot, lsof will be stuck and I cannot investigate more. In my observation this occur
on the server with 8T hard disk.

Here is the output from top command.
8382 yarn      20   0 3456408 277324  28040 D   0.0  0.2  11:16.21 java

Here is my environment.

·         Cloudera CDH 5.9.0

·         The disk on server is 8T volume

·         My MapReduce job is getting data from HBase, and store the result in HBase

I have used fsck command to check the disk, and no error found with the disk. I am not sure
if there is some configuration to be tuned for large-volume disk for Hadoop / YARN. Is there
any idea about this issue?
Thanks in advance.

Thanks,
Eric
Mime
View raw message