hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Raghu Angadi (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-3232) Datanodes time out
Date Wed, 14 May 2008 15:31:55 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-3232?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12596792#action_12596792
] 

Raghu Angadi commented on HADOOP-3232:
--------------------------------------


> Scheduling it with an Executor would still have a running thread that keeps track of
the scheduling, no? 
That is implementation dependent. JavaDoc does not say. There might be just one thread that
handles many such tasks. We at least let the implementation of Executors to optimize it.

DU looks like simple utility to users. If they do create these multiple times, each when ever
they need, they will be surprised to see threads hanging around. In that sense, it might be
better to add a "start()" method so that it is explicit to them that a thread might be started
and it needs to be shutdown(). 

+1 over all.

> Datanodes time out
> ------------------
>
>                 Key: HADOOP-3232
>                 URL: https://issues.apache.org/jira/browse/HADOOP-3232
>             Project: Hadoop Core
>          Issue Type: Bug
>          Components: dfs
>    Affects Versions: 0.16.2, 0.16.3, 0.16.4
>         Environment: 10 node cluster + 1 namenode
>            Reporter: Johan Oskarsson
>            Assignee: Johan Oskarsson
>            Priority: Critical
>             Fix For: 0.18.0
>
>         Attachments: du-nonblocking-v1.patch, du-nonblocking-v2-trunk.patch, du-nonblocking-v4-trunk.patch,
du-nonblocking-v5-trunk.patch, hadoop-hadoop-datanode-new.log, hadoop-hadoop-datanode-new.out,
hadoop-hadoop-datanode.out, hadoop-hadoop-namenode-master2.out
>
>
> I recently upgraded to 0.16.2 from 0.15.2 on our 10 node cluster.
> Unfortunately we're seeing datanode timeout issues. In previous versions we've often
seen in the nn webui that one or two datanodes "last contact" goes from the usual 0-3 sec
to ~200-300 before it drops down to 0 again.
> This causes mild discomfort but the big problems appear when all nodes do this at once,
as happened a few times after the upgrade.
> It was suggested that this could be due to namenode garbage collection, but looking at
the gc log output it doesn't seem to be the case.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message