hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Carrey Zhan (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HDFS-7609) startup used too much time to load edits
Date Wed, 14 Jan 2015 08:38:34 GMT
Carrey Zhan created HDFS-7609:

             Summary: startup used too much time to load edits
                 Key: HDFS-7609
                 URL: https://issues.apache.org/jira/browse/HDFS-7609
             Project: Hadoop HDFS
          Issue Type: Improvement
          Components: namenode
    Affects Versions: 2.2.0
            Reporter: Carrey Zhan

One day my namenode crashed because of two journal node timed out at the same time under very
high load, leaving behind about 100 million transactions in edits log.(I still have no idea
why they were not rolled into fsimage.)
I tryed to restart namenode, but it showed that almost 20 hours would be needed before finish,
and it was loading fsedits most of the time. I also tryed to restart namenode in recover mode,
the loading speed had no different.
I looked into the stack trace, judged that it is caused by the retry cache. So I set dfs.namenode.enable.retrycache
to false, the restart process finished in half an hour.

I think the retry cached is useless during startup, at least during recover process.

This message was sent by Atlassian JIRA

View raw message