hadoop-hdfs-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Harsh J (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HDFS-3590) Print a WARN if the edit log sync period takes more than X time units
Date Mon, 02 Jul 2012 17:55:21 GMT
Harsh J created HDFS-3590:

             Summary: Print a WARN if the edit log sync period takes more than X time units
                 Key: HDFS-3590
                 URL: https://issues.apache.org/jira/browse/HDFS-3590
             Project: Hadoop HDFS
          Issue Type: Improvement
          Components: name-node
            Reporter: Harsh J
            Priority: Minor

If an logSync operation, which happens for calls such as FS#create() after the edit has been
made at the NN metadata, takes longer than X seconds (I'd say if it took more than a minute,
there's something really wrong with the volume it probably got stuck on), we should log a
WARN with the volume that may have particularly caused it. This helps track down, if an NN
runs with multiple NFS volumes, which particular volume may have caused it, as there's no
per-NN-dir metrics of any kind.

I ran into a situation today where a hard-mounted NFS point hung for over X minutes but there
was no indication in NN's logs after it recovered (recovering so late caused its own slew
of issues for which I'll file other improvement JIRAs) that such an event happened, aside
of the Sync (Journal Sync) metric spiking with the elapsed sync time value rising up. A log
would have helped save time investigating this, and possibly would have also pin-pointed the
bad location more accurately.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message