hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Erik Krogen (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (HDFS-10713) Throttle FsNameSystem lock warnings
Date Tue, 27 Sep 2016 17:14:22 GMT

    [ https://issues.apache.org/jira/browse/HDFS-10713?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15526786#comment-15526786
] 

Erik Krogen edited comment on HDFS-10713 at 9/27/16 5:13 PM:
-------------------------------------------------------------

[~arpitagarwal], I'm confident that my HDFS-10817 patch would not have added latency cost
since it is just manipulating a ThreadLocal but I agree that there is potential for both this
patch and HDFS-10475, which I am working on, to add overheads (and this is part of why the
metric logic in HDFS-10475 is disabled by default). I was hoping to do a little bit of microbenchmarking
as part of HDFS-10475; I could include this patch in the comparison. 


was (Author: xkrogen):
[~arpitagarwal], I'm confident that my HDFS-10817 patch would not have added latency cost
since it is just manipulating a ThreadLocal but I agree that there is potential for both this
patch and HDFS-10475, which I am working on, to add overheads. I was hoping to do a little
bit of microbenchmarking as part of HDFS-10475; I could include this patch in the comparison.


> Throttle FsNameSystem lock warnings
> -----------------------------------
>
>                 Key: HDFS-10713
>                 URL: https://issues.apache.org/jira/browse/HDFS-10713
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: logging, namenode
>            Reporter: Arpit Agarwal
>            Assignee: Hanisha Koneru
>             Fix For: 2.8.0, 3.0.0-alpha2
>
>         Attachments: HDFS-10713.000.patch, HDFS-10713.001.patch, HDFS-10713.002.patch,
HDFS-10713.003.patch, HDFS-10713.004.patch, HDFS-10713.005.patch, HDFS-10713.006.patch, HDFS-10713.007.patch,
HDFS-10713.008.patch, HDFS-10713.009.patch, HDFS-10713.010.patch, HDFS-10713.011.patch, HDFS-10713.012.patch,
HDFS-10713.branch-2.000.patch
>
>
> The NameNode logs a message if the FSNamesystem write lock is held by a thread for over
1 second. These messages can be throttled to at one most one per x minutes to avoid potentially
filling up NN logs. We can also log the number of suppressed notices since the last log message.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org


Mime
View raw message