hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vinayakumar B (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-9952) Expose FSNamesystem lock wait time as metrics
Date Wed, 16 Mar 2016 16:13:33 GMT

    [ https://issues.apache.org/jira/browse/HDFS-9952?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15197587#comment-15197587

Vinayakumar B commented on HDFS-9952:

Thanks [~daryn] for review.
bq. I'm not sure if metrics can be null, but I see some places doing the same check
{code}if (metrics != null) { // Metrics non-null only when used inside name node{code}
One of the comment says, It will be null if used outside namenode. i.e. In Tests directly.
bq. NameNode.getNameNodeMetrics() must be called once to assign, null checked, used
bq. Write lock wait should only be updated in the "if" for first write lock. Otherwise reentrant
locking (with near 0 time) will dilute the value.

bq. Most importantly and adamantly, MutableRate#add is synchronized in an extremely critical
code path which will destroy concurrent read ops. Synchronization will not scale with 100
threads doing 20-40k ops/sec, with bursts of 60-100k+ ops/ sec. I'm not a metrics expert but
this must use a non-synchronized metric.
Yes, I understand. But, AFAIK all other metrics provide just counter. Not AvgTime.
BTW, each and every RPC will have to go through multiple calls to {{MutableRate#add}} to update
different metrics.
ex: RPCMetrics have to update same instance for all RPCs.
If you still find this as problem, then we have to find some other way.

> Expose FSNamesystem lock wait time as metrics
> ---------------------------------------------
>                 Key: HDFS-9952
>                 URL: https://issues.apache.org/jira/browse/HDFS-9952
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: namenode
>            Reporter: Vinayakumar B
>            Assignee: Vinayakumar B
>         Attachments: HDFS-9952-01.patch
> Expose FSNameSystem's readlock() and writeLock() wait time as metrics.

This message was sent by Atlassian JIRA

View raw message