hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Xue Liu (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-14406) Add per user RPC Processing time
Date Wed, 10 Apr 2019 23:31:00 GMT

    [ https://issues.apache.org/jira/browse/HDFS-14406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16814938#comment-16814938
] 

Xue Liu commented on HDFS-14406:
--------------------------------


Thanks for the comments and suggestions! New patch available, makes the user RPC metrics optional.


Agree on that we are now facing the bottleneck of lock waiting time, that would have an unfair
effect on per user RPC processing time. In this case, the difference of RPC processing time
may not be that big. However I do like to think of this metric as general indicator, that
we can probably expose to users. Also, it might be good we have some generosity here, so that
other ipc like Yarn may also use it.  

[~elgoiri][~daryn] Thanks for the suggestion, I have fixed checkstyles and made it optional.
[~xkrogen], the work on cost-based FCQ seems interesting, which is something we also want
in our prod cluster in the future. From your perspective, would it be interesting if we can
have a more detailed metrics, say per user per method rpc processing time?

> Add per user RPC Processing time
> --------------------------------
>
>                 Key: HDFS-14406
>                 URL: https://issues.apache.org/jira/browse/HDFS-14406
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>    Affects Versions: 3.2.0
>            Reporter: Xue Liu
>            Assignee: Xue Liu
>            Priority: Minor
>             Fix For: 3.2.0
>
>         Attachments: HDFS-14406.001.patch, HDFS-14406.002.patch
>
>
> For a shared cluster we would want to separate users' resources, as well as having our
metrics reflecting on the usage, latency, etc, for each user. 
> This JIRA aims to add per user RPC processing time metrics and expose it via JMX.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org


Mime
View raw message