hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Arpit Agarwal (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-10742) Measurement of lock held time in FsDatasetImpl
Date Tue, 30 Aug 2016 20:00:26 GMT

    [ https://issues.apache.org/jira/browse/HDFS-10742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15449980#comment-15449980
] 

Arpit Agarwal commented on HDFS-10742:
--------------------------------------

Hi [~vagarychen], a couple of comments:
# {{check()}} should have minimal overhead if the lock was not held for more lockWarningThresholdMs
which will be the common case. We can simplify the patch quite a bit and just add logging
along with the caller stack trace if the lock was held for too long. We can extend it later
if we want to add performance statistics but we'd have to do it in a way that avoids object
allocations.
# InstrumentedLock should probably not extend the concrete class AutoCloseableLock. We can
make InstrumentedLock a separate class. or alternatively AutoCloseableLock a Java interface
with multiple implementations.

bq. but in the context of a patch generating thread stacks and maintaining persistent maps
of timestamps, it's absurd to debate any of this.
Yes I agree. We should avoid that overhead.

> Measurement of lock held time in FsDatasetImpl
> ----------------------------------------------
>
>                 Key: HDFS-10742
>                 URL: https://issues.apache.org/jira/browse/HDFS-10742
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: datanode
>    Affects Versions: 3.0.0-alpha2
>            Reporter: Chen Liang
>            Assignee: Chen Liang
>         Attachments: HDFS-10742.001.patch, HDFS-10742.002.patch, HDFS-10742.003.patch,
HDFS-10742.004.patch, HDFS-10742.005.patch, HDFS-10742.006.patch, HDFS-10742.007.patch, HDFS-10742.008.patch
>
>
> This JIRA proposes to measure the time the of lock of {{FsDatasetImpl}} is held by a
thread. Doing so will allow us to measure lock statistics.
> This can be done by extending the {{AutoCloseableLock}} lock object in {{FsDatasetImpl}}.
In the future we can also consider replacing the lock with a read-write lock.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org


Mime
View raw message