hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Colin Patrick McCabe (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (HDFS-7698) Fix locking on HDFS read statistics and add a method for clearing them.
Date Thu, 29 Jan 2015 03:11:35 GMT

    [ https://issues.apache.org/jira/browse/HDFS-7698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14296318#comment-14296318
] 

Colin Patrick McCabe edited comment on HDFS-7698 at 1/29/15 3:11 AM:
---------------------------------------------------------------------

* It is not necessary (or useful) to synchronize {{HdfsDataInputStream#getReadStatistics}}
on the {{HdfsDataInputStream}} lock.  getReadStatistics is synchronized inside {{DFSInputStream}}
already, and the methods modifying the statistics are not synchronized on {{HdfsDataInputStream}}.

* Add a {{ReadStatistics#clear}} method which clears the stats.

* Add {{libhdfs}} support for clearing the read statistics.

* The {{ReadStatistics}} need to be synchronized on {{infoLock}}, not on the positional lock.
 {{pread}} modifies these statistics, and it shouldn't have to take the positional lock to
do that.  This could have resulted in lost updates or incorrect read statistics being read
when doing {{preads}} (but not when doing regular reads under the positional lock).


was (Author: cmccabe):
* It is not necessary (or useful) to synchronized {{HdfsDataInputStream#getReadStatistics}}.
 getReadStatistics is synchronized inside {{DFSInputStream}} already, and the methods modifying
the statistics are not synchronized on {{HdfsDataInputStream}}.

* Add a {{ReadStatistics#clear}} method which clears the stats.

* Add {{libhdfs}} support for clearing the read statistics.

* The {{ReadStatistics}} need to be synchronized on {{infoLock}}, not on the positional lock.
 {{pread}} modifies these statistics, and it shouldn't have to take the positional lock to
do that.  This could have resulted in lost updates or incorrect read statistics being read
when doing {{preads}} (but not when doing regular reads under the positional lock).

> Fix locking on HDFS read statistics and add a method for clearing them.
> -----------------------------------------------------------------------
>
>                 Key: HDFS-7698
>                 URL: https://issues.apache.org/jira/browse/HDFS-7698
>             Project: Hadoop HDFS
>          Issue Type: Bug
>    Affects Versions: 2.7.0
>            Reporter: Colin Patrick McCabe
>            Assignee: Colin Patrick McCabe
>         Attachments: HDFS-7698.001.patch
>
>
> Fix locking on HDFS read statistics and add a method for clearing them.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message