hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Lars Hofhansl (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-6735) A minor optimization to avoid pread() be blocked by read() inside the same DFSInputStream
Date Tue, 25 Nov 2014 01:20:14 GMT

    [ https://issues.apache.org/jira/browse/HDFS-6735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14223901#comment-14223901
] 

Lars Hofhansl commented on HDFS-6735:
-------------------------------------

The remaining findbugs warning is due to cachingStrategy. I am 100% sure that the locking
is correct, every single reference to cachingStrategy is guarded by the infoLock.
This should good to go (happy to squash the bogus findbugs warning if somebody has a suggestion
how).

The findbugs website states this for IS2_INCONSISTENT_SYNC:
{quote}
Note that there are various sources of inaccuracy in this detector; for example, the detector
cannot statically detect all situations in which a lock is held.  Also, even when the detector
is accurate in distinguishing locked vs. unlocked accesses, the code in question may still
be correct.
{quote}


> A minor optimization to avoid pread() be blocked by read() inside the same DFSInputStream
> -----------------------------------------------------------------------------------------
>
>                 Key: HDFS-6735
>                 URL: https://issues.apache.org/jira/browse/HDFS-6735
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: hdfs-client
>    Affects Versions: 3.0.0
>            Reporter: Liang Xie
>            Assignee: Liang Xie
>         Attachments: HDFS-6735-v2.txt, HDFS-6735-v3.txt, HDFS-6735-v4.txt, HDFS-6735-v5.txt,
HDFS-6735.txt
>
>
> In current DFSInputStream impl, there're a couple of coarser-grained locks in read/pread
path, and it has became a HBase read latency pain point so far. In HDFS-6698, i made a minor
patch against the first encourtered lock, around getFileLength, in deed, after reading code
and testing, it shows still other locks we could improve.
> In this jira, i'll make a patch against other locks, and a simple test case to show the
issue and the improved result.
> This is important for HBase application, since in current HFile read path, we issue all
read()/pread() requests in the same DFSInputStream for one HFile. (Multi streams solution
is another story i had a plan to do, but probably will take more time than i expected)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message