hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Lars Hofhansl (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HDFS-6735) A minor optimization to avoid pread() be blocked by read() inside the same DFSInputStream
Date Thu, 27 Nov 2014 00:31:14 GMT

     [ https://issues.apache.org/jira/browse/HDFS-6735?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Lars Hofhansl updated HDFS-6735:
--------------------------------
    Attachment: HDFS-6735-v6.txt

Updated patch.
The findbugs tweak is still necessary. Locking was correct before, findbugs does not seem
to realize that all references to cachingStrategy is always guarded by the infoLock.

I'll run a 2.4.1 version of this patch against HBase again.


> A minor optimization to avoid pread() be blocked by read() inside the same DFSInputStream
> -----------------------------------------------------------------------------------------
>
>                 Key: HDFS-6735
>                 URL: https://issues.apache.org/jira/browse/HDFS-6735
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: hdfs-client
>    Affects Versions: 3.0.0
>            Reporter: Liang Xie
>            Assignee: Liang Xie
>         Attachments: HDFS-6735-v2.txt, HDFS-6735-v3.txt, HDFS-6735-v4.txt, HDFS-6735-v5.txt,
HDFS-6735-v6.txt, HDFS-6735-v6.txt, HDFS-6735.txt
>
>
> In current DFSInputStream impl, there're a couple of coarser-grained locks in read/pread
path, and it has became a HBase read latency pain point so far. In HDFS-6698, i made a minor
patch against the first encourtered lock, around getFileLength, in deed, after reading code
and testing, it shows still other locks we could improve.
> In this jira, i'll make a patch against other locks, and a simple test case to show the
issue and the improved result.
> This is important for HBase application, since in current HFile read path, we issue all
read()/pread() requests in the same DFSInputStream for one HFile. (Multi streams solution
is another story i had a plan to do, but probably will take more time than i expected)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message