hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Lars Hofhansl (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-1783) Ability for HDFS client to write replicas in parallel
Date Fri, 08 Jun 2012 07:58:25 GMT

    [ https://issues.apache.org/jira/browse/HDFS-1783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13291612#comment-13291612
] 

Lars Hofhansl commented on HDFS-1783:
-------------------------------------

Thanks Todd. I see your point. I'm still oversees until the end of the month with no physical
access to a cluster. Ram and Andy said, that they might get a chance to do some performance
test before that. (It's hard to beat the pipelining on throughput, so I only expect latency
to be improved.)

As for the complexity, I find it manageable... The pipelining as such has not changed, only
that the client opens up N pipelines on length 1.
Once this change is in, one could get fancier (for example 2 pipelines of length 2 for 4 replicas,
etc, or maybe we could open pipelines to multiple clusters, etc).

                
> Ability for HDFS client to write replicas in parallel
> -----------------------------------------------------
>
>                 Key: HDFS-1783
>                 URL: https://issues.apache.org/jira/browse/HDFS-1783
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: hdfs client
>            Reporter: dhruba borthakur
>            Assignee: Lars Hofhansl
>         Attachments: HDFS-1783-trunk-v2.patch, HDFS-1783-trunk-v3.patch, HDFS-1783-trunk-v4.patch,
HDFS-1783-trunk.patch
>
>
> The current implementation of HDFS pipelines the writes to the three replicas. This introduces
some latency for realtime latency sensitive applications. An alternate implementation that
allows the client to write all replicas in parallel gives much better response times to these
applications. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message