hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Inigo Goiri (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-10629) Federation Router
Date Tue, 31 Jan 2017 22:35:52 GMT

    [ https://issues.apache.org/jira/browse/HDFS-10629?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15847662#comment-15847662

Inigo Goiri commented on HDFS-10629:

[~grey], I just uploaded a figure showing the latency of the version we tested 4 months ago.
Executing listing requests against one Namenode we got a latency of a little less than 1 ms
and a saturation point of around 40k requests per seconds.
Adding a Router in front, the latency almost double but still under 2ms; the saturation point
was a little worse too.
When comparing to 4 Namenodes and 12 Router for 4 Namenodes, both latencies and saturation
points are worse but much better than for a single subcluster.

Note that this is the worst case for us, as this was one of the cheapest operations.
When reading data, the overhead is negligible.
In addition, we have done some optimizations but the order of magnitude should be the same.

> Federation Router
> -----------------
>                 Key: HDFS-10629
>                 URL: https://issues.apache.org/jira/browse/HDFS-10629
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: hdfs
>            Reporter: Inigo Goiri
>            Assignee: Jason Kace
>         Attachments: HDFS-10629.000.patch, HDFS-10629.001.patch, HDFS-10629-HDFS-10467-002.patch,
HDFS-10629-HDFS-10467-003.patch, HDFS-10629-HDFS-10467-004.patch, HDFS-10629-HDFS-10467-005.patch,
HDFS-10629-HDFS-10467-006.patch, HDFS-10629-HDFS-10467-007.patch, routerlatency.png
> Component that routes calls from the clients to the right Namespace. It implements {{ClientProtocol}}.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org

View raw message