hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vinayakumar B (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-10467) Router-based HDFS federation
Date Mon, 18 Jul 2016 10:50:20 GMT

    [ https://issues.apache.org/jira/browse/HDFS-10467?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15382072#comment-15382072

Vinayakumar B commented on HDFS-10467:

This will be nice addition to Federation. 

Apparently, I was also working on similar feature, which almost has same design.

Design looks great, I have some comments though.

{quote}3.3.2 Namenode heartbeat HA
For high availability and flexibility, multiple Routers can monitor the same Namenode and
heartbeat the information to the State Store{quote}
{quote}If a Router tries to contact the active Namenode but is unable to do it, the Router
will try the other
Namenodes in the subcluster.{quote}
Why cant we use DFSClient with HA here directly? Like how current clients connects to HA,
each subcluster can be connected to a subcluster using a HA configured DFSClient. DFSClient
itself will handle switching between NNs in case of failover. This DFSClient can be kept as-is
and re-used later for next requests on same subcluster. So it should know the current active
By doing this, there will not be any need of Heartbeat between Router and NameNode to monitor
the NameNode status.

bq. MountTable
How about supporting a default Mount-point for '/' as well. Could be optional also? Instead
of rejecting requests for the paths which doesnt match with other other mountpoints.
There might be some usecases where there might be multiple first level directories other than
mounted path. Which could go under /.
Similar to Linux's FileSystem mounts.

I will try to review the code on sub jiras.

> Router-based HDFS federation
> ----------------------------
>                 Key: HDFS-10467
>                 URL: https://issues.apache.org/jira/browse/HDFS-10467
>             Project: Hadoop HDFS
>          Issue Type: New Feature
>          Components: fs
>    Affects Versions: 2.7.2
>            Reporter: Inigo Goiri
>            Assignee: Inigo Goiri
>         Attachments: HDFS Router Federation.pdf, HDFS-10467.PoC.001.patch, HDFS-10467.PoC.patch,
> Add a Router to provide a federated view of multiple HDFS clusters.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org

View raw message