hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Yiqun Lin (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-12615) Router-based HDFS federation phase 2
Date Wed, 11 Apr 2018 03:27:00 GMT

    [ https://issues.apache.org/jira/browse/HDFS-12615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16433344#comment-16433344
] 

Yiqun Lin commented on HDFS-12615:
----------------------------------

It looks amazing that we have fixed and made so many improvements for RBF. Thanks all the
contributors!
 Just revisiting the previous plan of RBF phase 2 and ensuring we are in the same direction:
{quote}At this point the only critical task missing is enabling security (mainly HDFS-12284),
Sherwood Zheng has been doing progress there and we are only missing the delegation tokens.
 The other tasks are mostly done we should try to complete them, the most important in my
opinion:
 * HDFS-12773 to do a proper HDFS State Store.
 * HDFS-12792 to have complete unit tests.

Then there are a bunch of open topics:
 # Rebalancer of data across subclusters. We have some internal prototype based on DistCp
that I shared with Wei Yan (we could also include moving blocks across blockpools). We could
start a separate umbrella for this.
 # DNs interacting with the Routers. I recently opened HDFS-13098 to discuss this but this
could.
 # Spreading a mount point across multiple mount points. This is similar to merge mount points
and we have most of it working in production.{quote}
The #3 (*Spreading a mount point across multiple mount points*) mentioned in open topics has
been implemented now (the doc of this (HDFS-13237) will be merged soon ). HDFS-12773 and HDFS-12792
are both merged.

Now there are two major works in phase 2:
 * RBF security tasks: HDFS-12284 and HDFS-13358.
 * DBMS State Store implementation: HDFS-13245.

[~elgoiri], I have some questions in my mind:
 * Besides this, any other thing we need to implement?
 * What's the next plan in RBF phase2?
 * What's the plan of RBF Rebalancer? It looks a useful tool for users.

> Router-based HDFS federation phase 2
> ------------------------------------
>
>                 Key: HDFS-12615
>                 URL: https://issues.apache.org/jira/browse/HDFS-12615
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>            Reporter: Íñigo Goiri
>            Assignee: Íñigo Goiri
>            Priority: Major
>              Labels: RBF
>
> This umbrella JIRA tracks set of improvements over the Router-based HDFS federation (HDFS-10467).



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org


Mime
View raw message