hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ming Ma (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-10629) Federation Router
Date Mon, 03 Oct 2016 21:30:21 GMT

    [ https://issues.apache.org/jira/browse/HDFS-10629?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15543484#comment-15543484
] 

Ming Ma commented on HDFS-10629:
--------------------------------

Thanks [~jakace] for the explanation and new patch.

bq. The status report is used by the NN heartbeat service to register/discover a NN and store
it's state in the store.
Got it. How about dynamic stats such as numOfFiles which are only used by rebalancer? We could
consider adding dynamic stats later when we agree to have rebalancer fetch such stats from
store/router instead of getting them directly from actual namenodes.

bq. Other APIs such as mkdir were modfiied to check all locations to ensure the directory
doesn't exist before permitting a create.

* Wonder why mkdirs check all locations during validation, while only use the first location
for the actual mkdirs call.
* Can you add test cases to cover this scenario?
* It will be useful to document the scenarios for multiple destination, what specific mount
table settings this will cover, nfly, mergeFs, etc.
* It will have extra RPC calls for all scenarios? It might be useful to check {PathLocation#hasMultipleDestinations}
first. If there is only one destination, getFileInfo can be skipped.  

* Should rename method do some validation to make sure src ns != destination ns?

* Router#startTime and Router#routerId not used. Let us add them later when they are required.
There are other unused methods in other files. Can you check if they are needed?

* It seems abandonBlock just needs to refer to one name service based on the block id, thus
can call invokeSingle?

* There are still other javadoc and code mismatch in several files.

* failIfLocked isn't used. What is for?

> Federation Router
> -----------------
>
>                 Key: HDFS-10629
>                 URL: https://issues.apache.org/jira/browse/HDFS-10629
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: hdfs
>            Reporter: Inigo Goiri
>            Assignee: Jason Kace
>         Attachments: HDFS-10629-HDFS-10467-002.patch, HDFS-10629-HDFS-10467-003.patch,
HDFS-10629-HDFS-10467-004.patch, HDFS-10629.000.patch, HDFS-10629.001.patch
>
>
> Component that routes calls from the clients to the right Namespace. It implements {{ClientProtocol}}.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org


Mime
View raw message