hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mohammad Arshad (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-13443) RBF: Update mount table cache immediately after changing (add/update/remove) mount table entries.
Date Tue, 24 Apr 2018 14:34:01 GMT

    [ https://issues.apache.org/jira/browse/HDFS-13443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16449967#comment-16449967

Mohammad Arshad commented on HDFS-13443:

Thanks [~elgoiri] for your review comments.
 # Regarding local and remote routers: Suppose there are three routers R1, R2 and R3. User
want to add new mount table entry. He will connect to only one router, not all the routers.
Suppose He connects to R1 and calls add mount table entry API or CLI. Now in this context
R1 is local router, R2 and R3 are remote routers. Because add mount table entry is invoked
on R1, R1 will update the cache locally it need not to make RPC call. But R1 will make RPC
calls to update cache on R2 and R3.
 # Corrected.
 # Corrected
 # getHostPortString is there in org.apache.hadoop.net.NetUtils. Not using this method as
I need to replace with actual machine IP when router is getting registered. If a router
registers itself with IP, other routers can not contact it.
 # Corrected
 # RouterClient object creation is very straight forward. I think it should be OK to create
this way.
 # Suppose a router is stopped, but its state not yet updated in state store. it is in RUNNING
state.In this case RouterClient will fail and I think it is OK to ignore this router.
 # Corrected
 # These test cases are more like functional test cases, validating the functionality end
to end,without using any mock. shall i change it or it is OK to have like this?
 # Corrected

> RBF: Update mount table cache immediately after changing (add/update/remove) mount table
> -------------------------------------------------------------------------------------------------
>                 Key: HDFS-13443
>                 URL: https://issues.apache.org/jira/browse/HDFS-13443
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: fs
>            Reporter: Mohammad Arshad
>            Assignee: Mohammad Arshad
>            Priority: Major
>              Labels: RBF
>         Attachments: HDFS-13443-branch-2.001.patch, HDFS-13443-branch-2.002.patch, HDFS-13443.001.patch,
HDFS-13443.002.patch, HDFS-13443.003.patch
> Currently mount table cache is updated periodically, by default cache is updated every
minute. After change in mount table, user operations may still use old mount table. This is
bit wrong.
> To update mount table cache, maybe we can do following
>  * *Add refresh API in MountTableManager which will update mount table cache.*
>  * *When there is a change in mount table entries, router admin server can update its
cache and ask other routers to update their cache*. For example if there are three routers
R1,R2,R3 in a cluster then add mount table entry API, at admin server side, will perform following
sequence of action
>  ## user submit add mount table entry request on R1
>  ## R1 adds the mount table entry in state store
>  ## R1 call refresh API on R2
>  ## R1 calls refresh API on R3
>  ## R1 directly freshest its cache
>  ## Add mount table entry response send back to user.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org

View raw message