hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mohammad Arshad (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-13443) RBF: Update mount table cache immediately after changing (add/update/remove) mount table entries.
Date Thu, 26 Apr 2018 16:40:02 GMT

    [ https://issues.apache.org/jira/browse/HDFS-13443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16454512#comment-16454512
] 

Mohammad Arshad commented on HDFS-13443:
----------------------------------------

bq. I think we need some metric to make sure we run the update operations. Right now, it
might be just luck that we hit the new mount table entries. I would propagate the error of
not all Router request completed all the way to the client

# refreshMountTableEntries( RefreshMountTableEntriesRequest request) updates cache of only
one router, where is connected
#  addMountTableEntry( AddMountTableEntryRequest request), updateMountTableEntry and removeMountTableEntry
update cache of all the routers

# For refreshMountTableEntries() success true or false is enough
# For other API, shall we introduce new fields in the their response to indicate whether all
router cache was updated or not, If not then which routers were not updated. May be use can
try to directly refreshMountTableEntries on those failed routers and if still failed then
can wait up to periodic cache refresh interval.
is this OK? any other way to go about it?

> RBF: Update mount table cache immediately after changing (add/update/remove) mount table
entries.
> -------------------------------------------------------------------------------------------------
>
>                 Key: HDFS-13443
>                 URL: https://issues.apache.org/jira/browse/HDFS-13443
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: fs
>            Reporter: Mohammad Arshad
>            Assignee: Mohammad Arshad
>            Priority: Major
>              Labels: RBF
>         Attachments: HDFS-13443-branch-2.001.patch, HDFS-13443-branch-2.002.patch, HDFS-13443.001.patch,
HDFS-13443.002.patch, HDFS-13443.003.patch, HDFS-13443.004.patch, HDFS-13443.005.patch
>
>
> Currently mount table cache is updated periodically, by default cache is updated every
minute. After change in mount table, user operations may still use old mount table. This is
bit wrong.
> To update mount table cache, maybe we can do following
>  * *Add refresh API in MountTableManager which will update mount table cache.*
>  * *When there is a change in mount table entries, router admin server can update its
cache and ask other routers to update their cache*. For example if there are three routers
R1,R2,R3 in a cluster then add mount table entry API, at admin server side, will perform following
sequence of action
>  ## user submit add mount table entry request on R1
>  ## R1 adds the mount table entry in state store
>  ## R1 call refresh API on R2
>  ## R1 calls refresh API on R3
>  ## R1 directly freshest its cache
>  ## Add mount table entry response send back to user.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org


Mime
View raw message