hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "liuhongtong (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-13346) RBF: Fix synchronization of router quota and ns quota
Date Tue, 27 Mar 2018 08:36:00 GMT

    [ https://issues.apache.org/jira/browse/HDFS-13346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16415229#comment-16415229
] 

liuhongtong commented on HDFS-13346:
------------------------------------

[~linyiqun] Thanks for your recommendation.

RouterAdminServer invokes this.rpcServer.setQuota to set Quota of Name Service and then updates
or adds MountTableEntry to State Store. 

But I am worried about how to ensure consistency and atomicity between different Routers.

RouterAdmin A sets nsQuota 200                     |     RouterAdmin B sets
nsQuota 300 

                       |                                   
               |                             |

RouterAdminServer A set NS nsQuota 200      |                           
 |

                       |                                   
               |    RouterAdminServer B set NS nsQuota 300 

                       |                                   
               |    RouterAdminServer B update State Store

RouterAdminServer A update State Store         |                       
    |

Now Router nsQuota is 200, but Name Service nsQuota is 300.

> RBF: Fix synchronization of router quota and ns quota
> -----------------------------------------------------
>
>                 Key: HDFS-13346
>                 URL: https://issues.apache.org/jira/browse/HDFS-13346
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>            Reporter: liuhongtong
>            Assignee: liuhongtong
>            Priority: Major
>         Attachments: HDFS-13346.001.patch, HDFS-13346.002.patch
>
>
> Check Router Quota and ns Quota:
> {code}
> $ hdfs dfsrouteradmin -ls /ns10t
> Mount Table Entries:
> Source                    Destinations              Owner                     Group 
                   Mode                      Quota/Usage              
> /ns10t                    ns10->/ns10t              hadp                      hadp
                     rwxr-xr-x                 [NsQuota: 150/319, SsQuota: -/-]
> /ns10t/ns1mountpoint      ns1->/a/tt                hadp                      hadp
                     rwxr-xr-x                 [NsQuota: -/-, SsQuota: -/-]
> $ hdfs dfs -count -q hdfs://ns10/ns10t
>          150            -155            none             inf            3          302
                 0 hdfs://ns10/ns10t
> {code}
> Update Router Quota:
> {code:java}
> $ hdfs dfsrouteradmin -setQuota /ns10t -nsQuota 400
> Successfully set quota for mount point /ns10t
> {code}
> Check Router Quota and ns Quota:
> {code:java}
> $ hdfs dfsrouteradmin -ls /ns10t
> Mount Table Entries:
> Source                    Destinations              Owner                     Group 
                   Mode                      Quota/Usage              
> /ns10t                    ns10->/ns10t              hadp                      hadp
                     rwxr-xr-x                 [NsQuota: 400/319, SsQuota: -/-]
> /ns10t/ns1mountpoint      ns1->/a/tt                hadp                      hadp
                     rwxr-xr-x                 [NsQuota: -/-, SsQuota: -/-]
> $ hdfs dfs -count -q hdfs://ns10/ns10t
>          150            -155            none             inf            3          302
                 0 hdfs://ns10/ns10t
> {code}
> Now Router Quota has updated successfully, but ns Quota not.
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org


Mime
View raw message