hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Yiqun Lin (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (HDFS-13811) RBF: Race condition between router admin quota update and periodic quota update service
Date Fri, 30 Nov 2018 08:54:00 GMT

    [ https://issues.apache.org/jira/browse/HDFS-13811?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16704423#comment-16704423
] 

Yiqun Lin edited comment on HDFS-13811 at 11/30/18 8:53 AM:
------------------------------------------------------------

Hi [~dibyendu_hadoop], 
{quote}
I haven't change the existing logic for getQuotaUsage. I have added a check to verify the
quota cache value for the requested mount entry, if the cache is null, it will try to refresh
the cache for that mount entry only.
{quote}
I suppose we can extract this as a new function and be called in {{getQuotaUsage(String path)}}.

{quote}
 If we don't use updateQuotaCache flag, from RouterQuotaUpdateService#periodicInvoke getQuotaSetMountTables
will eventually call getQuotaUsage, which will again call periodicInvoke and it will go into
an infinite loop. That's why updateQuotaCache flag is required.
{quote}
I am not fully understanding these. I don't mean we must hard-coded the flag {{updateQuotaCache}}
value as false in {{GetMountTableEntriesRequest}}. As I see {{getMountTableEntries(boolean
updateQuotaCache)}} is used only by {{getQuotaSetMountTables}}. And then quota service passes
a false value to  {{getQuotaSetMountTables}}. No other places use this function. So I suggest
to remove the parameter in function.

Please feel free to attach next patch now.


was (Author: linyiqun):
Hi [~dibyendu_hadoop], 
{quote}
I haven't change the existing logic for getQuotaUsage. I have added a check to verify the
quota cache value for the requested mount entry, if the cache is null, it will try to refresh
the cache for that mount entry only.
{quote}
I suppose we can extract this as a new function and be called in {{getQuotaUsage(String path)}}.

{quote}
 If we don't use updateQuotaCache flag, from RouterQuotaUpdateService#periodicInvoke getQuotaSetMountTables
will eventually call getQuotaUsage, which will again call periodicInvoke and it will go into
an infinite loop. That's why updateQuotaCache flag is required.
{quote}
I am not fully understanding these. I don't mean we must hard-coded the flag {{updateQuotaCache}}
value as value in {{GetMountTableEntriesRequest}}. As I see {{getMountTableEntries(boolean
updateQuotaCache)}} is used only by {{getQuotaSetMountTables}}. And then quota service passes
a false value to  {{getQuotaSetMountTables}}. And no others places use this function. So I
suggest to remove the parameter in function.

Please feel free to attach next patch now.

> RBF: Race condition between router admin quota update and periodic quota update service
> ---------------------------------------------------------------------------------------
>
>                 Key: HDFS-13811
>                 URL: https://issues.apache.org/jira/browse/HDFS-13811
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>            Reporter: Dibyendu Karmakar
>            Assignee: Dibyendu Karmakar
>            Priority: Major
>         Attachments: HDFS-13811-000.patch, HDFS-13811-HDFS-13891-000.patch
>
>
> If we try to update quota of an existing mount entry and at the same time periodic quota
update service is running on the same mount entry, it is leading the mount table to _inconsistent state._
> Here transactions are:
> A - Quota update service is fetching mount table entries.
> B - Quota update service is updating the mount table with current usage.
> A' - User is trying to update quota using admin cmd.
> and the transaction sequence is [ A A' B ]
> quota update service is updating the mount table with old quota value.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org


Mime
View raw message