Mailing-List: contact hdfs-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Date: Tue, 12 Dec 2017 07:11:00 +0000 (UTC)
From: "Wang XL (JIRA)" <jira@apache.org>
To: hdfs-issues@hadoop.apache.org
Message-ID: <JIRA.13121225.1511858912000.437000.1513062660180@Atlassian.JIRA>
In-Reply-To: <JIRA.13121225.1511858912000@Atlassian.JIRA>
References: <JIRA.13121225.1511858912000@Atlassian.JIRA> <JIRA.13121225.1511858912371@jira-lw-us.apache.org>
Subject: [jira] [Updated] (HDFS-12862) CacheDirective may invalidata,when NN
 restart or make a transition to Active.
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
archived-at: Tue, 12 Dec 2017 07:11:06 -0000


     [ https://issues.apache.org/jira/browse/HDFS-12862?page=3Dcom.atlassia=
n.jira.plugin.system.issuetabpanels:all-tabpanel ]

Wang XL updated HDFS-12862:
---------------------------
              Labels: patch  (was: )
       Fix Version/s: 2.7.1
    Target Version/s: 2.7.1
              Status: Patch Available  (was: Open)

> CacheDirective may invalidata,when NN restart or make a transition to Act=
ive.
> -------------------------------------------------------------------------=
----
>
>                 Key: HDFS-12862
>                 URL: https://issues.apache.org/jira/browse/HDFS-12862
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: caching, hdfs
>    Affects Versions: 2.7.1
>         Environment:=20
>            Reporter: Wang XL
>              Labels: patch
>             Fix For: 2.7.1
>
>         Attachments: HDFS-12862-branch-2.7.1.001.patch
>
>
> The logic in FSNDNCacheOp#modifyCacheDirective is not correct.  when modi=
fy cacheDirective,the expiration in directive may be a relative expiryTime,=
 and EditLog will serial a relative expiry time.
> {code:java}
> // Some comments here
> static void modifyCacheDirective(
>       FSNamesystem fsn, CacheManager cacheManager, CacheDirectiveInfo dir=
ective,
>       EnumSet<CacheFlag> flags, boolean logRetryCache) throws IOException=
 {
>     final FSPermissionChecker pc =3D getFsPermissionChecker(fsn);
>     cacheManager.modifyDirective(directive, pc, flags);
>     fsn.getEditLog().logModifyCacheDirectiveInfo(directive, logRetryCache=
);
>   }
> {code}
> But when SBN replay the log ,it will invoke FSImageSerialization#readCach=
eDirectiveInfo  as a absolute expiryTime.It will result in the inconsistenc=
y .
> {code:java}
>   public static CacheDirectiveInfo readCacheDirectiveInfo(DataInput in)
>       throws IOException {
>     CacheDirectiveInfo.Builder builder =3D
>         new CacheDirectiveInfo.Builder();
>     builder.setId(readLong(in));
>     int flags =3D in.readInt();
>     if ((flags & 0x1) !=3D 0) {
>       builder.setPath(new Path(readString(in)));
>     }
>     if ((flags & 0x2) !=3D 0) {
>       builder.setReplication(readShort(in));
>     }
>     if ((flags & 0x4) !=3D 0) {
>       builder.setPool(readString(in));
>     }
>     if ((flags & 0x8) !=3D 0) {
>       builder.setExpiration(
>           CacheDirectiveInfo.Expiration.newAbsolute(readLong(in)));
>     }
>     if ((flags & ~0xF) !=3D 0) {
>       throw new IOException("unknown flags set in " +
>           "ModifyCacheDirectiveInfoOp: " + flags);
>     }
>     return builder.build();
>   }
> {code}
> In other words=EF=BC=8C fsn.getEditLog().logModifyCacheDirectiveInfo(dire=
ctive, logRetryCache)  may serial a relative expiry time,But  builder.setEx=
piration(CacheDirectiveInfo.Expiration.newAbsolute(readLong(in)))   read it=
 as a absolute expiryTime.


--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org