Mailing-List: contact hdfs-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Date: Tue, 1 Aug 2017 18:22:00 +0000 (UTC)
From: "Wei-Chiu Chuang (JIRA)" <jira@apache.org>
To: hdfs-issues@hadoop.apache.org
Message-ID: <JIRA.13090709.1501227800000.64276.1501611720571@Atlassian.JIRA>
In-Reply-To: <JIRA.13090709.1501227800000@Atlassian.JIRA>
References: <JIRA.13090709.1501227800000@Atlassian.JIRA> <JIRA.13090709.1501227800969@jira-lw-us.apache.org>
Subject: [jira] [Commented] (HDFS-12209) VolumeScanner scan cursor not save
 periodic
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
archived-at: Tue, 01 Aug 2017 18:22:05 -0000


    [ https://issues.apache.org/jira/browse/HDFS-12209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16109457#comment-16109457 ] 

Wei-Chiu Chuang commented on HDFS-12209:
----------------------------------------

Hi [~fatkun] great bug report!
Hi [~vagarychen] I thought about it and initially reached the same conclusion as you, but after second thought, DataNode may be restarted between each scan cursor, so Time.monotonicNow may not be appropriate (each JVM process may have different nanoTime origin time).

Would it be possible to make a unit test? Or would someone else like to contribute a test?

> VolumeScanner scan cursor not save periodic
> -------------------------------------------
>
>                 Key: HDFS-12209
>                 URL: https://issues.apache.org/jira/browse/HDFS-12209
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: hdfs
>    Affects Versions: 2.6.0
>         Environment: cdh5.4.0
>            Reporter: fatkun
>         Attachments: HDFS-12209.patch
>
>
> The bug introduce from HDFS-7430 , the time is not same, one is monotonicMs and other is clock time. It should use Time.now() both
> VolumeScanner.java
> {code:java}
>         long saveDelta = monotonicMs - curBlockIter.getLastSavedMs();
>         if (saveDelta >= conf.cursorSaveMs) {
>           LOG.debug("{}: saving block iterator {} after {} ms.",
>                   this, curBlockIter, saveDelta);
>           saveBlockIterator(curBlockIter);
>         }
> {code}
> curBlockIter.getLastSavedMs() init here
> FsVolumeImpl.java
> {code:java}
>     BlockIteratorState() {
>       lastSavedMs = iterStartMs = Time.now();
>       curFinalizedDir = null;
>       curFinalizedSubDir = null;
>       curEntry = null;
>       atEnd = false;
>     }
> {code}


--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org