hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Wei-Chiu Chuang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-12209) VolumeScanner scan cursor not save periodic
Date Tue, 01 Aug 2017 18:22:00 GMT

    [ https://issues.apache.org/jira/browse/HDFS-12209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16109457#comment-16109457
] 

Wei-Chiu Chuang commented on HDFS-12209:
----------------------------------------

Hi [~fatkun] great bug report!
Hi [~vagarychen] I thought about it and initially reached the same conclusion as you, but
after second thought, DataNode may be restarted between each scan cursor, so Time.monotonicNow
may not be appropriate (each JVM process may have different nanoTime origin time).

Would it be possible to make a unit test? Or would someone else like to contribute a test?

> VolumeScanner scan cursor not save periodic
> -------------------------------------------
>
>                 Key: HDFS-12209
>                 URL: https://issues.apache.org/jira/browse/HDFS-12209
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: hdfs
>    Affects Versions: 2.6.0
>         Environment: cdh5.4.0
>            Reporter: fatkun
>         Attachments: HDFS-12209.patch
>
>
> The bug introduce from HDFS-7430 , the time is not same, one is monotonicMs and other
is clock time. It should use Time.now() both
> VolumeScanner.java
> {code:java}
>         long saveDelta = monotonicMs - curBlockIter.getLastSavedMs();
>         if (saveDelta >= conf.cursorSaveMs) {
>           LOG.debug("{}: saving block iterator {} after {} ms.",
>                   this, curBlockIter, saveDelta);
>           saveBlockIterator(curBlockIter);
>         }
> {code}
> curBlockIter.getLastSavedMs() init here
> FsVolumeImpl.java
> {code:java}
>     BlockIteratorState() {
>       lastSavedMs = iterStartMs = Time.now();
>       curFinalizedDir = null;
>       curFinalizedSubDir = null;
>       curEntry = null;
>       atEnd = false;
>     }
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org


Mime
View raw message