hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Xiaoyu Yao (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-7475) Make TestLazyPersistFiles#testLazyPersistBlocksAreSaved deterministic
Date Wed, 10 Dec 2014 00:51:12 GMT

    [ https://issues.apache.org/jira/browse/HDFS-7475?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14240425#comment-14240425
] 

Xiaoyu Yao commented on HDFS-7475:
----------------------------------

Thanks Arpit for the review. countPendingLazyPersistTasks() will eventually converge to 0
without synchronization using an infinite loop. This will achieve the same results as put
the validation logic in a loop. 

Agree periodically checking all replicas location until success or timeout will be simpler
without additional test hook in code. I will update the patch for that. 

> Make TestLazyPersistFiles#testLazyPersistBlocksAreSaved deterministic 
> ----------------------------------------------------------------------
>
>                 Key: HDFS-7475
>                 URL: https://issues.apache.org/jira/browse/HDFS-7475
>             Project: Hadoop HDFS
>          Issue Type: Test
>          Components: test
>    Affects Versions: 2.6.0
>            Reporter: Xiaoyu Yao
>            Assignee: Xiaoyu Yao
>         Attachments: HDFS-7475.0.patch
>
>
> TestLazyPersistFiles#testLazyPersistBlocksAreSaved verifies that the memory block is
saved in lazy persist directory after some sleep with the following code. 
> {code}
>     // Sleep for a short time to allow the lazy writer thread to do its job
>     Thread.sleep(6 * LAZY_WRITER_INTERVAL_SEC * 1000);
> {code} 
> However, the sleep only guarantees an async lazy persist task is scheduled. The task
running on a per volume thread pool thread can take longer to finish persisting. It will be
more deterministic to wait the pending lazy persist task count reduce to 0 before verifying
the block file exist in lazy persist dir. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message