hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Daryn Sharp (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-6773) MiniDFSCluster can run dramatically faster
Date Tue, 29 Jul 2014 22:01:39 GMT

    [ https://issues.apache.org/jira/browse/HDFS-6773?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14078460#comment-14078460

Daryn Sharp commented on HDFS-6773:

The reason is every NN operation currently takes the full hit for a spindle sync which can
cost 10+ms.

> MiniDFSCluster can run dramatically faster
> ------------------------------------------
>                 Key: HDFS-6773
>                 URL: https://issues.apache.org/jira/browse/HDFS-6773
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: namenode
>    Affects Versions: 2.0.0-alpha, 3.0.0
>            Reporter: Daryn Sharp
> The mini cluster is unnecessarily running with durable edit logs.  The following change
cut runtime of a single test from ~30s to ~10s.
> {code}EditLogFileOutputStream.setShouldSkipFsyncForTesting(true);{code}
> The mini cluster should default to this behavior after identifying the few edit log tests
that probably depend on durable logs.

This message was sent by Atlassian JIRA

View raw message