hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chris Nauroth (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-7090) Use unbuffered writes when persisting in-memory replicas
Date Fri, 10 Oct 2014 15:58:34 GMT

    [ https://issues.apache.org/jira/browse/HDFS-7090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14167030#comment-14167030
] 

Chris Nauroth commented on HDFS-7090:
-------------------------------------

Thanks for incorporating the feedback, Xiaoyu.  Here are a few additional comments:
# The current patch still breaks native compilation on BSD-based systems that are not FreeBSD.
 Specifically, I'm thinking of OSX.  {{__FreeBSD__}} won't be defined for the ifdef in that
case.
# In the JavaDocs, I don't think we can link to {{Storage}}, because it's in hadoop-hdfs.
 hadoop-common doesn't have a dependency on hadoop-hdfs, so the javadoc tool wouldn't be able
to find the method.  I'm not sure why this wasn't flagged as a javadoc warning.
# The test still has an unneeded {{mkdir}}.  I think this line can be deleted: {{assumeTrue(TEST_DIR.mkdir())}}.
# Instead of catching exceptions and calling {{fail}}, we can just let the exception be thrown.
 We'll actually get more debugging information this way if a test fails, because JUnit will
print the full stack trace.
# I recommend using {{IOUtils#cleanup}} to close {{channel}} and {{raSrcFile}}.  If an exception
is thrown in the try block, but then a {{close}} also fails in the finally block, then the
second exception will mask the first.  It's probably going to be more helpful for us to see
the exception from the try block.
# Unfortunately, it looks like the build artifacts from the pre-commit job are gone, so I
can't review the -1 from Jenkins.  I expect the release audit warning is unrelated.  Let's
investigate findbugs and tests.

> Use unbuffered writes when persisting in-memory replicas
> --------------------------------------------------------
>
>                 Key: HDFS-7090
>                 URL: https://issues.apache.org/jira/browse/HDFS-7090
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: datanode
>    Affects Versions: 2.6.0
>            Reporter: Arpit Agarwal
>            Assignee: Xiaoyu Yao
>             Fix For: 3.0.0
>
>         Attachments: HDFS-7090.0.patch, HDFS-7090.1.patch
>
>
> The LazyWriter thread just uses {{FileUtils.copyFile}} to copy block files to persistent
storage. It would be better to use unbuffered writes to avoid churning page cache.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message