hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Pete Wyckoff (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-3805) improve fuse-dfs write performance which is 33% slower than hadoop dfs -copyFromLocal
Date Fri, 01 Aug 2008 13:32:31 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-3805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12619009#action_12619009

Pete Wyckoff commented on HADOOP-3805:

Discussion of options and kernels needed for better write performance in fuse:


basically need to use -o big_writes and have a kernel > 2.6.26-rc*

I don't currently have access to such a kernel, but it's probably worth while just adding
the -obig_writes option by default and having a conifg vars no_big_writes use_big_writes.

I will do up a patch.

> improve fuse-dfs write performance which is 33% slower than hadoop dfs -copyFromLocal
> -------------------------------------------------------------------------------------
>                 Key: HADOOP-3805
>                 URL: https://issues.apache.org/jira/browse/HADOOP-3805
>             Project: Hadoop Core
>          Issue Type: Improvement
>          Components: contrib/fuse-dfs
>            Reporter: Pete Wyckoff
>            Priority: Minor
> on hadoop17 running on the namenode/fuse mount point, fuse is 33% slower for writing
a 1GB file. I don't know right now how to improve this but thought I should open up a JIRA
for it.
> Note - this benchmark is with no tuning or special fuse config params.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message