hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Pete Wyckoff (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-4) tool to mount dfs on linux
Date Tue, 26 Feb 2008 01:15:51 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-4?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12572334#action_12572334

Pete Wyckoff commented on HADOOP-4:

Hi Rui,

It's because of the order FUSE calls the implementation. The file is created with an open
and then closed and then opened again. 

So in <16, that second open in write mode will fail :(  I have tried hacking things so
that the code doesn't do the close on the file in between and caches the filehandle and so
fakes the second open and everything works fine.

So, I think as long as appends are coming in, it should work with 16. I haven't looked at
the implementation, but hopefully it buffers things till it gets to a full block and isn't
creating small blocks all over the place :)  If it does, it's easy enough to buffer in fuse
(although 128MB is a big buffer).


ps although we're on 15.3 at FB, I can't upgrade our test cluster to 16 yet as we want to
install HBASE and need to try it there. I'm gonna try applying the hbase patch that  the powerset
guys nicely gave me soon and then if all goes well, I can upgrade our test cluster to 16 and
test things.

> tool to mount dfs on linux
> --------------------------
>                 Key: HADOOP-4
>                 URL: https://issues.apache.org/jira/browse/HADOOP-4
>             Project: Hadoop Core
>          Issue Type: Improvement
>          Components: fs
>    Affects Versions: 0.5.0
>         Environment: linux only
>            Reporter: John Xing
>            Assignee: Pete Wyckoff
>         Attachments: fuse-dfs.tar.gz, fuse-dfs.tar.gz, fuse-dfs.tar.gz, fuse-dfs.tar.gz,
fuse-dfs.tar.gz, fuse-hadoop-0.1.0_fuse-j.2.2.3_hadoop.0.5.0.tar.gz, fuse-hadoop-0.1.0_fuse-j.2.4_hadoop.0.5.0.tar.gz,
fuse-hadoop-0.1.1.tar.gz, fuse-j-hadoopfs-03.tar.gz, fuse_dfs.c, fuse_dfs.c, fuse_dfs.c, fuse_dfs.c,
fuse_dfs.sh, Makefile
> tool to mount dfs on linux

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message