hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Craig Macdonald (JIRA)" <j...@apache.org>
Subject [jira] Issue Comment Edited: (HADOOP-4) tool to mount dfs on linux
Date Tue, 26 Feb 2008 18:54:51 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-4?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12572599#action_12572599
] 

craigm edited comment on HADOOP-4 at 2/26/08 10:54 AM:
----------------------------------------------------------------

Hi Pete,

The block stuff in fuse is appallingly documented. I have hunted the Web for info on this
all afternoon, to understand it further. To be honest, the only thing I have found useful
is reading the source of ntfs-3g.c at http://ntfs-3g.cvs.sourceforge.net/ntfs-3g/ntfs-3g/src/ntfs-3g.c?revision=1.106&view=markup

I test I did do a few days ago was to comparing reading an NFS mounted file directly vs, the
same file read via NFS via a FUSE fs - http://mattwork.potsdam.edu/projects/wiki/index.php/Rofs#rofs_code_.28C.29
(ROFS, the Read-Only Filesystem). Speed results were fairly comparable between NFS & NFS+ROFS,
so it suggests that FUSE doesnt add too much overhead to IO. Hence then we can only suspect
that the problem is in either (a) JNI interface, or (b) the size of the reads we're performing.

I dont have any objections to pretty large buffer sizes for fuse_dfs.c - HDFS is designed
for large files, and streaming read access.

Btw, you mentioned you are re-exporting the mounted FS as NFS - have you had any issues vs
the issues described in fuses' README.NFS? 

Regards

Craig


      was (Author: craigm):
    Hi Pete,

The block stuff in fuse is appallingly documented. I have hunted the Web for info on this
all afternoon, to understand it further. To be honest, the only thing I have found useful
is reading the source of ntfs-3g.c at http://ntfs-3g.cvs.sourceforge.net/ntfs-3g/ntfs-3g/src/ntfs-3g.c?revision=1.106&view=markup

I test I did do a few days ago was to comparing reading an NFS mounted file directly vs, the
same file read via NFS via a FUSE fs - http://mattwork.potsdam.edu/projects/wiki/index.php/Rofs#rofs_code_.28C.29
(ROFS, the Read-Only Filesystem).

Btw, you mentioned you are re-exporting the mounted FS as NFS - have you had any issues vs
the issues described in fuses' README.NFS? 

Regards

Craig

  
> tool to mount dfs on linux
> --------------------------
>
>                 Key: HADOOP-4
>                 URL: https://issues.apache.org/jira/browse/HADOOP-4
>             Project: Hadoop Core
>          Issue Type: Improvement
>          Components: fs
>    Affects Versions: 0.5.0
>         Environment: linux only
>            Reporter: John Xing
>            Assignee: Pete Wyckoff
>         Attachments: fuse-dfs.tar.gz, fuse-dfs.tar.gz, fuse-dfs.tar.gz, fuse-dfs.tar.gz,
fuse-dfs.tar.gz, fuse-hadoop-0.1.0_fuse-j.2.2.3_hadoop.0.5.0.tar.gz, fuse-hadoop-0.1.0_fuse-j.2.4_hadoop.0.5.0.tar.gz,
fuse-hadoop-0.1.1.tar.gz, fuse-j-hadoopfs-03.tar.gz, fuse_dfs.c, fuse_dfs.c, fuse_dfs.c, fuse_dfs.c,
fuse_dfs.sh, Makefile
>
>
> tool to mount dfs on linux

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message