hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Brian Bockelman (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HDFS-860) fuse-dfs truncate behavior causes issues with scp
Date Tue, 17 Aug 2010 03:05:16 GMT

    [ https://issues.apache.org/jira/browse/HDFS-860?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12899258#action_12899258

Brian Bockelman commented on HDFS-860:


Jakob - thanks for the reminder.

1) You can see 'scp' calling truncate by downloading from a remote server to a mounted FUSE
HDFS instance, like so:

[brian@red ~]$ strace scp brian-test:/tmp/hello_world /mnt/hadoop/dropfiles/test_scp 2>&1
| grep truncate
ftruncate(3, 13)                        = 0

2) IIRC, I tried your suggestion, but the size of the file in the namenode isn't updated until
close() is called, right?  [Actually, now that I say that out loud, I now suppose we can take
advantage of the single-thread-writer rule and just track the number of bytes in the client?
 That seems doable upon 30 seconds of reflection at 10PM...]

> fuse-dfs truncate behavior causes issues with scp
> -------------------------------------------------
>                 Key: HDFS-860
>                 URL: https://issues.apache.org/jira/browse/HDFS-860
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: contrib/fuse-dfs
>            Reporter: Brian Bockelman
>            Assignee: Brian Bockelman
>            Priority: Minor
>         Attachments: HDFS-860.patch
> For whatever reason, scp issues a "truncate" once it's written a file to truncate the
file to the # of bytes it has written (i.e., if a file is X bytes, it calls truncate(X)).
> This fails on the current fuse-dfs.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message