hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Joel Baranick (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HADOOP-14124) S3AFileSystem silently deletes "fake" directories when writing a file.
Date Mon, 27 Feb 2017 06:03:45 GMT

     [ https://issues.apache.org/jira/browse/HADOOP-14124?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Joel Baranick updated HADOOP-14124:
-----------------------------------
    Description: I realize that you guys probably have a good reason for {{S3AFileSystem}}
to cleanup "fake" folders when a file is written to S3.  That said, that fact that it silently
does this feels like a separation of concerns issue.  It also leads to weird behavior issues
where calls to {{AmazonS3Client.getObjectMetadata}} for folders work before calling {{S3AFileSystem.create}}
but not after.  Also, there seems to be no mention in the javadoc that the {{deleteUnnecessaryFakeDirectories}}
method is automatically invoked. Lastly, it seems like the goal of {{FileSystem}} should be
to ensure that code built on top of it is portable to different implementations.  This behavior
is an example of a case where this can break down.  (was: I realize that you guys probably
have a good reason for S3AFileSystem to cleanup "fake" folders when a file is written to S3.
 That said, that fact that it silently does this feels like a separation of concerns issue.
 It also leads to weird behavior issues where calls to AmazonS3Client.getObjectMetadata for
folders work before calling `S3AFileSystem.create` but not after.  Also, there seems to be
no mention in the javadoc that the `deleteUnnecessaryFakeDirectories` method is automatically
invoked. Lastly, it seems like the goal of FileSystem should be to ensure that code built
on top of it is portable to different implementations.  This behavior is an example of a case
where this can break down.)

> S3AFileSystem silently deletes "fake" directories when writing a file.
> ----------------------------------------------------------------------
>
>                 Key: HADOOP-14124
>                 URL: https://issues.apache.org/jira/browse/HADOOP-14124
>             Project: Hadoop Common
>          Issue Type: Bug
>    Affects Versions: 2.6.0
>            Reporter: Joel Baranick
>              Labels: filesystem, s3
>
> I realize that you guys probably have a good reason for {{S3AFileSystem}} to cleanup
"fake" folders when a file is written to S3.  That said, that fact that it silently does this
feels like a separation of concerns issue.  It also leads to weird behavior issues where calls
to {{AmazonS3Client.getObjectMetadata}} for folders work before calling {{S3AFileSystem.create}}
but not after.  Also, there seems to be no mention in the javadoc that the {{deleteUnnecessaryFakeDirectories}}
method is automatically invoked. Lastly, it seems like the goal of {{FileSystem}} should be
to ensure that code built on top of it is portable to different implementations.  This behavior
is an example of a case where this can break down.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org


Mime
View raw message