hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mingliang Liu (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HADOOP-14428) s3a: mkdir appears to be broken
Date Wed, 24 May 2017 21:59:04 GMT

     [ https://issues.apache.org/jira/browse/HADOOP-14428?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Mingliang Liu updated HADOOP-14428:
    Attachment: HADOOP-14428.000.patch

The {{Path::getParent()}} behavior has been known for years but we accepted it, e.g.
  protected void processNonexistentPath(PathData item) throws IOException {
    // check if parent exists. this is complicated because getParent(a/b/c/) returns a/b/c,
    // we want a/b
    if (!createParents &&
        !item.fs.exists(new Path(item.path.toString()).getParent())) {
      throw new PathNotFoundException(item.toString());
    if (!item.fs.mkdirs(item.path)) {
      throw new PathIOException(item.toString());

Here I'd suggest not to change the existing behavior of {{Path::getParent()}} which returns
{{s3a://my-bucket/dir}} for {{s3a://my-bucket/dir/}}. Instead we bypass this by creating a
new Path object which removes the trailing "/".

Thanks [~stevel@apache.org] for the suggestion for adding unit tests. The newly added tests
can not pass w/o this patch, and pass w/ this patch.

> s3a: mkdir appears to be broken
> -------------------------------
>                 Key: HADOOP-14428
>                 URL: https://issues.apache.org/jira/browse/HADOOP-14428
>             Project: Hadoop Common
>          Issue Type: Bug
>          Components: fs/s3
>    Affects Versions: 3.0.0-alpha2, HADOOP-13345
>            Reporter: Aaron Fabbri
>            Assignee: Mingliang Liu
>            Priority: Blocker
>         Attachments: HADOOP-14428.000.patch
> Reproduction is:
> hadoop fs -mkdir s3a://my-bucket/dir/
> hadoop fs -ls s3a://my-bucket/dir/
> ls: `s3a://my-bucket/dir/': No such file or directory
> I believe this is a regression from HADOOP-14255.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org

View raw message