hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Thomas Marquardt (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HADOOP-14583) wasb throws an exception if you try to create a file and there's no parent directory
Date Fri, 18 Aug 2017 00:15:00 GMT

     [ https://issues.apache.org/jira/browse/HADOOP-14583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Thomas Marquardt updated HADOOP-14583:
--------------------------------------
    Attachment: HADOOP-14583-003.patch

Attaching HADOOP-14583-003.patch.

This updates {{retrieveMetadata}} to treat {{blob.exists() == false}} and a file not found
error from {{blob.downloadAttributes()}} in the same manner.  It fixes a race condition in
the {{retrieveMetadata}} implementation where {{blob.exists()}} returns true, then the file
is deleted by an external agent, and then {{downloadAttributes}} is called and fails with
a file not found error.

I also updated the tests to use {{Callable<V>}} instead of {{Runnable}} so we can more
easily validate the contract of the {{FileSystem.create}} and {{FileSystem.delete}} APIs and
check for exceptions.  The new test {{testConcurrentCreateDeleteFile}} fails intermittently
without this fix.  I have run the new test and the updated existing test in a loop hundreds
of times to ensure that they both  pass 100% of the time.

All tests are passing against my US West storage account.

Tests run: 775, Failures: 0, Errors: 0, Skipped: 155
Total time: 17:13 minutes

> wasb throws an exception if you try to create a file and there's no parent directory
> ------------------------------------------------------------------------------------
>
>                 Key: HADOOP-14583
>                 URL: https://issues.apache.org/jira/browse/HADOOP-14583
>             Project: Hadoop Common
>          Issue Type: Sub-task
>          Components: fs/azure
>    Affects Versions: 2.9.0
>            Reporter: Steve Loughran
>            Assignee: Esfandiar Manii
>            Priority: Minor
>         Attachments: HADOOP-14583-001.patch, HADOOP-14583-002.patch, HADOOP-14583-003.patch
>
>
> It's a known defect of the Hadoop FS API (and one we don't explicitly test for enough),
but you can create a file on a path which doesn't exist. In that situation, the create() logic
is expectd to create the entries.
> Wasb appears to raise an exception if you try to call {{create(filepath)}} without calling
{{mkdirs(filepath.getParent()}} first. That's the semantics expected of {{createNonRecursive()}}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org


Mime
View raw message