hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Steve Loughran (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-9371) Define Semantics of FileSystem and FileContext more rigorously
Date Sat, 25 May 2013 13:32:23 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-9371?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13667065#comment-13667065

Steve Loughran commented on HADOOP-9371:

Konstantin -good points

-I'm going to redo it as .apt so the linking isn't going to be useful (soon). MD may be well
tooled, but as there isn't a consistent format for handling tables, it's not that much better
than APT (though it does make it easier to use angle brackets in in-line code, and doesn't
tie you to a single build tool forever.

# Atomic recursive deletes? it sort of happens today in every real FS as the toplevel inode
goes away. I don't know how that spans filesystems -can I actually do an rm -rf above a mounted
FS in Unix?

That said: saying "no guarantees about atomicity" is one thing -it gives us flexibility in
future - but as all "normal" filesystems appear to provide this, code will tend to assume
it anyway. I think we should do it -but call out blobstores for breaking some of these rules.

# atomic rename where the parent dir stays the same does seem a good compromise on atomicity;
it means that more distributed filesystems don't do it. 

In fact, we could say "there are no guarantees that rename() across filesystems work at all".
And then add an explicit exception {{RenameAcrossFileSystemsUnsupported}} for this. I'm confident
that you can't rename file:///c:/something.txt to file:///d:/something.txt on windows.

> Define Semantics of FileSystem and FileContext more rigorously
> --------------------------------------------------------------
>                 Key: HADOOP-9371
>                 URL: https://issues.apache.org/jira/browse/HADOOP-9371
>             Project: Hadoop Common
>          Issue Type: Sub-task
>          Components: fs
>    Affects Versions: 1.2.0, 3.0.0
>            Reporter: Steve Loughran
>            Assignee: Steve Loughran
>         Attachments: HADOOP-9361.2.patch, HADOOP-9361.patch, HadoopFilesystemContract.pdf
>   Original Estimate: 48h
>  Remaining Estimate: 48h
> The semantics of {{FileSystem}} and {{FileContext}} are not completely defined in terms
> # core expectations of a filesystem
> # consistency requirements.
> # concurrency requirements.
> # minimum scale limits
> Furthermore, methods are not defined strictly enough in terms of their outcomes and failure
> The requirements and method semantics should be defined more strictly.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message