hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Steve Loughran (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-9371) Define Semantics of FileSystem and FileContext more rigorously
Date Sat, 25 May 2013 13:32:23 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-9371?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13667065#comment-13667065
] 

Steve Loughran commented on HADOOP-9371:
----------------------------------------

Konstantin -good points

-I'm going to redo it as .apt so the linking isn't going to be useful (soon). MD may be well
tooled, but as there isn't a consistent format for handling tables, it's not that much better
than APT (though it does make it easier to use angle brackets in in-line code, and doesn't
tie you to a single build tool forever.

# Atomic recursive deletes? it sort of happens today in every real FS as the toplevel inode
goes away. I don't know how that spans filesystems -can I actually do an rm -rf above a mounted
FS in Unix?

That said: saying "no guarantees about atomicity" is one thing -it gives us flexibility in
future - but as all "normal" filesystems appear to provide this, code will tend to assume
it anyway. I think we should do it -but call out blobstores for breaking some of these rules.

# atomic rename where the parent dir stays the same does seem a good compromise on atomicity;
it means that more distributed filesystems don't do it. 

In fact, we could say "there are no guarantees that rename() across filesystems work at all".
And then add an explicit exception {{RenameAcrossFileSystemsUnsupported}} for this. I'm confident
that you can't rename file:///c:/something.txt to file:///d:/something.txt on windows.




                
> Define Semantics of FileSystem and FileContext more rigorously
> --------------------------------------------------------------
>
>                 Key: HADOOP-9371
>                 URL: https://issues.apache.org/jira/browse/HADOOP-9371
>             Project: Hadoop Common
>          Issue Type: Sub-task
>          Components: fs
>    Affects Versions: 1.2.0, 3.0.0
>            Reporter: Steve Loughran
>            Assignee: Steve Loughran
>         Attachments: HADOOP-9361.2.patch, HADOOP-9361.patch, HadoopFilesystemContract.pdf
>
>   Original Estimate: 48h
>  Remaining Estimate: 48h
>
> The semantics of {{FileSystem}} and {{FileContext}} are not completely defined in terms
of 
> # core expectations of a filesystem
> # consistency requirements.
> # concurrency requirements.
> # minimum scale limits
> Furthermore, methods are not defined strictly enough in terms of their outcomes and failure
modes.
> The requirements and method semantics should be defined more strictly.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message