hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Lei (Eddy) Xu (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-13736) Change PathMetadata to hold S3AFileStatus instead of FileStatus.
Date Thu, 20 Oct 2016 00:02:16 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-13736?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15590296#comment-15590296

Lei (Eddy) Xu commented on HADOOP-13736:

Thanks for the great summary, [~fabbri].    

As [~fabbri] mentioned, the motivation of this change is that: when I was working on HADOOP-13650
and HADOOP-13449, the PathMetadata and a few other assumptions that abstract away {{S3AFileStatus}}
make implemeting {{DynamoDBMetadataStore}}, {{CLI}} and integrating with {{S3AFileSystem}}
harder than they should. I lean to get a simpler and S3A-specific implementation which is
limited within the {{S3Guard}} scope. After we gain more experience on this matter, we might
be able to bring a better solution for other projects (i.e., HADOOP-12876).  Additionally,
IIUC, only the {{InMemoryMetadataStore}} is useful for other project. 

However I am not aware of the timeline for HADOOP-12876, so it is appreciate if [~stevel@apache.org]
and [~vishwajeet.dusane] can chime in.

Thanks !

> Change PathMetadata to hold S3AFileStatus instead of FileStatus.
> ----------------------------------------------------------------
>                 Key: HADOOP-13736
>                 URL: https://issues.apache.org/jira/browse/HADOOP-13736
>             Project: Hadoop Common
>          Issue Type: Sub-task
>          Components: fs/s3
>            Reporter: Lei (Eddy) Xu
>            Assignee: Lei (Eddy) Xu
> {{S3AFileStatus}} is implemented differently with {{FileStatus}}, for instance {{S3AFileStatus#isEmptyDirectory()}}
is not implemented in {{FileStatus()}}. And {{access_time}}, {{block_replication}}, {{owner}},
{{group}} and a few other fields are not meaningful in {{S3AFileStatus}}.  
> So in the scope of {{S3guard}}, it should use {{S3AFileStatus}} in  instead of {{FileStatus}}
in {{PathMetadaa}} to avoid casting the types back and forth in S3A. 

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org

View raw message