hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vishwajeet Dusane (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-12666) Support Microsoft Azure Data Lake - as a file system in Hadoop
Date Wed, 24 Feb 2016 14:33:18 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-12666?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15163080#comment-15163080
] 

Vishwajeet Dusane commented on HADOOP-12666:
--------------------------------------------

*For the common concern over the dependency on `org.apache.hadoop.hdfs.web` packaging* - Already
explained in the previous replies. However i would like to reiterate that due to current design
constraint in `org.apache.hadoop.hdfs.web` namespace, extended file system from `WebHdfsFileSystem`
can not access certain functionalities outside `org.apache.hadoop.hdfs.web`. Example :  Control
over additional or existing query parameters, HTTP configuration .. etc. Being said that,
We do desire to have only `org.apache.hadoop.fs.adl` package which contains all the functionalities.


In order to achieve our common goal, I would have to file few more JIRA's on the `org.apache.hadoop.hdfs.web`
package and work on to make extended FileSystem from `org.apache.hadoop.hdfs.web` configurable
and refactor existing ADL package accordingly. I would take up this activity once the Rev
1 i.e. this patch set is pushed in to ASF.

> Support Microsoft Azure Data Lake - as a file system in Hadoop
> --------------------------------------------------------------
>
>                 Key: HADOOP-12666
>                 URL: https://issues.apache.org/jira/browse/HADOOP-12666
>             Project: Hadoop Common
>          Issue Type: New Feature
>          Components: fs, fs/azure, tools
>            Reporter: Vishwajeet Dusane
>            Assignee: Vishwajeet Dusane
>         Attachments: HADOOP-12666-002.patch, HADOOP-12666-003.patch, HADOOP-12666-004.patch,
HADOOP-12666-005.patch, HADOOP-12666-006.patch, HADOOP-12666-1.patch
>
>   Original Estimate: 336h
>          Time Spent: 336h
>  Remaining Estimate: 0h
>
> h2. Description
> This JIRA describes a new file system implementation for accessing Microsoft Azure Data
Lake Store (ADL) from within Hadoop. This would enable existing Hadoop applications such has
MR, HIVE, Hbase etc..,  to use ADL store as input or output.
>  
> ADL is ultra-high capacity, Optimized for massive throughput with rich management and
security features. More details available at https://azure.microsoft.com/en-us/services/data-lake-store/



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message