hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ling Zhou (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-12756) Incorporate Aliyun OSS file system implementation
Date Mon, 25 Apr 2016 06:59:13 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-12756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15255983#comment-15255983

Ling Zhou commented on HADOOP-12756:

Hi Liu Yi,
Ali-oss client does rely on a higher version of httpclient, it is required by Aliyun OSS SDK.
Currently it uses 4.4 version, and hadoop uses 4.2.5 version which does not work for OSS SDK.
And about the name "oss", we will change it according to your suggestion.

Hi Steve Loughran,
We have read the filesystem specification docs, this implementation is similar to S3A. So
operations like rename and delete are still not atomic. Aliyun OSS is much like general object
storage system except it is strong consistent.

> Incorporate Aliyun OSS file system implementation
> -------------------------------------------------
>                 Key: HADOOP-12756
>                 URL: https://issues.apache.org/jira/browse/HADOOP-12756
>             Project: Hadoop Common
>          Issue Type: New Feature
>          Components: fs
>            Reporter: shimingfei
>            Assignee: shimingfei
>         Attachments: 0001-OSS-filesystem-integration-with-Hadoop.patch, HCFS User manual.md,
OSS integration.pdf, OSS integration.pdf
> Aliyun OSS is widely used among China’s cloud users, but currently it is not easy to
access data laid on OSS storage from user’s Hadoop/Spark application, because of no original
support for OSS in Hadoop.
> This work aims to integrate Aliyun OSS with Hadoop. By simple configuration, Spark/Hadoop
applications can read/write data from OSS without any code change. Narrowing the gap between
user’s APP and data storage, like what have been done for S3 in Hadoop 

This message was sent by Atlassian JIRA

View raw message