hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hudson (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-14383) Implement FileSystem that reads from HTTP / HTTPS endpoints
Date Mon, 08 May 2017 23:12:04 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-14383?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16001763#comment-16001763
] 

Hudson commented on HADOOP-14383:
---------------------------------

SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #11698 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/11698/])
HADOOP-14383. Implement FileSystem that reads from HTTP / HTTPS (wheat9: rev ff5ec3b841612f7f28ad8be5bbfec4168a8ac1f0)
* (add) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/http/package-info.java
* (edit) hadoop-common-project/hadoop-common/src/main/resources/META-INF/services/org.apache.hadoop.fs.FileSystem
* (edit) hadoop-project/pom.xml
* (edit) hadoop-common-project/hadoop-common/pom.xml
* (add) hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/http/TestHttpFileSystem.java
* (add) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/http/AbstractHttpFileSystem.java
* (add) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/http/HttpsFileSystem.java
* (add) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/http/HttpFileSystem.java


> Implement FileSystem that reads from HTTP / HTTPS endpoints
> -----------------------------------------------------------
>
>                 Key: HADOOP-14383
>                 URL: https://issues.apache.org/jira/browse/HADOOP-14383
>             Project: Hadoop Common
>          Issue Type: Improvement
>          Components: fs
>            Reporter: Haohui Mai
>            Assignee: Haohui Mai
>             Fix For: 2.9.0, 3.0.0-alpha3
>
>         Attachments: HADOOP-14383.000.patch, HADOOP-14383.001.patch, HADOOP-14383.002.patch
>
>
> We have a use case where YARN applications would like to localize resources from Artifactory.
Putting the resources on HDFS itself might not be ideal as we would like to leverage Artifactory
to manage different versions of the resources.
> It would be nice to have something like {{HttpFileSystem}} that implements the Hadoop
filesystem API and reads from a HTTP endpoint.
> Note that Samza has implemented the proposal by themselves:
> https://github.com/apache/samza/blob/master/samza-yarn/src/main/scala/org/apache/samza/util/hadoop/HttpFileSystem.scala
> The downside of this approach is that it requires the YARN cluster to put the Samza jar
into the classpath for each NM.
> It would be much nicer for Hadoop to have this feature built-in.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org


Mime
View raw message