hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tsz Wo Nicholas Sze (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HADOOP-10741) A lightweight WebHDFS client library
Date Mon, 23 Jun 2014 23:01:26 GMT
Tsz Wo Nicholas Sze created HADOOP-10741:

             Summary: A lightweight WebHDFS client library
                 Key: HADOOP-10741
                 URL: https://issues.apache.org/jira/browse/HADOOP-10741
             Project: Hadoop Common
          Issue Type: New Feature
          Components: tools
            Reporter: Tsz Wo Nicholas Sze
            Assignee: Mohammad Kamrul Islam

One of the motivations for creating WebHDFS is for applications connecting to HDFS from outside
the cluster.  In order to do so, users have to either
# install Hadoop and use WebHdfsFileSsytem, or
# develop their own client using the WebHDFS REST API.

For #1, it is very difficult to manage and unnecessarily complicated for other applications
since Hadoop is not a lightweight library.  For #2, it is not easy to deal with security and
handle transient errors.

Therefore, we propose adding a lightweight WebHDFS client as a separated library which does
not depend on Common and HDFS.  The client can be packaged as a standalone jar.  Other applications
simply add the jar to their classpath for using it.

This message was sent by Atlassian JIRA

View raw message