hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alejandro Abdelnur (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-1695) Implement the rest (writable APIs) of RM web-services
Date Tue, 11 Feb 2014 18:36:20 GMT

    [ https://issues.apache.org/jira/browse/YARN-1695?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13898112#comment-13898112

Alejandro Abdelnur commented on YARN-1695:

Before we start coding work on this, it would be great to see how security will handled (authentication,
ACLs, tokens, etc).

I'm a bit a concern about introducing a second protocol everywhere. From a maintenance and
security risk point of view is doubling our development/support efforts. Granted, HDFS offers
data over RPC and HTTP. But, HTTP, when using HttpFS (how I recommend using it HTTP access)
is a gateway that ends up doing RPC to HDFS. Thus the only protocol accessing HDFS is RPC.

Have we considered having a C implementation of Hadoop RPC, with the multiplatform support
of protobuffers that may give us the multi-platform support we are aiming to with a single
protocol interface.


> Implement the rest (writable APIs) of RM web-services
> -----------------------------------------------------
>                 Key: YARN-1695
>                 URL: https://issues.apache.org/jira/browse/YARN-1695
>             Project: Hadoop YARN
>          Issue Type: New Feature
>            Reporter: Vinod Kumar Vavilapalli
>            Assignee: Vinod Kumar Vavilapalli
> MAPREDUCE-2863 added the REST web-services to RM and NM. But all the APIs added there
were only focused on obtaining information from the cluster. We need to have the following
REST APIs to finish the feature
>  - Application submission/termination (Priority): This unblocks easy client interaction
with a YARN cluster
>  - Application Client protocol: For resource scheduling by apps written in an arbitrary
language. Will have to think about throughput concerns
>  - ContainerManagement Protocol: Again for arbitrary language apps.
> One important thing to note here is that we already have client libraries on all the
three protocols that do some some heavy-lifting. One part of the effort is to figure out if
they can be made any thinner and/or how web-services will implement the same functionality.

This message was sent by Atlassian JIRA

View raw message