hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Todd Lipcon (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-9160) Adopt JMX for management protocols
Date Fri, 04 Jan 2013 18:24:14 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-9160?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13544096#comment-13544096
] 

Todd Lipcon commented on HADOOP-9160:
-------------------------------------

Just as a reference, here's a simple Hadoop-2 compatible C++ RPC client I hacked up in a weekend
a few months back: https://github.com/toddlipcon/libhrpc
It uses Boost::ASIO so it's actually more complicated than a minimal blocking client. It's
also prototype-y code, not ready for real use. That said, my point here is that it wouldn't
be too difficult to get simple clients done in a couple other languages and just have _one_
included RPC mechanism instead of three (RPC, REST, and JMX). Then those who want to add other
mechanisms for specific integrations could do so via plugins.
                
> Adopt JMX for management protocols
> ----------------------------------
>
>                 Key: HADOOP-9160
>                 URL: https://issues.apache.org/jira/browse/HADOOP-9160
>             Project: Hadoop Common
>          Issue Type: Improvement
>            Reporter: Luke Lu
>
> Currently we use Hadoop RPC (and some HTTP, notably fsck) for admin protocols. We should
consider adopt JMX for future admin protocols, as it's the industry standard for java server
management with wide client support.
> Having an alternative/redundant RPC mechanism is very desirable for admin protocols.
I've seen in the past in multiple cases, where NN and/or JT RPC were locked up solid due to
various bugs and/or RPC thread pool exhaustion, while HTTP and/or JMX worked just fine.
> Other desirable benefits include admin protocol backward compatibility and introspectability,
which is convenient for a centralized management system to manage multiple Hadoop clusters
of different versions. Another notable benefit is that it's much easier to implement new admin
commands in JMX (especially with MXBean) than Hadoop RPC, especially in trunk (as well as
0.23+ and 2.x).
> Since Hadoop RPC doesn't guarantee backward compatibility (probably not ever for branch-1),
there are few external tools depending on it. We can keep the old protocols for as long as
needed. New commands should be in JMX. The transition can be gradual and backward-compatible.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message