hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Luke Lu (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HADOOP-9160) Adopt Jolokia as the JMX HTTP/JSON bridge.
Date Sat, 13 Jul 2013 08:01:53 GMT

     [ https://issues.apache.org/jira/browse/HADOOP-9160?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Luke Lu updated HADOOP-9160:
----------------------------

    Description: 
The current JMX HTTP bridge has served its purpose, while a more complete solution: Jolokia
(formerly Jmx4Perl) has been developed/matured over the years.

Jolokia provides comprehensive JMX features over HTTP/JSON including search and list of JMX
attributes and operations metadata, which helps to support inter framework/platform compatibility.
It has first class language bindings for Perl, Python, Javascript, Java.

It's trivial (see demo patch) to incorporate Jolokia servlet into Hadoop HTTP servers and
use the same security mechanisms.

Adopting Jolokia will substantially improve the manageability of Hadoop and its ecosystem.



  was:
Currently we use Hadoop RPC (and some HTTP, notably fsck) for admin protocols. We should consider
adopt JMX for future admin protocols, as it's the industry standard for java server management
with wide client support.

Having an alternative/redundant RPC mechanism is very desirable for admin protocols. I've
seen in the past in multiple cases, where NN and/or JT RPC were locked up solid due to various
bugs and/or RPC thread pool exhaustion, while HTTP and/or JMX worked just fine.

Other desirable benefits include admin protocol backward compatibility and introspectability,
which is convenient for a centralized management system to manage multiple Hadoop clusters
of different versions. Another notable benefit is that it's much easier to implement new admin
commands in JMX (especially with MXBean) than Hadoop RPC, especially in trunk (as well as
0.23+ and 2.x).

Since Hadoop RPC doesn't guarantee backward compatibility (probably not ever for branch-1),
there are few external tools depending on it. We can keep the old protocols for as long as
needed. New commands should be in JMX. The transition can be gradual and backward-compatible.

        Summary: Adopt Jolokia as the JMX HTTP/JSON bridge.  (was: Adopt JMX for management
protocols)

Limit the scope to Jolokia adoption. The current planned write operations will be limited
to resource management for multi-framework/platform (including Hadoop 1 and 2+) integration.
The default config will be read-only to be compatible with common JMX security setup.
                
> Adopt Jolokia as the JMX HTTP/JSON bridge.
> ------------------------------------------
>
>                 Key: HADOOP-9160
>                 URL: https://issues.apache.org/jira/browse/HADOOP-9160
>             Project: Hadoop Common
>          Issue Type: Improvement
>            Reporter: Luke Lu
>         Attachments: hadoop-9160-demo-branch-1.txt
>
>
> The current JMX HTTP bridge has served its purpose, while a more complete solution: Jolokia
(formerly Jmx4Perl) has been developed/matured over the years.
> Jolokia provides comprehensive JMX features over HTTP/JSON including search and list
of JMX attributes and operations metadata, which helps to support inter framework/platform
compatibility. It has first class language bindings for Perl, Python, Javascript, Java.
> It's trivial (see demo patch) to incorporate Jolokia servlet into Hadoop HTTP servers
and use the same security mechanisms.
> Adopting Jolokia will substantially improve the manageability of Hadoop and its ecosystem.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message