geode-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From r..@apache.org
Subject [28/50] [abbrv] incubator-geode git commit: Created JMX Federation in Geode (markdown)
Date Sun, 03 May 2015 21:37:56 GMT
Created JMX Federation in Geode (markdown)

Project: http://git-wip-us.apache.org/repos/asf/incubator-geode/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-geode/commit/155dad68
Tree: http://git-wip-us.apache.org/repos/asf/incubator-geode/tree/155dad68
Diff: http://git-wip-us.apache.org/repos/asf/incubator-geode/diff/155dad68

Branch: refs/heads/gh-wiki
Commit: 155dad6830ecfc8d226e9fe15227fca3fce8c2bc
Parents: 888482c
Author: Rishitesh Mishra <rishi80.mishra@gmail.com>
Authored: Fri Apr 10 14:19:32 2015 +0530
Committer: Rishitesh Mishra <rishi80.mishra@gmail.com>
Committed: Fri Apr 10 14:19:32 2015 +0530

----------------------------------------------------------------------
 JMX-Federation-in-Geode.md | 160 ++++++++++++++++++++++++++++++++++++++++
 1 file changed, 160 insertions(+)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/incubator-geode/blob/155dad68/JMX-Federation-in-Geode.md
----------------------------------------------------------------------
diff --git a/JMX-Federation-in-Geode.md b/JMX-Federation-in-Geode.md
new file mode 100644
index 0000000..9891c11
--- /dev/null
+++ b/JMX-Federation-in-Geode.md
@@ -0,0 +1,160 @@
+### JMX Federation in Geode
+
+This document explains how Geode JMX federation works and some internal mechanisms to get
developers get started. Basic JMX knowledge is required to go through this document.
+
+#### JMX Federation overview:
+
+Geode defines number of MXBeans which can be used to monitor and manage Geode cluster. 
+For list of MXBeans which are exposed by Geode see the link [Geode Documentation](http://gemfire.docs.pivotal.io/latest/userguide/index.html#managing/management/list_of_gemfire_mbeans_full.html#topic_48194A5BDF3F40F68E95A114DD702413)
+
+These MXBeans follows standard JMX structure and protocols. For simplicity rest of the document
will mention MBean instead of MXBean.
+
+##### The concept of Single-Agent View
+With a single-agent view, the application interacts with one MBeanServer. Application  development
is much easier because they don't have to find the right MBeanServer to make a request on
an MBean. One MBeanServer is responsible for aggregating all of the MBeans in the MBeanServers
with which it is federated. This is very convenient because the application  can interact
with a local MBeanServer that has services and adapters to interact with other local or remote
MBeanServers. The location of the federated MBeanServers does not change how the application
or manager is programmed to interact with its MBeanServer. 
+
+For the sake of this discussion, we will refer to the single-agent MBeanServer as the Manager
Node. The MBeanServers that the master interacts with (and the application does not) will
be referred to as Managed Nodes .A distributed system may have one or more Manager nodes.
It is advised to keep only one Manager node. All other nodes in the system are Managed nodes.
Manager node is responsible for maintaining  proxies of all MBeans defined in other nodes.
In addition there are some local MBeans defined in Manager node.
+
+The following diagram shows how all MBeans are accessible from a master( i.e. Manager)  node.
+
+
+[[images/Figure9.13.jpg]]
+
+
+
+Apart from ease of use,  other goals in mind for Single-Agent view  were 
+* It should be scalable.
+* Performance impact of federation should be minimal. 
+* User should be able to monitor the Distributed System from any other third party tool,
compliant with JMX.
+
+
+#### Federation Internals
+
+
+##### Federation In Picture
+
+
+[[images/Overview.png]]
+
+
+
+##### Unique IDs for MBeans and Proxies
+Each MBean from each node has a proxy counterpart with the Manager. This needs a unique identification
mechanism. For this each Geode MBean is given a unique ObjectName. e.g. GemFire:type=Member,member=<name-or-dist-member-id>
or GemFire:type=Member,service=Region,name=<regionName>,member=<name-or-dist-member-id>.
+The <name-or-dist-member-id> part in ObjectName identifies a resource uniquely if the
resource is distributed across members. For example a DistributedRegion or a PartitionedRegion.
+
+
+            
+##### MBean state
+Most of the MBean state (not all) is maintained by implementing ****com.gemstone.gemfire.internal.statistics.StatisticsListener****
interface. This interface is a clean way to listen for any statistic update. Each Member periodically
replicates all MBean states to Manager node/s. Rather than using JMX protocol we use internal
Geode support for replication. Two system internal replicated regions are defined for this.
ObjectName of the MBean being the key of this region and ****com.gemstone.gemfire.management.internal.FederationComponent****
is the value. 
+Each MBean is represented by one FederationComponent. This component is optimized for serialization
as it implements DataSerializable & DataSerializableFixedID.All Managed nodes replicate
a set of FederationComponents to the Manager node/s.
+
+At Manager node side FederationComponents are wrapped by a proxy handler which has the intelligence
to delegate MBean state access to stored FederationComponents.
+
+Following sequence diagram shows how a getter method on MBean is invoked on Manager side.
+
+[[images/getter.png]]
+          
+Some of the MBean data changes very rarely or never. For example region definition. Such
kind of data access and other MBean operations are delegated to FunctionService to fetch data
from the respective node. The proxy handler for FederationComponents know where to delegate
such a request. Also all setters on MBeans follows the same code path.
+
+[[images/setter.png]]
+
+##### Custom Data types
+As MXBeans support custom data types , Geode proxy layer needs to be aware of different open
data types like javax.management.openmbean.CompositeData and javax.management.openmbean.TabularData.
Remember Geode proxy layer does not use JMX. Hence there was a need for a conversion layer
which can understand OpenData types.
+****com.gemstone.gemfire.management.internal.MXBeanProxyInvocationHandler**** is the class
which handles conversion of all open data types as understood by JMX layer.
+
+##### JMX connector
+Manager node in addition to having proxies from all the nodes also opens up a JMX connector.
Any JMX compliant client like JConsole can connect to it and do JMX operations on all the
MBean proxies irrespective of whether that MBean belongs to local Manager node or a remote
Managed node.
+The default port on which the connector opens is 1099. User can change it by specifying "jmx-manager-port"
cache config attribute while starting a manager.
+
+A remote client can connect to exposed connector by standard JMX protocols. Code will change
a bit depending on whether the JMX client has access to Geode jars or not.
+
+    Map env = new HashMap();
+    String[] creds = {"user", "pwd"};
+    env.put(JMXConnector.CREDENTIALS, creds);
+
+    JMXServiceURL url = new JMXServiceURL("service:jmx:rmi:///jndi/rmi://:1099/jmxrmi");
+    JMXConnector jmxc = JMXConnectorFactory.connect(url, env);
+    MBeanServerConnection mbsc = jmxc.getMBeanServerConnection();
+			
+    ObjectName distSystemMBean = new ObjectName("GemFire:service=System,type=Distributed");
+
+    //With client having access to Geode jar
+    DistributedSystemMXBean dsMbean = JMX.newMBeanProxy(mbsc, distSystemMBean, DistributedSystemMXBean.class,
true);
+    String[] members = dsMbean.listMembers();
+
+    //With client NOT having access to Geode jar
+    String[] members = (String[])mbsc.invoke(distSystemMBean, "listMembers", null, null);
+
+##### Java API 
+Users also can access all the MBeans and proxies by Geode APIs. The main entry point for
using MBeans is com.gemstone.gemfire.management.ManagementService.
+
+    //Example Java API calls on a Manager node. For detailed APIs please see Geode javadocs.
+    //To get hold of a ManagementService instance 
+    ManagementService service = ManagementService.getManagementService(cache);
+
+    //Access a local RegionMBean
+
+    service.getLocalRegionMBean(regionPath);
+
+    //Query MBeans from a Member.
+    DistributedMember member = someMember;
+
+    Set<ObjectName> objectNamesOfMember = service.queryMBeanNames(member);
+
+    ObjectName objectName = objectNamesOfMember.get(0);//Lets assume we want to invoke some
operation on the first MBean
+
+    //Access an MBean instance or proxy
+    Class  interfaceClass = RegionMXBean.class;
+    RegionMXBean bean = service.getMBeanInstance(objectName, interfaceClass);// bean can
be a local MBean or a proxy MBean depending on the ObjectName.
+
+
+
+
+##### Notifications
+All notifications originating from a Managed node MBean also gets propagated to Manager node
and emitted against the corresponding proxy MBean. A notification is immediate and the client
can assume to get a notification almost at the same time when it originated. There will be
some network latency, but that should be acceptable for most real life monitoring systems.
+
+A JMX client can register any notification handler even with a proxy MBean on Manager and
it will get all the notifications from the corresponding Managed node MBean.
+All Geode alerts are also wrapped in JMX notifications and emitted through Manager. Please
see the following URL for more information [System Alerts] (http://gemfire.docs.pivotal.io/latest/userguide/index.html#getting_started/quickstart_examples/system_alerts_jmx.html
)
+
+
+See the following example to see how a notification listener is registered for a MBean and
proxy. In both the cases the code remains same.
+
+    //Example Java API calls on a Manager node. For detailed APIs please see Geode javadocs.
+    //To get hold of a ManagementService instance 
+    ManagementService service = ManagementService.getManagementService(cache);
+
+
+    //Query MBeans from a Member.
+    DistributedMember member = someMember;
+
+    Set<ObjectName> objectNamesOfMember = service.queryMBeanNames(member);
+
+    ObjectName objectName = objectNamesOfMember.get(0);//Lets assume we want to register
a notification handler on the first MBean
+
+
+
+    MBeanServer mbsc = ManagementFactory.getPlatformMBeanServer();
+
+    mbsc.addNotificationListener(objectName, listener, null, null);//Listener can be any
class implementing javax.management.NotificationListener.
+
+
+
+##### Manager Node
+Any node of Geode can act as a Manager provided it has been allowed to act as a Manager.
   
+Following cache config property determines whether a node starts as a manager.
+* jmx-manager : Identifies a node as a potential manager , but does not start the manager.

+* jmx-manager-start : Start as a manager when the node starts.
+
+
+
+##### Manager Node Transition
+Only potential managers can be transitioned into a manager at run time. All locators are
potential manager by default. Users can disable a Locator to become a manager by specifying
jmx-manager=false while starting a locator.
+When GFSH or Pulse tries to connect to a Locator , it transitions self as a Manager and start
to build up proxies for all the MBeans. If Locator has been disabled to act as a Manager,
it will try to look up for any potential Managers in the system and mandate any one of them
to act as a Manager.
+
+Users can also manually transition any node to Manager by invoking ManagerMXBean.startManager().
But as no Manager will be present before doing this they have to access the local node to
do so.
+
+
+##### Overhead of Manager nodes 
+Manager has been designed to be as less resource consuming as possible by using proven Geode
serialization and replication protocols. However some memory and CPU consumption is obvious.
So it is advisable to keep Manager nodes separate from Data or Server nodes.
+
+##### Fail over of Manager nodes
+If a manager node stops/crashes all JMX clients connected to the manager gets disconnected.
However GFSH and Pulse has inbuilt mechanism to fail over to a new manager. GFSH and Pulse
ask the locator for another Manager end point and connect with it. Normal JMX clients have
to do it manually.
\ No newline at end of file


Mime
View raw message