db-derby-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Apache Wiki <wikidi...@apache.org>
Subject [Db-derby Wiki] Update of "Requirement Specifications for Monitoring & Management Extensions using JMX" by SanketSharma
Date Tue, 13 Jun 2006 17:49:15 GMT
Dear Wiki user,

You have subscribed to a wiki page or wiki category on "Db-derby Wiki" for change notification.

The following page has been changed by SanketSharma:
http://wiki.apache.org/db-derby/Requirement_Specifications_for_Monitoring_%26_Management_Extensions_using_JMX

New page:
= Apache Derby =

== Requirement Specifications for Monitoring and Management Extensions using JMX ==

== Document Contents ==

1. [#Revision_History Revision History][[BR]]
2. [#Disclaimer Disclaimer and Legal Information][[BR]]
3. [#About About this document][[BR]]
	3.1 [#Purpose Purpose][[BR]]
	3.2 [#Audience Intended Audience][[BR]]
	3.3 [#Notations Notations and Conventions][[BR]]
	3.4 [#Structure Structure of this document][[BR]]
	3.5 [#Note A note about ranking][[BR]]
4. [#Assumptions Assumptions and Dependencies][[BR]]
5. [#Analysis Requirement Analysis][[BR]]
	5.1 [#Overview Overview][[BR]]
	5.2 [#Current Current Scenario][[BR]]
		5.2.1 [#Embedded Embedded Instance][[BR]]
		5.2.2 [#Embedded_Server Embedded in Server Framework][[BR]]
		5.2.3 [#Network_Server Network Server][[BR]]
	5.3 [#Conclusion Conclusion][[BR]]
6. [#Identified Requirements Identified][[BR]]
	6.1 [#Feature_Overview Overview][[BR]]
	6.2 [#Feature_Identified Features Identified][[BR]]
	6.3 [#RemoteLocal Remote vs. Local][[BR]]
	6.4 [#Feature_Conclusion Conclusion][[BR]]
7. [#References References][[BR]]
== 1. Revision History ==
[[Anchor(Revision_History)]]

||<rowstyle="font-weight: bold;	font-size: 12pt;color: white; text-align: center; background-color:
#828DA6; font-family: Helvetica, Arial, Tahoma, Verdana, 'Nimbus Sans L', lucida-sans, lucidasans,
sanserif; padding: 6px; border: none;"> Version || Version Information || Date || Modified
by || Comments ||
|| 1.0 || Document Created || Tuesday, June 06, 2006 || Sanket Sharma || Comments are welcome!
||

== 2. Disclaimer and Legal Information ==
[[Anchor(Disclaimer)]]
Copyright 2005 The Apache Software Foundation or its licensors, as applicable. 

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file
except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0.


Unless required by applicable law or agreed to in writing, software distributed under the
License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND,
either express or implied. See the License for the specific language governing permissions
and limitations under the License. 

== 3. About this Document ==
[[Anchor(About)]]
=== 3.1. Purpose ===
[[Anchor(Purpose)]]
         This document identifies the requirements and design goals for adding Management
extensions to Apache Derby as Google Summer of Code 2006 project. Consequently the document
identifies a set of requirements, categorizes them and attempts to assign priority to the
identified requirements for their implementation. However, the priorities assigned reflect
my own personal views and are left open for the community to decide. The final priorities
will be decided by the community as a whole. This document does not describe any high level
architecture or implementation details. These will be described in a subsequent documents.
[[BR]]
=== 3.2. Intended Audience ===
[[Anchor(Audience)]]
          The intended audience includes a wide community of those interested in developing
monitoring and management applications for the Apache Derby. It also includes the end users
of the database who will ultimately use these features and the developers who might use these
features for fine tuning their applications. 
[[BR]]
=== 3.3. Notations and Conventions ===
[[Anchor(Notations)]]
          The documentation set uses the following conventions.
[[BR]]
          ||<rowstyle="font-weight: bold;	font-size: 12pt;color: white; text-align: center;
background-color: #828DA6; font-family: Helvetica, Arial, Tahoma, Verdana, 'Nimbus Sans L',
lucida-sans, lucidasans, sanserif; padding: 6px; border: none;"> Convention || Explanation
|| Example ||
          ||<|5> {{{ Monospace }}} || Filename || {{{ MyFile.java }}} ||
          || Directory Names and Path Names || {{{ \MyDir\MyDir2 }}} ||
          || Commands and Command line options || {{{ ecl-02 }}} ||
          || Method names, Classes, Interfaces in running text || Use the {{{ createService()
}}} method to... ||
          || Parameters or other placeholders || {{{ doSomething(int val1, int val2) }}}||
          || ''' {{{ Monospace bold }}} ''' || User input || ''' {{{ [C:] dir }}} '''||
          || '' Italics '' || Emphasis; Introducing or defining terms || The term '' access
'' takes as its subject ||
          || [ ]     || Optional Items || {{{ -Fa[c] }}} Indicates these command-line options:
{{{ -Fa }}} and {{{ -Fac }}} ||
          ||<|4> {|} ||<|4> Choice of one item from a selection of two or more
items. || -aX{K | W | P} Indicates these command-line options: ||
          || {{{ -aXK }}} ||
          || {{{ -aXW }}} ||
          || {{{ -aXP }}} ||
[[BR]]
=== 3.4. Structure of this document ===
[[Anchor(Structure)]]
          This section is divided into seven sections. Section one to four provide basic information
about legal stuff and notational conventions used in this document. Section five describes
what features currently exist in Derby to monitor and tune the performance of the system and
why we need a better monitoring and management system. Section six then tries to identify
features that might be useful to developers and administrators. While this section enumerates
a whole bunch of features, this section also tries to assign a priority to their implementation.
Section seven enlists the sources of information for this document. 
[[BR]]
=== 3.5. A Note about Ranking ===
[[Anchor(Note)]]
           Developers and Derby users who read this document are requested to assign their
priorities to the features enlisted in section six. I have provided an initial version of
priorities based on my personal opinion. Developers and Users are requested add a column to
the table, add their names as coloumn heading and assign priorities to the features. This
will help us in deciding which features have to be implemented and in which order. 
[[BR]]
== 4. Assumptions and Dependencies ==
[[Anchor(Assumptions)]]
[[BR]]
	    This document follows Java Coding conventions defined at http://java.sun.com/docs/codeconv/html/CodeConvTOC.doc.html
[[BR]]
== 5. Requirement Analysis ==
[[Anchor(Analysis)]]

=== 5.1. Overview ===
[[Anchor(Overview)]]
	   Apache Derby is a relational, light weight, embedded database developed entirely in Java.
Being an embedded database it is very flexible in its use and can be used either embedded
inside an application sharing the same JVM as the application or can be embedded inside a
server framework (such as Geronimo) or can be deployed as a standalone server (with the Network
server framework) in a traditional client server environment. However, currently derby does
not support any mature monitoring and management features which lets a developer or an administrator
view statistics about the health of a system and fine tune system parameters via a management
console or application. Such functionality is indispensable for any database system of any
scale for a multitude of reasons. This section tries to identify the requirements for such
a monitoring and management framework.
[[BR]]
=== 5.2. Current Scenario ===
[[Anchor(Current)]]
            The following section identifies the what is currently supported in derby with
reference to:
[[BR]]
   	   ''' 1. Monitoring the state the system ''' [[BR]]
		[[BR]]
   	   ''' 2. Tuning system parameters ''' [[BR]]
		[[BR]]
        Although current features can not be termed as monitoring or management features,
but they do provide some primitive functionality. The following section describes these in
details with respect to the three ways Derby can be used - embedded and as network server.

[[BR]]
==== 5.2.1. Embedded Instance ====
[[Anchor(Embedded)]]
	  When Derby runs as an embedded database engine sharing the same JVM there are quite a few
ways for setting the properties for a Derby instance, both system wide and database. Such
properties can be set either in the derby.properties file or can be at run time by: [[BR]]
		[[BR]]
		* Using parameters while starting application e.g. java -Dderby.xxx=something YourApplication.
[[BR]]
		[[BR]]
		* Using API's provided.  [[BR]]
		[[BR]]


	While these features let an application read or update system properties, there is no interactive
way of doing it. Further, these can not be termed as monitoring and management features. 

	What is needed is a functionality which a developer can use to tune the system while examining
its state from outside the application. A typical usage scenario would be when a user runs
his application and then starts the management console to view the current state of system
or to debug his application. Without writing any code in his application he is able to change
the system state and view how the system responds. Such functionality should be available
to all applications without any coding. The developer/administrator should be able to tune
the system at runtime without changing his code. 
[[BR]]
==== 5.2.2. Embedded inside a server framework ====
[[Anchor(Embedded_Server)]]

	     [[BR]]Essentially, this is same as 5.2.1 where the application server or some other
server framework becomes client for Derby. Most application servers come with a management
console that lets administrators and developers configure various system wide settings locally
or remotely. Since Derby does not provide any management framework that integrates into application
server frameworks, developers of such systems using Derby have to write wrappers around Derby
API calls to provide any management functionality, if at all.

	These servers usually generate a lot of runtime statistics and building monitoring extensions
right into Derby will make integration much more transparent. The need for a standard management
framework is accentuated in this case. Currently, the only way a developer can change system
settings is via API calls or derby.properties file. 

[[BR]]
==== 5.2.3. Network Server ====
[[Anchor(Network_Server)]]
	   
	   In this mode Derby runs in embedded mode, wrapped inside a server framework that multiple
remote clients connect to. Derby can run as full fledged relational database server with the
help of bundled server framework. Multiple remote clients can connect to database server simultaneously
while the server takes care of concurrency issues and ACID properties.

	Again, .properties file and API calls are the only mechanisms to tune the system. However,
in this mode Derby provides a Servlet that can be used for remote administration of Server.
The servlet is first deployed in a servlet container like Tomcat which enables administration
of the server over HTTP. However the servlet interface only provides management support without
any support for monitoring the server. 

	The proposed extensions will enable a server framework using derby to provide live monitoring
of system along with the "regular" administration features found in most commercial database
systems. Some of these features have been identified in the following sections. 
[[BR]]
=== 5.3. Conclusion ===
[[Anchor(Conclusion)]]
	   Whether running in embedded mode or as server, the developer or user has very limited
choices for controlling the behavior of system. In light of the facts above it is desirable
to have a monitoring and management extension for derby for that enables developers and administrators
to view the performance of a Derby instance without writing any extra code. The exact list
of features to be supported by such a extension are discussed below
[[BR]]
== 6. Requirements Identified ==
[[Anchor(Identified)]]

=== 6.1. Overview ===
[[Anchor(Feature_Overview)]]
		   This section describes the requirements of the new monitoring and management system for
Apache Derby. The requirements are classified in three categories: [[BR]]

	''' 1. Requirements for the Monitoring System: ''' Requirements identified under this category
are critical for assessing the health of the system. These are either counters or variables
that provide insight into the state of the system. [[BR]]
	''' 2. Requirements for Management System: ''' Requirements identified under this category
are essential to fine tuning the state of the system. These operations provide the user ability
to change system properties. [[BR]]
	''' 3. Requirements for Notification System: ''' Requirements identified under this category
are not essential for a management or monitoring system but provide extended functionality
to enterprise applications that embed derby. [[BR]]
The requirements identified are for the core Derby engine and do not take into consideration
any special requirements for a specific case. 


=== 6.2. Proposed Features ===
[[Anchor(Feature_Identified)]]

		||<rowstyle="font-weight: bold;	font-size: 12pt;color: white; text-align: center; background-color:
#828DA6; font-family: Helvetica, Arial, Tahoma, Verdana, 'Nimbus Sans L', lucida-sans, lucidasans,
sanserif; padding: 6px; border: none;">Cateogory/Code || Features || Priority (Sanket)
|| Priority (David) ||
		||<rowstyle="font-weight: bold;	font-size: 12pt;color: white; text-align: center; BACKGROUND:
#00ccff; font-family: Helvetica, Arial, Tahoma, Verdana, 'Nimbus Sans L', lucida-sans, lucidasans,
sanserif; padding: 6px; border: none;"> a) Monitoring || || || ||
		||  	MON1	||  View derby system/database properties (derby.xxx) ||	0	||0||
		|| 	MON2 	||  Sys info e.g. version, jvm version etc.||	0	||0||
		|| 	MON3	||  Locks held in the system ||		1		||2||
		||	MON4	||  Number of commits/sec	||	0		||1||
		||	MON5	||  Number of roll backs/sec	||	0		||1||
		||	MON6	||  Time since last save point	||	2		||2||
		||	MON7	||  Number of system threads	||	0		||1||
		||	MON8	||  Number of users connected	||	0		||1||
		||	MON9	||  Number of open connections	||	0		||1||
		||	MON10   ||  Transaction Granularity	||	2		||2||
		||	MON11	||  Open cursors		||	1		||1||
		||	MON12	||  Databases 			||			|| ||
		||	MON12.1	||	Database properties (derby.xxx)||	0	||0||
		||	MON12.2	||	Database Objects	||		2	||2||
		||	MON12.3	||	Database Size		||	1		||2||
		||	MON13	||  Memory usage stats 		||	 1		||1||
		||	MON14	||  CPU usage stats		||	1		||1||
		||	MON15	|| Stats about Garbage collection ||	1		||2||
		||	MON16	|| View number of cached objects (Query)||	2       ||2||
		||	MON17 	||  Deadlocks detected/trace	||	1		||2||
		||	MON18	|| Stats about disk I/O		||	0		||1||
		||	MON19	|| Query plan			||	2		||2||
		||<rowstyle="font-weight: bold;	font-size: 12pt;color: white; text-align: center; BACKGROUND:
#00ccff; font-family: Helvetica, Arial, Tahoma, Verdana, 'Nimbus Sans L', lucida-sans, lucidasans,
sanserif; padding: 6px; border: none;">  b) Management || || || ||
		||	MGMT1	||  Modify derby/database properties (derby.xxx) ||	0||0||
		||	MGMT2	||  Boot a database		||	1		||2||
		||	MGMT3	||  Shutdown a database		||	0		||0||
		||	MGMT4	||  Drop a database		||	2		||2||
		||	MGMT5	||  Create a database		||	2		||3||
		||	MGMT6	||  Create database objects	||	2		||3||
		||	MGMT7	||  Backup a database		||	1		||1||
		||	MGMT8	||  Restore a database from backup ||	1		||1||
		||	MGMT9	||  Recovery			||	1		||? Not sure if this is needed - we do auto-recovery||
		||	MGMT10	||  Force a checkpoint		||	2		||3||
		||	MGMT11	||  Rebuild indexes		||	2		||2||
		||	MGMT12	||  Security			||			|| ||
		||	MGMT12.1||	Modify Authorization	||	1		||2||
		||	MGMT12.2||	Modify Authentication scheme ||	1		||2||
		||	MGMT13	|| Replacement of management servlet||	2		||3||
		||<rowstyle="font-weight: bold;	font-size: 12pt;color: white; text-align: center; BACKGROUND:
#00ccff; font-family: Helvetica, Arial, Tahoma, Verdana, 'Nimbus Sans L', lucida-sans, lucidasans,
sanserif; padding: 6px; border: none;"> c) Notifications || || || ||
		||	NOT1	|| When a transaction fails	||	0		||1||
		||	NOT2	|| When a database is booted	||	1		||2||
		||	NOT3	|| When a database is shutdown	||	1		||2||
		||	NOT4	|| When a user connects		||	2		||4||
		||	NOT5	|| When a user disconnects	||	2		||4||
		||	NOT6	|| Every time a query is fired	||	2		||4||
		||	NOT7	|| Every time a stored procedure/function is run ||	2	||3||
		||	NOT8	|| All database operations like creating/deleting/modifying etc. ||	2		||4||

		   This is not an exhaustive list and is open to further discussions by the community. In
particular, I would like to see some more suggestions on notifications. 

Some of these attributes or features may lack sufficient detail at the moment, but these details
will be worked out as the work progresses. 


=== 6.3. Remote VS. Local ===
[[Anchor(RemoteLocal)]]
		In my opinion, remote access to these features should not be provided. My opinion is based
on the following observations: 

	* ''' Embedded Nature of derby: ''' The scope of the database can not extend beyond that
of application. If the application as a whole does not provide any remote administration feature,
it does not make much sense for only a part of the application to be remotely accessible.[[BR]]
	* ''' Overhead involved: ''' I think adding remote manageability will incur more overhead
on the hosting machine. This will be against Derby’s design philosophy. [[BR]]
	* ''' Design Considerations: ''' Let us think why such functionality will be needed: 


		''' a) Embedded inside application: ''' If the developer really needs to provide remote
management functionality for just the database, I think it is a design flaw. The database
should be removed from the application and deployed as a server which can be administered
more easily. In case he wants to make the entire application remotely accessible, it will
not be very difficult. [[BR]]
		''' b) Embedded inside a server framework: ''' In this case I think it is the server’s
responsibility to expose this functionality to the management application. How would feel
if your car’s speed-o-meter were on its tires, fuel gauges on the fuel tank, torque meter
on the engine etc. [[BR]]
		''' c) As a database server: ''' Continuing what I said in 5.2.3 and (2) above, I think
it is the responsibility of the server framework to expose this functionality to outside world,
not the database engine. [[BR]]

	However, keeping in mind the pressing demand by users of Derby, a middle path can be providing
remote administration as "Optional Services" that can be enabled and disabled via some command
line option when Derby is stared. [[BR]]

=== 6.4. Conclusion ===
[[Anchor(Feature_Conclusion)]]
	[[BR]]
		The priorities assigned reflect my own personal views and are open for community discussion.
The final rakings will be decided by the community. 


== 7.0 References ==
[[Anchor(References)]]

	*Java Platform Profiling Architecture, JSR 163 at http://www.jcp.org/en/jsr/overview 
        
	*Java 2 Standard Edition, http://java.sun.com/j2se/1.5.0/docs/api/ 

        *Apache Foundation’ Derby project http://db.apache.org/derby

        *IBM’s Cloudscape developer page: http://www-128.ibm.com/developerworks/db2/zones/cloudscape/

        *Java Management Extensions (JMX): http://java.sun.com/j2se/1.5.0/docs/guide/jmx/index.html

        *Java Specification Request for JMX: http://www.jcp.org/en/jsr/detail?id=160

        *Java Specification Request for JMX: http://www.jcp.org/en/jsr/detail?id=03

Mime
View raw message