axis-java-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Samisa Abeysinghe <samisa.abeysin...@gmail.com>
Subject Re: [GSoC Project Proposal] Distributed TCP Monitor
Date Thu, 01 Apr 2010 00:51:12 GMT
On Thu, Apr 1, 2010 at 4:53 AM, nilupa bandara <nilupas@gmail.com> wrote:

> I was thinking about Java Swing based UI


Swing is a good start. But if you look at some of the requirements like
monitoring multiple instances and correlate them, discussed in this thread,
it sounds as if we would be better off having a Web interface, rather than a
desktop client.


> because the client may have
> to cache whatever messages it already has and will receive,


If you are looking to support message-correlation, as discussed in some
replies in this thread above, you need to have a persistence storage (a DB
or something like that) rather than a memory cache.
Because, some message sequences, like those involved in an async
invocations, need to have a way to keep the messages, in there, longer, and
may be there is a server re-start, in the middle.
Of course, this requirements spend on the scope of this effort.

Plus, if you want to deal with historic data, and look at what was happening
to message sequences over time, you got to use a persistence layer. Again
this might be out of scope for this project, but if this is to be extended
to support these, you better think of this right now.


> build the
> message flow and update the UI accordingly. Perhaps there is a better
> way ?


We already do what you are trying to do, and much more with WSO2 BAM. I am
tempted to say that Shinding based gadget dashboard is really a compelling
UI for this.

I am not trying to take away you project or trying to hinder this effort at
all.
I think it will be a good idea for you to have a look at BAM and see what
the missing pieces are and figure out the right model for this tool.

Also, having had worked on WSO2 BAM project, I also have some understanding
on sort of problems that you might run into, when doing this. For e.g., if
you want to correlate message sequences in a high volume setup, you will be
swamped by the volume of messages, sooner than later. So correlation for the
purpose of monitoring is not an easy thing to do. Correlation, for the
purpose of debugging will not run into this problem, if you run this on a
controlled environment. However, what you really want to debug is the deal
clustered setup, and not a staged dev setup, the problem again surfaces.

You might want to list the objectives and then break them down
into monitoring and debugging spaces.

Samisa...


>
> Thanks
> Nilupa.
>
> On Wed, Mar 31, 2010 at 4:12 PM, Samisa Abeysinghe
> <samisa.abeysinghe@gmail.com> wrote:
> > What UI technology would be used for this?
> >
> > Samisa...
> >
> > On Wed, Mar 31, 2010 at 4:26 AM, Andreas Veithen
> > <andreas.veithen@gmail.com> wrote:
> >> I was thinking more about something like ITCAM for SOA.
> >>
> >> On Tue, Mar 30, 2010 at 13:40, Sanjiva Weerawarana
> >> <sanjiva@opensource.lk> wrote:
> >>> That runtime monitoring / correlation is what the WSO2 Business
> Activity
> >>> Monitor does / is for ...
> >>> See: http://wso2.com/products/business-activity-monitor/
> >>> Sanjiva.
> >>>
> >>> On Tue, Mar 30, 2010 at 2:27 PM, Andreas Veithen <
> andreas.veithen@gmail.com>
> >>> wrote:
> >>>>
> >>>> I was actually referring to scenarios where a service may call other
> >>>> services, which in turn may call again other services. In these
> >>>> scenarios, it is not sufficient to simply collect received/sent
> >>>> messages on different hosts, unless the system is isolated
> >>>> (development environment) or the request rate is sufficiently low so
> >>>> that messages can be correlated based on time. Here is a concrete
> >>>> scenario from a project in my company:
> >>>>
> >>>> - Request comes in on an ESB that does security and validation.
> >>>> - Request is processed by an application server which persists the
> >>>> received information and publishes a JMS message (pub/sub events).
> >>>> - The event is consumed by one or more components that may in turn
> >>>> interact with other services.
> >>>>
> >>>> If you want track this flow, it is not sufficient to intercept the
> >>>> messages on the different hosts: in addition, you need to instrument
> >>>> the services so that the outgoing requests are correlated with the
> >>>> incoming requests (by adding SOAP and/or transport headers). What I
> >>>> would like to be able to do in the above scenario is to enable
> >>>> end-to-end monitoring on a per-message basis: I add a special SOAP or
> >>>> HTTP header to the initial request to enable logging and this
> >>>> information is propagated along the chain. Every service/component
> >>>> then sends a copy of the incoming/outgoing requests/responses to a
> >>>> central place where the sequence of events is reconstructed.
> >>>>
> >>>> One way this could be achieved is with a tool that postprocesses the
> >>>> artifacts from the build process. For each artifact, the tool would
> >>>> disassemble the artifact, instrument the code using a set of AspectJ
> >>>> aspects and reassemble the artifact for deployment. The responsibility
> >>>> of the aspects would be to intercept messages, log them and modify
> >>>> them to ensure proper correlation. Of course this only works if the
> >>>> artifacts are J2EE deployables (WARs or EARs). Note that we already
> >>>> use AspectJ for a similar use case (although at a much smaller scale)
> >>>> in the transport test suites [1]. Interestingly, this approach would
> >>>> allow to cover interactions that are not SOAP based, e.g. one could
> >>>> even intercept the database queries executed during the flow.
> >>>>
> >>>> Andreas
> >>>>
> >>>> [1]
> >>>>
> https://svn.apache.org/repos/asf/axis/axis2/java/transports/trunk/modules/jms/src/test/java/org/apache/axis2/transport/jms/LogAspect.java
> >>>>
> >>>> On Tue, Mar 30, 2010 at 09:11, Srinath Perera <hemapani@gmail.com>
> wrote:
> >>>> > Hi Nilupa;
> >>>> >
> >>>> > When we collect messages from a one location by installing proper
> >>>> > handler that will intercept and send messages to to that one
> location
> >>>> > (this one location can be a single server, pub/sub channel etc).
> There
> >>>> > is many ways to make sense of those collected messages. What Andreas
> >>>> > mentioned (following complete transaction) is a one possibility.
> >>>> >
> >>>> > I think you should come up with few scenarios on how you would
make
> >>>> > sense of the message.
> >>>> >
> >>>> > Thanks
> >>>> > Srinath
> >>>> >
> >>>> > On Sun, Mar 28, 2010 at 9:46 PM, nilupa bandara <nilupas@gmail.com>
> >>>> > wrote:
> >>>> >> Hi,
> >>>> >>
> >>>> >> First let me thank you for commenting.
> >>>> >>
> >>>> >> As far as I understood, what you would like to see from the
> proposed
> >>>> >> tool is to view set of messages that are exchanged in reponse
to a
> >>>> >> particular input message. With the understanding that I am
having
> at
> >>>> >> the momnet, one way to do it is to filter out the central
> repository
> >>>> >> of messages based on 'To' , 'From' headers and try to contruct
the
> >>>> >> message chain from it. We can allow the client GUI wich connects
to
> >>>> >> the central repository to provide the paramenters (For instance
the
> >>>> >> value of 'To' header) from which an intelligent filtering can
be
> done
> >>>> >> for the set of messages avialable at the central repository.
> >>>> >>
> >>>> >> Perhaps someone has an idea of a better way of doing it and
is
> willing
> >>>> >> to share it with us. It would be really nice to hear from them.
> >>>> >>
> >>>> >> Thanks in advance ..!!
> >>>> >>
> >>>> >> Best Regards,
> >>>> >> Nilupa
> >>>> >>
> >>>> >> On Sat, Mar 27, 2010 at 6:40 PM, Andreas Veithen
> >>>> >> <andreas.veithen@gmail.com> wrote:
> >>>> >>> Personally, I think that the added value of extending the
SOAP
> monitor
> >>>> >>> module to collect messages in a central place is too limited
to
> >>>> >>> attract enough interest from the user community, so that
it will
> be
> >>>> >>> difficult to ensure the evolution of such a project in
the future.
> >>>> >>> However, what many people are looking for is a tool that
allows to
> >>>> >>> track the flow of a message through a distributed system,
or more
> >>>> >>> generally to track the sequence of events triggered by
a given
> input
> >>>> >>> message (sort of end-to-end transaction monitoring).
> >>>> >>>
> >>>> >>> On Wed, Mar 24, 2010 at 23:03, nilupa bandara <nilupas@gmail.com>
> >>>> >>> wrote:
> >>>> >>>> Hello,
> >>>> >>>>
> >>>> >>>> I am graduate student at Politecnica de Madrid and
I am thinking
> of
> >>>> >>>> proposing a GSoC project this summer. I would like
to take
> project
> >>>> >>>> "Distribute TCP Monitor" descried[1] which is very
interesting
> and
> >>>> >>>> also should be helpful to any Java developer using
Apache Axis2
> Web
> >>>> >>>> services middleware. I will submit more detailed proposal
later.
> I
> >>>> >>>> would really like to hear any feedback from you. Any
suggestions
> or
> >>>> >>>> features that you would like to see in a "Distributed
TCP
> Monitor"
> >>>> >>>> are
> >>>> >>>> most welcome.
> >>>> >>>>
> >>>> >>>> Best Regards,
> >>>> >>>> Nilupa Bandara
> >>>> >>>>
> >>>> >>>> [1]
> >>>> >>>>
> >>>> >>>> Distributed TCP monitor
> >>>> >>>>
> >>>> >>>> To use TCP monitor we have to tweak the endpoints,
which is bit
> hard
> >>>> >>>> sometimes (e.g. two channels Async case for Axis2).
We can solve
> the
> >>>> >>>> problem by writing a Axis2 module (Handler) that intercept
> messages
> >>>> >>>> comes in and goes out of a Axis2 server and send to
a UI (via a
> >>>> >>>> pub/sub channel or a message Box e.g. ) or record them
so user
> can
> >>>> >>>> pull those messages. Also, we can do something to turn
the module
> on
> >>>> >>>> and off remotely, so users can have his module deployed,
but turn
> it
> >>>> >>>> on only when they want to debug the system.
> >>>> >>>>
> >>>> >>>> Then we can take this to next level by adding this
module to all
> >>>> >>>> services in a system , configuring modules to send
all collected
> >>>> >>>> messages to a pub/sub channel, and subscribing to those
messages
> via
> >>>> >>>> a
> >>>> >>>> UI, and depicting those messages through a UI. Then
users have a
> >>>> >>>> TCPMon for a whole system.
> >>>> >>>>
> >>>> >>>>
> ---------------------------------------------------------------------
> >>>> >>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> >>>> >>>> For additional commands, e-mail: java-dev-help@axis.apache.org
> >>>> >>>>
> >>>> >>>>
> >>>> >>>
> >>>> >>>
> ---------------------------------------------------------------------
> >>>> >>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> >>>> >>> For additional commands, e-mail: java-dev-help@axis.apache.org
> >>>> >>>
> >>>> >>>
> >>>> >>
> >>>> >>
> ---------------------------------------------------------------------
> >>>> >> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> >>>> >> For additional commands, e-mail: java-dev-help@axis.apache.org
> >>>> >>
> >>>> >>
> >>>> >
> >>>> >
> >>>> >
> >>>> > --
> >>>> > ============================
> >>>> > Srinath Perera, Ph.D.
> >>>> >   WSO2 Inc. http://wso2.com
> >>>> >   Blog: http://srinathsview.blogspot.com/
> >>>> >
> >>>> >
> ---------------------------------------------------------------------
> >>>> > To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> >>>> > For additional commands, e-mail: java-dev-help@axis.apache.org
> >>>> >
> >>>> >
> >>>>
> >>>> ---------------------------------------------------------------------
> >>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> >>>> For additional commands, e-mail: java-dev-help@axis.apache.org
> >>>>
> >>>
> >>>
> >>>
> >>> --
> >>> Sanjiva Weerawarana, Ph.D.
> >>> Founder, Director & Chief Scientist; Lanka Software Foundation;
> >>> http://www.opensource.lk/
> >>> Founder, Chairman & CEO; WSO2, Inc.; http://www.wso2.com/
> >>> Member; Apache Software Foundation; http://www.apache.org/
> >>> Director; Sahana Software Foundation; http://www.sahanafoundation.org/
> >>> Visiting Lecturer; University of Moratuwa; http://www.cse.mrt.ac.lk/
> >>>
> >>> Blog: http://sanjiva.weerawarana.org/
> >>>
> >>
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> >> For additional commands, e-mail: java-dev-help@axis.apache.org
> >>
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> > For additional commands, e-mail: java-dev-help@axis.apache.org
> >
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> For additional commands, e-mail: java-dev-help@axis.apache.org
>
> Samisa...
--
blog: http://samisa-abeysinghe.blogspot.com/

Mime
View raw message