axis-java-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Samisa Abeysinghe <samisa.abeysin...@gmail.com>
Subject Re: [GSoC Project Proposal] Distributed TCP Monitor
Date Thu, 01 Apr 2010 04:35:13 GMT
On Thu, Apr 1, 2010 at 7:04 AM, Sanka Samaranayake <ssanka@gmail.com> wrote:

> Samisa,
>
> Quick question, does WSO2 BAM project provide such a debugging
> facility. I mean can one monitor what is going on for a specific input
> in SOA senario ?
>

Yes it can.

However, as I mentioned in an earlier explanation, that is not the main
purpose of BAM.

Samisa...



> Sanka
>
>
>
> On Thu, Apr 1, 2010 at 3:28 AM, Sanka Samaranayake <ssanka@gmail.com>
> wrote:
> > Hi Samisa,
> >
> > I think the this scope of this project would be to provide a
> > light-weight tool which allows to monitor the message flows, execution
> > in each node in response to those message flows to a given input
> > message primarily for debugging purposes . (Just like the way tcpmon
> > is used in developing Web services)
> >
> > And I guess now the question is whether such a tool is useful ..
> >
> > Anyone cares to share their thoughts ?
> >
> > Nilupa
> >
> > On Thu, Apr 1, 2010 at 2:51 AM, Samisa Abeysinghe
> > <samisa.abeysinghe@gmail.com> wrote:
> >>
> >>
> >> On Thu, Apr 1, 2010 at 4:53 AM, nilupa bandara <nilupas@gmail.com>
> wrote:
> >>>
> >>> I was thinking about Java Swing based UI
> >>
> >> Swing is a good start. But if you look at some of the requirements like
> >> monitoring multiple instances and correlate them, discussed in this
> thread,
> >> it sounds as if we would be better off having a Web interface, rather
> than a
> >> desktop client.
> >>
> >>>
> >>> because the client may have
> >>> to cache whatever messages it already has and will receive,
> >>
> >> If you are looking to support message-correlation, as discussed in some
> >> replies in this thread above, you need to have a persistence storage (a
> DB
> >> or something like that) rather than a memory cache.
> >> Because, some message sequences, like those involved in an async
> >> invocations, need to have a way to keep the messages, in there, longer,
> and
> >> may be there is a server re-start, in the middle.
> >> Of course, this requirements spend on the scope of this effort.
> >> Plus, if you want to deal with historic data, and look at what was
> happening
> >> to message sequences over time, you got to use a persistence layer.
> Again
> >> this might be out of scope for this project, but if this is to be
> extended
> >> to support these, you better think of this right now.
> >>
> >>>
> >>> build the
> >>> message flow and update the UI accordingly. Perhaps there is a better
> >>> way ?
> >>
> >> We already do what you are trying to do, and much more with WSO2 BAM. I
> am
> >> tempted to say that Shinding based gadget dashboard is really a
> compelling
> >> UI for this.
> >> I am not trying to take away you project or trying to hinder this effort
> at
> >> all.
> >> I think it will be a good idea for you to have a look at BAM and see
> what
> >> the missing pieces are and figure out the right model for this tool.
> >> Also, having had worked on WSO2 BAM project, I also have some
> understanding
> >> on sort of problems that you might run into, when doing this. For e.g.,
> if
> >> you want to correlate message sequences in a high volume setup, you will
> be
> >> swamped by the volume of messages, sooner than later. So correlation for
> the
> >> purpose of monitoring is not an easy thing to do. Correlation, for the
> >> purpose of debugging will not run into this problem, if you run this on
> a
> >> controlled environment. However, what you really want to debug is the
> deal
> >> clustered setup, and not a staged dev setup, the problem again surfaces.
> >> You might want to list the objectives and then break them down
> >> into monitoring and debugging spaces.
> >> Samisa...
> >>
> >>>
> >>> Thanks
> >>> Nilupa.
> >>>
> >>> On Wed, Mar 31, 2010 at 4:12 PM, Samisa Abeysinghe
> >>> <samisa.abeysinghe@gmail.com> wrote:
> >>> > What UI technology would be used for this?
> >>> >
> >>> > Samisa...
> >>> >
> >>> > On Wed, Mar 31, 2010 at 4:26 AM, Andreas Veithen
> >>> > <andreas.veithen@gmail.com> wrote:
> >>> >> I was thinking more about something like ITCAM for SOA.
> >>> >>
> >>> >> On Tue, Mar 30, 2010 at 13:40, Sanjiva Weerawarana
> >>> >> <sanjiva@opensource.lk> wrote:
> >>> >>> That runtime monitoring / correlation is what the WSO2 Business
> >>> >>> Activity
> >>> >>> Monitor does / is for ...
> >>> >>> See: http://wso2.com/products/business-activity-monitor/
> >>> >>> Sanjiva.
> >>> >>>
> >>> >>> On Tue, Mar 30, 2010 at 2:27 PM, Andreas Veithen
> >>> >>> <andreas.veithen@gmail.com>
> >>> >>> wrote:
> >>> >>>>
> >>> >>>> I was actually referring to scenarios where a service may
call
> other
> >>> >>>> services, which in turn may call again other services.
In these
> >>> >>>> scenarios, it is not sufficient to simply collect received/sent
> >>> >>>> messages on different hosts, unless the system is isolated
> >>> >>>> (development environment) or the request rate is sufficiently
low
> so
> >>> >>>> that messages can be correlated based on time. Here is
a concrete
> >>> >>>> scenario from a project in my company:
> >>> >>>>
> >>> >>>> - Request comes in on an ESB that does security and validation.
> >>> >>>> - Request is processed by an application server which persists
the
> >>> >>>> received information and publishes a JMS message (pub/sub
events).
> >>> >>>> - The event is consumed by one or more components that
may in turn
> >>> >>>> interact with other services.
> >>> >>>>
> >>> >>>> If you want track this flow, it is not sufficient to intercept
the
> >>> >>>> messages on the different hosts: in addition, you need
to
> instrument
> >>> >>>> the services so that the outgoing requests are correlated
with the
> >>> >>>> incoming requests (by adding SOAP and/or transport headers).
What
> I
> >>> >>>> would like to be able to do in the above scenario is to
enable
> >>> >>>> end-to-end monitoring on a per-message basis: I add a special
SOAP
> or
> >>> >>>> HTTP header to the initial request to enable logging and
this
> >>> >>>> information is propagated along the chain. Every service/component
> >>> >>>> then sends a copy of the incoming/outgoing requests/responses
to a
> >>> >>>> central place where the sequence of events is reconstructed.
> >>> >>>>
> >>> >>>> One way this could be achieved is with a tool that postprocesses
> the
> >>> >>>> artifacts from the build process. For each artifact, the
tool
> would
> >>> >>>> disassemble the artifact, instrument the code using a set
of
> AspectJ
> >>> >>>> aspects and reassemble the artifact for deployment. The
> >>> >>>> responsibility
> >>> >>>> of the aspects would be to intercept messages, log them
and modify
> >>> >>>> them to ensure proper correlation. Of course this only
works if
> the
> >>> >>>> artifacts are J2EE deployables (WARs or EARs). Note that
we
> already
> >>> >>>> use AspectJ for a similar use case (although at a much
smaller
> scale)
> >>> >>>> in the transport test suites [1]. Interestingly, this approach
> would
> >>> >>>> allow to cover interactions that are not SOAP based, e.g.
one
> could
> >>> >>>> even intercept the database queries executed during the
flow.
> >>> >>>>
> >>> >>>> Andreas
> >>> >>>>
> >>> >>>> [1]
> >>> >>>>
> >>> >>>>
> https://svn.apache.org/repos/asf/axis/axis2/java/transports/trunk/modules/jms/src/test/java/org/apache/axis2/transport/jms/LogAspect.java
> >>> >>>>
> >>> >>>> On Tue, Mar 30, 2010 at 09:11, Srinath Perera <hemapani@gmail.com
> >
> >>> >>>> wrote:
> >>> >>>> > Hi Nilupa;
> >>> >>>> >
> >>> >>>> > When we collect messages from a one location by installing
> proper
> >>> >>>> > handler that will intercept and send messages to to
that one
> >>> >>>> > location
> >>> >>>> > (this one location can be a single server, pub/sub
channel etc).
> >>> >>>> > There
> >>> >>>> > is many ways to make sense of those collected messages.
What
> >>> >>>> > Andreas
> >>> >>>> > mentioned (following complete transaction) is a one
possibility.
> >>> >>>> >
> >>> >>>> > I think you should come up with few scenarios on how
you would
> make
> >>> >>>> > sense of the message.
> >>> >>>> >
> >>> >>>> > Thanks
> >>> >>>> > Srinath
> >>> >>>> >
> >>> >>>> > On Sun, Mar 28, 2010 at 9:46 PM, nilupa bandara <
> nilupas@gmail.com>
> >>> >>>> > wrote:
> >>> >>>> >> Hi,
> >>> >>>> >>
> >>> >>>> >> First let me thank you for commenting.
> >>> >>>> >>
> >>> >>>> >> As far as I understood, what you would like to
see from the
> >>> >>>> >> proposed
> >>> >>>> >> tool is to view set of messages that are exchanged
in reponse
> to a
> >>> >>>> >> particular input message. With the understanding
that I am
> having
> >>> >>>> >> at
> >>> >>>> >> the momnet, one way to do it is to filter out
the central
> >>> >>>> >> repository
> >>> >>>> >> of messages based on 'To' , 'From' headers and
try to contruct
> the
> >>> >>>> >> message chain from it. We can allow the client
GUI wich
> connects
> >>> >>>> >> to
> >>> >>>> >> the central repository to provide the paramenters
(For instance
> >>> >>>> >> the
> >>> >>>> >> value of 'To' header) from which an intelligent
filtering can
> be
> >>> >>>> >> done
> >>> >>>> >> for the set of messages avialable at the central
repository.
> >>> >>>> >>
> >>> >>>> >> Perhaps someone has an idea of a better way of
doing it and is
> >>> >>>> >> willing
> >>> >>>> >> to share it with us. It would be really nice to
hear from them.
> >>> >>>> >>
> >>> >>>> >> Thanks in advance ..!!
> >>> >>>> >>
> >>> >>>> >> Best Regards,
> >>> >>>> >> Nilupa
> >>> >>>> >>
> >>> >>>> >> On Sat, Mar 27, 2010 at 6:40 PM, Andreas Veithen
> >>> >>>> >> <andreas.veithen@gmail.com> wrote:
> >>> >>>> >>> Personally, I think that the added value of
extending the SOAP
> >>> >>>> >>> monitor
> >>> >>>> >>> module to collect messages in a central place
is too limited
> to
> >>> >>>> >>> attract enough interest from the user community,
so that it
> will
> >>> >>>> >>> be
> >>> >>>> >>> difficult to ensure the evolution of such
a project in the
> >>> >>>> >>> future.
> >>> >>>> >>> However, what many people are looking for
is a tool that
> allows
> >>> >>>> >>> to
> >>> >>>> >>> track the flow of a message through a distributed
system, or
> more
> >>> >>>> >>> generally to track the sequence of events
triggered by a given
> >>> >>>> >>> input
> >>> >>>> >>> message (sort of end-to-end transaction monitoring).
> >>> >>>> >>>
> >>> >>>> >>> On Wed, Mar 24, 2010 at 23:03, nilupa bandara
<
> nilupas@gmail.com>
> >>> >>>> >>> wrote:
> >>> >>>> >>>> Hello,
> >>> >>>> >>>>
> >>> >>>> >>>> I am graduate student at Politecnica de
Madrid and I am
> thinking
> >>> >>>> >>>> of
> >>> >>>> >>>> proposing a GSoC project this summer.
I would like to take
> >>> >>>> >>>> project
> >>> >>>> >>>> "Distribute TCP Monitor" descried[1] which
is very
> interesting
> >>> >>>> >>>> and
> >>> >>>> >>>> also should be helpful to any Java developer
using Apache
> Axis2
> >>> >>>> >>>> Web
> >>> >>>> >>>> services middleware. I will submit more
detailed proposal
> later.
> >>> >>>> >>>> I
> >>> >>>> >>>> would really like to hear any feedback
from you. Any
> suggestions
> >>> >>>> >>>> or
> >>> >>>> >>>> features that you would like to see in
a "Distributed TCP
> >>> >>>> >>>> Monitor"
> >>> >>>> >>>> are
> >>> >>>> >>>> most welcome.
> >>> >>>> >>>>
> >>> >>>> >>>> Best Regards,
> >>> >>>> >>>> Nilupa Bandara
> >>> >>>> >>>>
> >>> >>>> >>>> [1]
> >>> >>>> >>>>
> >>> >>>> >>>> Distributed TCP monitor
> >>> >>>> >>>>
> >>> >>>> >>>> To use TCP monitor we have to tweak the
endpoints, which is
> bit
> >>> >>>> >>>> hard
> >>> >>>> >>>> sometimes (e.g. two channels Async case
for Axis2). We can
> solve
> >>> >>>> >>>> the
> >>> >>>> >>>> problem by writing a Axis2 module (Handler)
that intercept
> >>> >>>> >>>> messages
> >>> >>>> >>>> comes in and goes out of a Axis2 server
and send to a UI (via
> a
> >>> >>>> >>>> pub/sub channel or a message Box e.g.
) or record them so
> user
> >>> >>>> >>>> can
> >>> >>>> >>>> pull those messages. Also, we can do something
to turn the
> >>> >>>> >>>> module on
> >>> >>>> >>>> and off remotely, so users can have his
module deployed, but
> >>> >>>> >>>> turn it
> >>> >>>> >>>> on only when they want to debug the system.
> >>> >>>> >>>>
> >>> >>>> >>>> Then we can take this to next level by
adding this module to
> all
> >>> >>>> >>>> services in a system , configuring modules
to send all
> collected
> >>> >>>> >>>> messages to a pub/sub channel, and subscribing
to those
> messages
> >>> >>>> >>>> via
> >>> >>>> >>>> a
> >>> >>>> >>>> UI, and depicting those messages through
a UI. Then users
> have a
> >>> >>>> >>>> TCPMon for a whole system.
> >>> >>>> >>>>
> >>> >>>> >>>>
> >>> >>>> >>>>
> ---------------------------------------------------------------------
> >>> >>>> >>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> >>> >>>> >>>> For additional commands, e-mail:
> java-dev-help@axis.apache.org
> >>> >>>> >>>>
> >>> >>>> >>>>
> >>> >>>> >>>
> >>> >>>> >>>
> >>> >>>> >>>
> ---------------------------------------------------------------------
> >>> >>>> >>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> >>> >>>> >>> For additional commands, e-mail:
> java-dev-help@axis.apache.org
> >>> >>>> >>>
> >>> >>>> >>>
> >>> >>>> >>
> >>> >>>> >>
> >>> >>>> >>
> ---------------------------------------------------------------------
> >>> >>>> >> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> >>> >>>> >> For additional commands, e-mail: java-dev-help@axis.apache.org
> >>> >>>> >>
> >>> >>>> >>
> >>> >>>> >
> >>> >>>> >
> >>> >>>> >
> >>> >>>> > --
> >>> >>>> > ============================
> >>> >>>> > Srinath Perera, Ph.D.
> >>> >>>> >   WSO2 Inc. http://wso2.com
> >>> >>>> >   Blog: http://srinathsview.blogspot.com/
> >>> >>>> >
> >>> >>>> >
> >>> >>>> >
> ---------------------------------------------------------------------
> >>> >>>> > To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> >>> >>>> > For additional commands, e-mail: java-dev-help@axis.apache.org
> >>> >>>> >
> >>> >>>> >
> >>> >>>>
> >>> >>>>
> ---------------------------------------------------------------------
> >>> >>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> >>> >>>> For additional commands, e-mail: java-dev-help@axis.apache.org
> >>> >>>>
> >>> >>>
> >>> >>>
> >>> >>>
> >>> >>> --
> >>> >>> Sanjiva Weerawarana, Ph.D.
> >>> >>> Founder, Director & Chief Scientist; Lanka Software Foundation;
> >>> >>> http://www.opensource.lk/
> >>> >>> Founder, Chairman & CEO; WSO2, Inc.; http://www.wso2.com/
> >>> >>> Member; Apache Software Foundation; http://www.apache.org/
> >>> >>> Director; Sahana Software Foundation;
> http://www.sahanafoundation.org/
> >>> >>> Visiting Lecturer; University of Moratuwa;
> http://www.cse.mrt.ac.lk/
> >>> >>>
> >>> >>> Blog: http://sanjiva.weerawarana.org/
> >>> >>>
> >>> >>
> >>> >>
> >>> >>
> ---------------------------------------------------------------------
> >>> >> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> >>> >> For additional commands, e-mail: java-dev-help@axis.apache.org
> >>> >>
> >>> >
> >>> > ---------------------------------------------------------------------
> >>> > To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> >>> > For additional commands, e-mail: java-dev-help@axis.apache.org
> >>> >
> >>> >
> >>>
> >>> ---------------------------------------------------------------------
> >>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> >>> For additional commands, e-mail: java-dev-help@axis.apache.org
> >>>
> >> Samisa...
> >> --
> >> blog: http://samisa-abeysinghe.blogspot.com/
> >>
> >
> >
> >
> > --
> > Sanka Samaranayake
> >
> > http://sankas.blogspot.com/
> >
>
>
>
> --
> Sanka Samaranayake
>
> http://sankas.blogspot.com/
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> For additional commands, e-mail: java-dev-help@axis.apache.org
>
> Samisa...
--
blog: http://samisa-abeysinghe.blogspot.com/

Mime
View raw message