axis-java-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Glen Daniels <g...@thoughtcraft.com>
Subject Re: [GSoC Project Proposal] Distributed TCP Monitor
Date Tue, 30 Mar 2010 13:11:14 GMT
I'm a really terribly inconsistent blogger.  However, my last actual entry,
from 2009, was about exactly this:

http://glendaniels.blogspot.com/2009/03/debugging-multi-hop-soa-interactions.html

I like Andreas' suggestion below, but I think it might be a bit heavyweight
for a GSoC project(?).  Maybe we could lower the bar to something like this:

* Build a distributed logging web service which can accept log messages and
correlationIDs, logging to a file per correlationID.

* Build a Module that processes incoming <debug:logInfo> headers and makes an
appropriate RemoteLogger (or whatever) available.  This could, at first, go
in one of Axis2's contexts (OperationContext?) as a demonstration.
Components who call RemoteLogger.log() send messages (hopefully on another
thread) to the central log service.

* The Module could automatically put the right metadata on outgoing messages,
but only the responses, so you still have the problem Andreas points out
about needing the app to transfer the logging info over to new
ServiceClients.  I was imagining that could be achieved by also setting a
ThreadLocal - so if your clients run on the same thread you're all set, and
if there are new threads it's up to the framework to copy that ThreadLocal over.

* Regardless of the discovery mechanism, you'd want something static like
RemoteLogger.getLogger() to hide it.

This approach is definitely a bit more intrusive, as it relies on the
components themselves to be looking for a RemoteLogger, but it seems like a
reasonable starting point which could be evolved.  However, maybe the AOP
stuff isn't as challenging as I'm imagining....

Thoughts?

--Glen

On 3/30/2010 4:57 AM, Andreas Veithen wrote:
> I was actually referring to scenarios where a service may call other
> services, which in turn may call again other services. In these
> scenarios, it is not sufficient to simply collect received/sent
> messages on different hosts, unless the system is isolated
> (development environment) or the request rate is sufficiently low so
> that messages can be correlated based on time. Here is a concrete
> scenario from a project in my company:
> 
> - Request comes in on an ESB that does security and validation.
> - Request is processed by an application server which persists the
> received information and publishes a JMS message (pub/sub events).
> - The event is consumed by one or more components that may in turn
> interact with other services.
> 
> If you want track this flow, it is not sufficient to intercept the
> messages on the different hosts: in addition, you need to instrument
> the services so that the outgoing requests are correlated with the
> incoming requests (by adding SOAP and/or transport headers). What I
> would like to be able to do in the above scenario is to enable
> end-to-end monitoring on a per-message basis: I add a special SOAP or
> HTTP header to the initial request to enable logging and this
> information is propagated along the chain. Every service/component
> then sends a copy of the incoming/outgoing requests/responses to a
> central place where the sequence of events is reconstructed.
> 
> One way this could be achieved is with a tool that postprocesses the
> artifacts from the build process. For each artifact, the tool would
> disassemble the artifact, instrument the code using a set of AspectJ
> aspects and reassemble the artifact for deployment. The responsibility
> of the aspects would be to intercept messages, log them and modify
> them to ensure proper correlation. Of course this only works if the
> artifacts are J2EE deployables (WARs or EARs). Note that we already
> use AspectJ for a similar use case (although at a much smaller scale)
> in the transport test suites [1]. Interestingly, this approach would
> allow to cover interactions that are not SOAP based, e.g. one could
> even intercept the database queries executed during the flow.
> 
> Andreas
> 
> [1] https://svn.apache.org/repos/asf/axis/axis2/java/transports/trunk/modules/jms/src/test/java/org/apache/axis2/transport/jms/LogAspect.java
> 
> On Tue, Mar 30, 2010 at 09:11, Srinath Perera <hemapani@gmail.com> wrote:
>> Hi Nilupa;
>>
>> When we collect messages from a one location by installing proper
>> handler that will intercept and send messages to to that one location
>> (this one location can be a single server, pub/sub channel etc). There
>> is many ways to make sense of those collected messages. What Andreas
>> mentioned (following complete transaction) is a one possibility.
>>
>> I think you should come up with few scenarios on how you would make
>> sense of the message.
>>
>> Thanks
>> Srinath
>>
>> On Sun, Mar 28, 2010 at 9:46 PM, nilupa bandara <nilupas@gmail.com> wrote:
>>> Hi,
>>>
>>> First let me thank you for commenting.
>>>
>>> As far as I understood, what you would like to see from the proposed
>>> tool is to view set of messages that are exchanged in reponse to a
>>> particular input message. With the understanding that I am having at
>>> the momnet, one way to do it is to filter out the central repository
>>> of messages based on 'To' , 'From' headers and try to contruct the
>>> message chain from it. We can allow the client GUI wich connects to
>>> the central repository to provide the paramenters (For instance the
>>> value of 'To' header) from which an intelligent filtering can be done
>>> for the set of messages avialable at the central repository.
>>>
>>> Perhaps someone has an idea of a better way of doing it and is willing
>>> to share it with us. It would be really nice to hear from them.
>>>
>>> Thanks in advance ..!!
>>>
>>> Best Regards,
>>> Nilupa
>>>
>>> On Sat, Mar 27, 2010 at 6:40 PM, Andreas Veithen
>>> <andreas.veithen@gmail.com> wrote:
>>>> Personally, I think that the added value of extending the SOAP monitor
>>>> module to collect messages in a central place is too limited to
>>>> attract enough interest from the user community, so that it will be
>>>> difficult to ensure the evolution of such a project in the future.
>>>> However, what many people are looking for is a tool that allows to
>>>> track the flow of a message through a distributed system, or more
>>>> generally to track the sequence of events triggered by a given input
>>>> message (sort of end-to-end transaction monitoring).
>>>>
>>>> On Wed, Mar 24, 2010 at 23:03, nilupa bandara <nilupas@gmail.com> wrote:
>>>>> Hello,
>>>>>
>>>>> I am graduate student at Politecnica de Madrid and I am thinking of
>>>>> proposing a GSoC project this summer. I would like to take project
>>>>> "Distribute TCP Monitor" descried[1] which is very interesting and
>>>>> also should be helpful to any Java developer using Apache Axis2 Web
>>>>> services middleware. I will submit more detailed proposal later. I
>>>>> would really like to hear any feedback from you. Any suggestions or
>>>>> features that you would like to see in a "Distributed TCP Monitor" are
>>>>> most welcome.
>>>>>
>>>>> Best Regards,
>>>>> Nilupa Bandara
>>>>>
>>>>> [1]
>>>>>
>>>>> Distributed TCP monitor
>>>>>
>>>>> To use TCP monitor we have to tweak the endpoints, which is bit hard
>>>>> sometimes (e.g. two channels Async case for Axis2). We can solve the
>>>>> problem by writing a Axis2 module (Handler) that intercept messages
>>>>> comes in and goes out of a Axis2 server and send to a UI (via a
>>>>> pub/sub channel or a message Box e.g. ) or record them so user can
>>>>> pull those messages. Also, we can do something to turn the module on
>>>>> and off remotely, so users can have his module deployed, but turn it
>>>>> on only when they want to debug the system.
>>>>>
>>>>> Then we can take this to next level by adding this module to all
>>>>> services in a system , configuring modules to send all collected
>>>>> messages to a pub/sub channel, and subscribing to those messages via
a
>>>>> UI, and depicting those messages through a UI. Then users have a
>>>>> TCPMon for a whole system.
>>>>>
>>>>> ---------------------------------------------------------------------
>>>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>>>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>>>>
>>>>>
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>>>
>>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>>
>>>
>>
>>
>>
>> --
>> ============================
>> Srinath Perera, Ph.D.
>>   WSO2 Inc. http://wso2.com
>>   Blog: http://srinathsview.blogspot.com/
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>
>>
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> For additional commands, e-mail: java-dev-help@axis.apache.org
> 

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
For additional commands, e-mail: java-dev-help@axis.apache.org


Mime
View raw message