axis-java-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andreas Veithen <>
Subject Re: [GSoC Project Proposal] Distributed TCP Monitor
Date Tue, 30 Mar 2010 22:17:25 GMT
Probably we are already going too far into implementation details. I
think that the project would have 3 parts:

1. Establishing the general (implementation independent) framework:

- Define protocols: SOAP headers, but also mappings to HTTP headers
(for REST) and JMS properties.
- Define the structure of the correlation identifiers so that the
entire sequence can be reconstructed in a meaningful way.
- Look for strategies to detect and/or avoid duplicate logging
(because of redundant interceptors or simply because a message is
logged at the transmitter and receiver side).
- Find solutions for cases where the message is not XML.
- Think about what happens if one of the services doesn't understand
the protocol.
- Think about security implications.
- ...

2. Build a generic Java framework that supports this style of logging
(with components for remote logging, classes that represent the
context, tools to browse the intercepted messages, etc.).

3. Build a set of interceptors that make use of this framework (and
that log the messages and propagate the context). This can be Axis2
modules, JAX-WS handlers or AspectJ aspects.

On Tue, Mar 30, 2010 at 23:28, nilupa bandara <> wrote:
> Hello Everyone,
> First let me thank all of you for your valuable inputs.
> Now let me ask few questions to make sure that I understood the
> suggested approach correctly.
> - What are the components that you meant by 'Components who call
> RemoteLogger.log()'  ? A Custom Message Receiver which logs the
> messages that are going in (and out) to a Service Impl perhaps ?
> - Does Service Impl needs check the availability of RemoteLogger (via
> a thread local variable) and log the message it sends ( and receive)
> using Service Clients to other applications ? I mean is it the case
> where service author needs to insert 'RemoteLogger.Log()' in
> appropriate places to ensure complete traceability ?
> Using AOP - I just read few documentation on AspectJ and something
> like the following could be done (But I'm not absolutely sure though
> !!)
> We could use some AOP code (package in a jar) which will, just before
> each Service Client invocation insert the appropriate debug info
> headers to outgoing payload , plus logs the response it gets.
> This AOP could be instrumented (or weaved) into all the service
> archive files in a given repository.
> We could provide a script (and AOP jar) to automate this. For instance
> ./our_script AOP.jar <path-to-repository> will make all the services
> that are available in the repository traceable.
> Any suggestions ?
> Nilupa
> On Tue, Mar 30, 2010 at 3:11 PM, Glen Daniels <> wrote:
>> I'm a really terribly inconsistent blogger.  However, my last actual entry,
>> from 2009, was about exactly this:
>> I like Andreas' suggestion below, but I think it might be a bit heavyweight
>> for a GSoC project(?).  Maybe we could lower the bar to something like this:
>> * Build a distributed logging web service which can accept log messages and
>> correlationIDs, logging to a file per correlationID.
>> * Build a Module that processes incoming <debug:logInfo> headers and makes
>> appropriate RemoteLogger (or whatever) available.  This could, at first, go
>> in one of Axis2's contexts (OperationContext?) as a demonstration.
>> Components who call RemoteLogger.log() send messages (hopefully on another
>> thread) to the central log service.
>> * The Module could automatically put the right metadata on outgoing messages,
>> but only the responses, so you still have the problem Andreas points out
>> about needing the app to transfer the logging info over to new
>> ServiceClients.  I was imagining that could be achieved by also setting a
>> ThreadLocal - so if your clients run on the same thread you're all set, and
>> if there are new threads it's up to the framework to copy that ThreadLocal over.
>> * Regardless of the discovery mechanism, you'd want something static like
>> RemoteLogger.getLogger() to hide it.
>> This approach is definitely a bit more intrusive, as it relies on the
>> components themselves to be looking for a RemoteLogger, but it seems like a
>> reasonable starting point which could be evolved.  However, maybe the AOP
>> stuff isn't as challenging as I'm imagining....
>> Thoughts?
>> --Glen
>> On 3/30/2010 4:57 AM, Andreas Veithen wrote:
>>> I was actually referring to scenarios where a service may call other
>>> services, which in turn may call again other services. In these
>>> scenarios, it is not sufficient to simply collect received/sent
>>> messages on different hosts, unless the system is isolated
>>> (development environment) or the request rate is sufficiently low so
>>> that messages can be correlated based on time. Here is a concrete
>>> scenario from a project in my company:
>>> - Request comes in on an ESB that does security and validation.
>>> - Request is processed by an application server which persists the
>>> received information and publishes a JMS message (pub/sub events).
>>> - The event is consumed by one or more components that may in turn
>>> interact with other services.
>>> If you want track this flow, it is not sufficient to intercept the
>>> messages on the different hosts: in addition, you need to instrument
>>> the services so that the outgoing requests are correlated with the
>>> incoming requests (by adding SOAP and/or transport headers). What I
>>> would like to be able to do in the above scenario is to enable
>>> end-to-end monitoring on a per-message basis: I add a special SOAP or
>>> HTTP header to the initial request to enable logging and this
>>> information is propagated along the chain. Every service/component
>>> then sends a copy of the incoming/outgoing requests/responses to a
>>> central place where the sequence of events is reconstructed.
>>> One way this could be achieved is with a tool that postprocesses the
>>> artifacts from the build process. For each artifact, the tool would
>>> disassemble the artifact, instrument the code using a set of AspectJ
>>> aspects and reassemble the artifact for deployment. The responsibility
>>> of the aspects would be to intercept messages, log them and modify
>>> them to ensure proper correlation. Of course this only works if the
>>> artifacts are J2EE deployables (WARs or EARs). Note that we already
>>> use AspectJ for a similar use case (although at a much smaller scale)
>>> in the transport test suites [1]. Interestingly, this approach would
>>> allow to cover interactions that are not SOAP based, e.g. one could
>>> even intercept the database queries executed during the flow.
>>> Andreas
>>> [1]
>>> On Tue, Mar 30, 2010 at 09:11, Srinath Perera <> wrote:
>>>> Hi Nilupa;
>>>> When we collect messages from a one location by installing proper
>>>> handler that will intercept and send messages to to that one location
>>>> (this one location can be a single server, pub/sub channel etc). There
>>>> is many ways to make sense of those collected messages. What Andreas
>>>> mentioned (following complete transaction) is a one possibility.
>>>> I think you should come up with few scenarios on how you would make
>>>> sense of the message.
>>>> Thanks
>>>> Srinath
>>>> On Sun, Mar 28, 2010 at 9:46 PM, nilupa bandara <>
>>>>> Hi,
>>>>> First let me thank you for commenting.
>>>>> As far as I understood, what you would like to see from the proposed
>>>>> tool is to view set of messages that are exchanged in reponse to a
>>>>> particular input message. With the understanding that I am having at
>>>>> the momnet, one way to do it is to filter out the central repository
>>>>> of messages based on 'To' , 'From' headers and try to contruct the
>>>>> message chain from it. We can allow the client GUI wich connects to
>>>>> the central repository to provide the paramenters (For instance the
>>>>> value of 'To' header) from which an intelligent filtering can be done
>>>>> for the set of messages avialable at the central repository.
>>>>> Perhaps someone has an idea of a better way of doing it and is willing
>>>>> to share it with us. It would be really nice to hear from them.
>>>>> Thanks in advance ..!!
>>>>> Best Regards,
>>>>> Nilupa
>>>>> On Sat, Mar 27, 2010 at 6:40 PM, Andreas Veithen
>>>>> <> wrote:
>>>>>> Personally, I think that the added value of extending the SOAP monitor
>>>>>> module to collect messages in a central place is too limited to
>>>>>> attract enough interest from the user community, so that it will
>>>>>> difficult to ensure the evolution of such a project in the future.
>>>>>> However, what many people are looking for is a tool that allows to
>>>>>> track the flow of a message through a distributed system, or more
>>>>>> generally to track the sequence of events triggered by a given input
>>>>>> message (sort of end-to-end transaction monitoring).
>>>>>> On Wed, Mar 24, 2010 at 23:03, nilupa bandara <>
>>>>>>> Hello,
>>>>>>> I am graduate student at Politecnica de Madrid and I am thinking
>>>>>>> proposing a GSoC project this summer. I would like to take project
>>>>>>> "Distribute TCP Monitor" descried[1] which is very interesting
>>>>>>> also should be helpful to any Java developer using Apache Axis2
>>>>>>> services middleware. I will submit more detailed proposal later.
>>>>>>> would really like to hear any feedback from you. Any suggestions
>>>>>>> features that you would like to see in a "Distributed TCP Monitor"
>>>>>>> most welcome.
>>>>>>> Best Regards,
>>>>>>> Nilupa Bandara
>>>>>>> [1]
>>>>>>> Distributed TCP monitor
>>>>>>> To use TCP monitor we have to tweak the endpoints, which is bit
>>>>>>> sometimes (e.g. two channels Async case for Axis2). We can solve
>>>>>>> problem by writing a Axis2 module (Handler) that intercept messages
>>>>>>> comes in and goes out of a Axis2 server and send to a UI (via
>>>>>>> pub/sub channel or a message Box e.g. ) or record them so user
>>>>>>> pull those messages. Also, we can do something to turn the module
>>>>>>> and off remotely, so users can have his module deployed, but
turn it
>>>>>>> on only when they want to debug the system.
>>>>>>> Then we can take this to next level by adding this module to
>>>>>>> services in a system , configuring modules to send all collected
>>>>>>> messages to a pub/sub channel, and subscribing to those messages
via a
>>>>>>> UI, and depicting those messages through a UI. Then users have
>>>>>>> TCPMon for a whole system.
>>>>>>> ---------------------------------------------------------------------
>>>>>>> To unsubscribe, e-mail:
>>>>>>> For additional commands, e-mail:
>>>>>> ---------------------------------------------------------------------
>>>>>> To unsubscribe, e-mail:
>>>>>> For additional commands, e-mail:
>>>>> ---------------------------------------------------------------------
>>>>> To unsubscribe, e-mail:
>>>>> For additional commands, e-mail:
>>>> --
>>>> ============================
>>>> Srinath Perera, Ph.D.
>>>>   WSO2 Inc.
>>>>   Blog:
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail:
>>>> For additional commands, e-mail:
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail:
>>> For additional commands, e-mail:
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail:
>> For additional commands, e-mail:
> ---------------------------------------------------------------------
> To unsubscribe, e-mail:
> For additional commands, e-mail:

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message