axis-java-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From nilupa bandara <nilu...@gmail.com>
Subject Re: [GSoC Project Proposal] Distributed TCP Monitor
Date Tue, 30 Mar 2010 21:28:32 GMT
Hello Everyone,

First let me thank all of you for your valuable inputs.

Now let me ask few questions to make sure that I understood the
suggested approach correctly.

- What are the components that you meant by 'Components who call
RemoteLogger.log()'  ? A Custom Message Receiver which logs the
messages that are going in (and out) to a Service Impl perhaps ?

- Does Service Impl needs check the availability of RemoteLogger (via
a thread local variable) and log the message it sends ( and receive)
using Service Clients to other applications ? I mean is it the case
where service author needs to insert 'RemoteLogger.Log()' in
appropriate places to ensure complete traceability ?

Using AOP - I just read few documentation on AspectJ and something
like the following could be done (But I'm not absolutely sure though
!!)

We could use some AOP code (package in a jar) which will, just before
each Service Client invocation insert the appropriate debug info
headers to outgoing payload , plus logs the response it gets.

This AOP could be instrumented (or weaved) into all the service
archive files in a given repository.

We could provide a script (and AOP jar) to automate this. For instance
./our_script AOP.jar <path-to-repository> will make all the services
that are available in the repository traceable.

Any suggestions ?

Nilupa


On Tue, Mar 30, 2010 at 3:11 PM, Glen Daniels <glen@thoughtcraft.com> wrote:
> I'm a really terribly inconsistent blogger.  However, my last actual entry,
> from 2009, was about exactly this:
>
> http://glendaniels.blogspot.com/2009/03/debugging-multi-hop-soa-interactions.html
>
> I like Andreas' suggestion below, but I think it might be a bit heavyweight
> for a GSoC project(?).  Maybe we could lower the bar to something like this:
>
> * Build a distributed logging web service which can accept log messages and
> correlationIDs, logging to a file per correlationID.
>
> * Build a Module that processes incoming <debug:logInfo> headers and makes an
> appropriate RemoteLogger (or whatever) available.  This could, at first, go
> in one of Axis2's contexts (OperationContext?) as a demonstration.
> Components who call RemoteLogger.log() send messages (hopefully on another
> thread) to the central log service.
>
> * The Module could automatically put the right metadata on outgoing messages,
> but only the responses, so you still have the problem Andreas points out
> about needing the app to transfer the logging info over to new
> ServiceClients.  I was imagining that could be achieved by also setting a
> ThreadLocal - so if your clients run on the same thread you're all set, and
> if there are new threads it's up to the framework to copy that ThreadLocal over.
>
> * Regardless of the discovery mechanism, you'd want something static like
> RemoteLogger.getLogger() to hide it.
>
> This approach is definitely a bit more intrusive, as it relies on the
> components themselves to be looking for a RemoteLogger, but it seems like a
> reasonable starting point which could be evolved.  However, maybe the AOP
> stuff isn't as challenging as I'm imagining....
>
> Thoughts?
>
> --Glen
>
> On 3/30/2010 4:57 AM, Andreas Veithen wrote:
>> I was actually referring to scenarios where a service may call other
>> services, which in turn may call again other services. In these
>> scenarios, it is not sufficient to simply collect received/sent
>> messages on different hosts, unless the system is isolated
>> (development environment) or the request rate is sufficiently low so
>> that messages can be correlated based on time. Here is a concrete
>> scenario from a project in my company:
>>
>> - Request comes in on an ESB that does security and validation.
>> - Request is processed by an application server which persists the
>> received information and publishes a JMS message (pub/sub events).
>> - The event is consumed by one or more components that may in turn
>> interact with other services.
>>
>> If you want track this flow, it is not sufficient to intercept the
>> messages on the different hosts: in addition, you need to instrument
>> the services so that the outgoing requests are correlated with the
>> incoming requests (by adding SOAP and/or transport headers). What I
>> would like to be able to do in the above scenario is to enable
>> end-to-end monitoring on a per-message basis: I add a special SOAP or
>> HTTP header to the initial request to enable logging and this
>> information is propagated along the chain. Every service/component
>> then sends a copy of the incoming/outgoing requests/responses to a
>> central place where the sequence of events is reconstructed.
>>
>> One way this could be achieved is with a tool that postprocesses the
>> artifacts from the build process. For each artifact, the tool would
>> disassemble the artifact, instrument the code using a set of AspectJ
>> aspects and reassemble the artifact for deployment. The responsibility
>> of the aspects would be to intercept messages, log them and modify
>> them to ensure proper correlation. Of course this only works if the
>> artifacts are J2EE deployables (WARs or EARs). Note that we already
>> use AspectJ for a similar use case (although at a much smaller scale)
>> in the transport test suites [1]. Interestingly, this approach would
>> allow to cover interactions that are not SOAP based, e.g. one could
>> even intercept the database queries executed during the flow.
>>
>> Andreas
>>
>> [1] https://svn.apache.org/repos/asf/axis/axis2/java/transports/trunk/modules/jms/src/test/java/org/apache/axis2/transport/jms/LogAspect.java
>>
>> On Tue, Mar 30, 2010 at 09:11, Srinath Perera <hemapani@gmail.com> wrote:
>>> Hi Nilupa;
>>>
>>> When we collect messages from a one location by installing proper
>>> handler that will intercept and send messages to to that one location
>>> (this one location can be a single server, pub/sub channel etc). There
>>> is many ways to make sense of those collected messages. What Andreas
>>> mentioned (following complete transaction) is a one possibility.
>>>
>>> I think you should come up with few scenarios on how you would make
>>> sense of the message.
>>>
>>> Thanks
>>> Srinath
>>>
>>> On Sun, Mar 28, 2010 at 9:46 PM, nilupa bandara <nilupas@gmail.com> wrote:
>>>> Hi,
>>>>
>>>> First let me thank you for commenting.
>>>>
>>>> As far as I understood, what you would like to see from the proposed
>>>> tool is to view set of messages that are exchanged in reponse to a
>>>> particular input message. With the understanding that I am having at
>>>> the momnet, one way to do it is to filter out the central repository
>>>> of messages based on 'To' , 'From' headers and try to contruct the
>>>> message chain from it. We can allow the client GUI wich connects to
>>>> the central repository to provide the paramenters (For instance the
>>>> value of 'To' header) from which an intelligent filtering can be done
>>>> for the set of messages avialable at the central repository.
>>>>
>>>> Perhaps someone has an idea of a better way of doing it and is willing
>>>> to share it with us. It would be really nice to hear from them.
>>>>
>>>> Thanks in advance ..!!
>>>>
>>>> Best Regards,
>>>> Nilupa
>>>>
>>>> On Sat, Mar 27, 2010 at 6:40 PM, Andreas Veithen
>>>> <andreas.veithen@gmail.com> wrote:
>>>>> Personally, I think that the added value of extending the SOAP monitor
>>>>> module to collect messages in a central place is too limited to
>>>>> attract enough interest from the user community, so that it will be
>>>>> difficult to ensure the evolution of such a project in the future.
>>>>> However, what many people are looking for is a tool that allows to
>>>>> track the flow of a message through a distributed system, or more
>>>>> generally to track the sequence of events triggered by a given input
>>>>> message (sort of end-to-end transaction monitoring).
>>>>>
>>>>> On Wed, Mar 24, 2010 at 23:03, nilupa bandara <nilupas@gmail.com>
wrote:
>>>>>> Hello,
>>>>>>
>>>>>> I am graduate student at Politecnica de Madrid and I am thinking
of
>>>>>> proposing a GSoC project this summer. I would like to take project
>>>>>> "Distribute TCP Monitor" descried[1] which is very interesting and
>>>>>> also should be helpful to any Java developer using Apache Axis2 Web
>>>>>> services middleware. I will submit more detailed proposal later.
I
>>>>>> would really like to hear any feedback from you. Any suggestions
or
>>>>>> features that you would like to see in a "Distributed TCP Monitor"
are
>>>>>> most welcome.
>>>>>>
>>>>>> Best Regards,
>>>>>> Nilupa Bandara
>>>>>>
>>>>>> [1]
>>>>>>
>>>>>> Distributed TCP monitor
>>>>>>
>>>>>> To use TCP monitor we have to tweak the endpoints, which is bit hard
>>>>>> sometimes (e.g. two channels Async case for Axis2). We can solve
the
>>>>>> problem by writing a Axis2 module (Handler) that intercept messages
>>>>>> comes in and goes out of a Axis2 server and send to a UI (via a
>>>>>> pub/sub channel or a message Box e.g. ) or record them so user can
>>>>>> pull those messages. Also, we can do something to turn the module
on
>>>>>> and off remotely, so users can have his module deployed, but turn
it
>>>>>> on only when they want to debug the system.
>>>>>>
>>>>>> Then we can take this to next level by adding this module to all
>>>>>> services in a system , configuring modules to send all collected
>>>>>> messages to a pub/sub channel, and subscribing to those messages
via a
>>>>>> UI, and depicting those messages through a UI. Then users have a
>>>>>> TCPMon for a whole system.
>>>>>>
>>>>>> ---------------------------------------------------------------------
>>>>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>>>>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>>>>>
>>>>>>
>>>>>
>>>>> ---------------------------------------------------------------------
>>>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>>>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>>>>
>>>>>
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>>>
>>>>
>>>
>>>
>>>
>>> --
>>> ============================
>>> Srinath Perera, Ph.D.
>>>   WSO2 Inc. http://wso2.com
>>>   Blog: http://srinathsview.blogspot.com/
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>>
>>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
>> For additional commands, e-mail: java-dev-help@axis.apache.org
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
> For additional commands, e-mail: java-dev-help@axis.apache.org
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
For additional commands, e-mail: java-dev-help@axis.apache.org


Mime
View raw message