activemq-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Gerdes, Mike" <mike.ger...@airbus.com>
Subject AW: AW: AW: AW: Final repost: SSL transport failure
Date Wed, 29 Mar 2006 08:30:10 GMT

Hi Steve,

I have tried to reproduce your exception and don't get the same,but a very similar exception.
I can send much much bigger messages with tcp then with ssl. With ssl I can send about a message
that contains 575 of these loops:

        int i = 0;
        String s = "<blabla>";
        while (i<3000) {
        	s = s + "<timestamp> 666 </timestamp>";
        	System.out.println(i+ "\n");
        	i++;
        }
    
        s = s+ "</blabla>";

With tcp I have tried about 3000 and no problem. So whenI look at my numbers and your numbers
and at tcp and ssl, I get the feeling that it is a timeout problem of some kind that depends
much on the power of your machine.
It looks like the ssl processing takes to much time and power, so that a timeout is thrown
out that causes these exceptions. Have you tried to reproduce the exception at another machine?
I will search a bit where the timeout is set and do a few experiments.

cya

mike

-----Ursprüngliche Nachricht-----
Von: Steve Barham [mailto:steve.barham@formicary.net]
Gesendet: Mittwoch, 29. März 2006 10:12
An: activemq-users@geronimo.apache.org
Betreff: Re: AW: AW: AW: Final repost: SSL transport failure



Hi Mike,

Thanks for your continued input. Unfortunately, the message definitely
does not get onto the queue - the client does not receive it, and after
the exception the server seems to be disconnected from the broker - it can
no longer send or consume messages.

The error did not occur with the TCP transport. I've not had a chance to
try any of the others, as the TCP and SSL transports seemed the most sane
for my application.

Steve


> hi,
>
> the whole thing sounds a bit strange. It looks like the file is transfered
> successfully into the queue and then the connection is closed. Can your
> client access the message successfully? I am not sure if this error is
> critical, it is just a warning after all. Have you tried it without the
> SSL protocol and did the error also occure?
> From my understanding of AMQ a message of about 700kb should not be a big
> problem.
> AMQ get a EOS packet and passes it through to the
> asyncchanneltoasynccommandchannel, where it throws the peer disconnected
> exception. That is then handed to the activemqconnection where the no
> exception listener warning it thrown out, if there would have been a
> exception listener it the error would have been passed further on.
> But to be honest I have no more clues beside this. My understanding of the
> internal mechanism and of activeio and how messages are transported is
> just to bad. I would say this exception is no big deal, but I have no idea
> what the EOS packet means.
>
> -----Ursprüngliche Nachricht-----
> Von: Steve Barham [mailto:steve.barham@formicary.net]
> Gesendet: Dienstag, 28. März 2006 17:48
> An: activemq-users@geronimo.apache.org
> Betreff: Re: AW: AW: Final repost: SSL transport failure
>
>
>
> Hi Mike,
>
> Thanks for the input. It's a single (albeit large) message. And I'm not
> sure as to why this would cause a failure in my 'server' application?
>
> 3163 SERVER <---> Broker : Queue <---> CLIENT 3175
>
> To me, the fact that it always occurs on a message of this size suggests
> that some internal buffer is being overflowed (hence the lines:)
>
>> Socket Connection: /127.0.0.1:61616 -> /127.0.0.1:3163, called close()
>> Socket Connection: /127.0.0.1:61616 -> /127.0.0.1:3163, called
> closeInternal(true)
>> Socket Connection: /127.0.0.1:61616 -> /127.0.0.1:3163, SEND TLSv1
> ALERT: warning, description = close_notify
>> Socket Connection: /127.0.0.1:61616 -> /127.0.0.1:3163, WRITE: TLSv1
> Alert, length = 18
>> Socket Connection: /127.0.0.1:61616 -> /127.0.0.1:3163, called close()
>> Socket Connection: /127.0.0.1:61616 -> /127.0.0.1:3163, called
> closeInternal(true)
>> Socket Connection: /127.0.0.1:61616 -> /127.0.0.1:3163, called close()
>> Socket Connection: /127.0.0.1:61616 -> /127.0.0.1:3163, called
> closeInternal(true)
>
> Surely it's the job of ActiveMQ to split large messages where the
> underlying transport requires short messages, or to maintain a connection
> where the timeout is so short? I've not seen such problems using SSL
> Sockets before now in other Java applications.
>
> Cheers,
>
> steve
>
>
>>
>> hmm I think it looks like the app 3175 isn'T replying.  When my
>> understanding is correct the timeout of SSL is about 500ms, maybe
>> something takes to long. Or could it be that the app has closed the
>> connection?
>> Have you tried to send a lower number of messages and tried to find at
>> if
>> it stops to work at a certain threshold?
>>
>> -----Ursprüngliche Nachricht-----
>> Von: Steve Barham [mailto:steve.barham@formicary.net]
>> Gesendet: Dienstag, 28. März 2006 17:07
>> An: activemq-users@geronimo.apache.org
>> Betreff: Re: AW: Final repost: SSL transport failure
>>
>>
>>
>> With SSL debug on, this is the output from the broker during the failure
>> (but no stack traces at debug log4j level):
>>
>> Socket Connection: /127.0.0.1:61616 -> /127.0.0.1:3163, READ: TLSv1
>> Application
>> Data, length = 16400
>> Socket Connection: /127.0.0.1:61616 -> /127.0.0.1:3163, READ: TLSv1
>> Application
>> Data, length = 16400
>> Socket Connection: /127.0.0.1:61616 -> /127.0.0.1:3163, READ: TLSv1
>> Application
>> Data, length = 16400
>> Socket Connection: /127.0.0.1:61616 -> /127.0.0.1:3163, READ: TLSv1
>> Application
>> Data, length = 16400
>> Socket Connection: /127.0.0.1:61616 -> /127.0.0.1:3163, READ: TLSv1
>> Application
>> Data, length = 16400
>> Socket Connection: /127.0.0.1:61616 -> /127.0.0.1:3163, READ: TLSv1
>> Application
>> Data, length = 16400
>> Socket Connection: /127.0.0.1:61616 -> /127.0.0.1:3163, READ: TLSv1
>> Application
>> Data, length = 16400
>> Socket Connection: /127.0.0.1:61616 -> /127.0.0.1:3163, READ: TLSv1
>> Application
>> Data, length = 16400
>> Socket Connection: /127.0.0.1:61616 -> /127.0.0.1:3163, READ: TLSv1
>> Application
>> Data, length = 16400
>> Socket Connection: /127.0.0.1:61616 -> /127.0.0.1:3163, READ: TLSv1
>> Application
>> Data, length = 16400
>> Socket Connection: /127.0.0.1:61616 -> /127.0.0.1:3163, READ: TLSv1
>> Application
>> Data, length = 16400
>> Socket Connection: /127.0.0.1:61616 -> /127.0.0.1:3163, READ: TLSv1
>> Application
>> Data, length = 16400
>> Socket Connection: /127.0.0.1:61616 -> /127.0.0.1:3163, READ: TLSv1
>> Application
>> Data, length = 16400
>> Socket Connection: /127.0.0.1:61616 -> /127.0.0.1:3163, READ: TLSv1
>> Application
>> Data, length = 16400
>> Socket Connection: /127.0.0.1:61616 -> /127.0.0.1:3163, READ: TLSv1
>> Application
>> Data, length = 16400
>> Socket Connection: /127.0.0.1:61616 -> /127.0.0.1:3163, READ: TLSv1
>> Application
>> Data, length = 16400
>> Socket Connection: /127.0.0.1:61616 -> /127.0.0.1:3163, READ: TLSv1
>> Application
>> Data, length = 16400
>> Socket Connection: /127.0.0.1:61616 -> /127.0.0.1:3163, READ: TLSv1
>> Application
>> Data, length = 16400
>> Socket Connection: /127.0.0.1:61616 -> /127.0.0.1:3163, READ: TLSv1
>> Application
>> Data, length = 16400
>> Socket Connection: /127.0.0.1:61616 -> /127.0.0.1:3163, READ: TLSv1
>> Application
>> Data, length = 16400
>> Socket Connection: /127.0.0.1:61616 -> /127.0.0.1:3163, READ: TLSv1
>> Application
>> Data, length = 16400
>> Socket Connection: /127.0.0.1:61616 -> /127.0.0.1:3163, READ: TLSv1
>> Application
>> Data, length = 16400
>> Socket Connection: /127.0.0.1:61616 -> /127.0.0.1:3163, READ: TLSv1
>> Application
>> Data, length = 16400
>> Socket Connection: /127.0.0.1:61616 -> /127.0.0.1:3163, READ: TLSv1
>> Application
>> Data, length = 16400
>> Socket Connection: /127.0.0.1:61616 -> /127.0.0.1:3163, READ: TLSv1
>> Application
>> Data, length = 16400
>> Socket Connection: /127.0.0.1:61616 -> /127.0.0.1:3163, READ: TLSv1
>> Application
>> Data, length = 16400
>> Socket Connection: /127.0.0.1:61616 -> /127.0.0.1:3163, READ: TLSv1
>> Application
>> Data, length = 16400
>> Socket Connection: /127.0.0.1:61616 -> /127.0.0.1:3163, READ: TLSv1
>> Application
>> Data, length = 16400
>> Socket Connection: /127.0.0.1:61616 -> /127.0.0.1:3163, READ: TLSv1
>> Application
>> Data, length = 16400
>> Socket Connection: /127.0.0.1:61616 -> /127.0.0.1:3163, READ: TLSv1
>> Application
>> Data, length = 16400
>> Socket Connection: /127.0.0.1:61616 -> /127.0.0.1:3163, READ: TLSv1
>> Application
>> Data, length = 16400
>> Socket Connection: /127.0.0.1:61616 -> /127.0.0.1:3163, READ: TLSv1
>> Application
>> Data, length = 16400
>> Socket Connection: /127.0.0.1:61616 -> /127.0.0.1:3163, READ: TLSv1
>> Application
>> Data, length = 16400
>> Socket Connection: /127.0.0.1:61616 -> /127.0.0.1:3163, READ: TLSv1
>> Application
>> Data, length = 16400
>> Socket Connection: /127.0.0.1:61616 -> /127.0.0.1:3163, READ: TLSv1
>> Application
>> Data, length = 4814
>> Socket Connection: /127.0.0.1:61616 -> /127.0.0.1:3163, WRITE: TLSv1
>> Application
>>  Data, length = 27
>> Socket Connection: /127.0.0.1:61616 -> /127.0.0.1:3175, handling
>> exception: java
>> .net.SocketTimeoutException: Read timed out
>> Socket Connection: /127.0.0.1:61616 -> /127.0.0.1:3175, handling
>> exception: java
>> .net.SocketTimeoutException: Read timed out
>> Socket Connection: /127.0.0.1:61616 -> /127.0.0.1:3175, handling
>> exception: java
>> .net.SocketTimeoutException: Read timed out
>> Socket Connection: /127.0.0.1:61616 -> /127.0.0.1:3175, handling
>> exception: java
>> .net.SocketTimeoutException: Read timed out
>> Socket Connection: /127.0.0.1:61616 -> /127.0.0.1:3175, handling
>> exception: java
>> .net.SocketTimeoutException: Read timed out
>> Socket Connection: /127.0.0.1:61616 -> /127.0.0.1:3175, handling
>> exception: java
>> .net.SocketTimeoutException: Read timed out
>> Socket Connection: /127.0.0.1:61616 -> /127.0.0.1:3175, handling
>> exception: java
>> .net.SocketTimeoutException: Read timed out
>> Socket Connection: /127.0.0.1:61616 -> /127.0.0.1:3175, handling
>> exception: java
>> .net.SocketTimeoutException: Read timed out
>> Socket Connection: /127.0.0.1:61616 -> /127.0.0.1:3175, handling
>> exception: java
>> .net.SocketTimeoutException: Read timed out
>> Socket Connection: /127.0.0.1:61616 -> /127.0.0.1:3175, handling
>> exception: java
>> .net.SocketTimeoutException: Read timed out
>> Socket Connection: /127.0.0.1:61616 -> /127.0.0.1:3163, called close()
>> Socket Connection: /127.0.0.1:61616 -> /127.0.0.1:3163, called
>> closeInternal(true)
>> Socket Connection: /127.0.0.1:61616 -> /127.0.0.1:3163, SEND TLSv1
>> ALERT:
>> warning, description = close_notify
>> Socket Connection: /127.0.0.1:61616 -> /127.0.0.1:3163, WRITE: TLSv1
>> Alert, length = 18
>> Socket Connection: /127.0.0.1:61616 -> /127.0.0.1:3163, called close()
>> Socket Connection: /127.0.0.1:61616 -> /127.0.0.1:3163, called
>> closeInternal(true)
>> Socket Connection: /127.0.0.1:61616 -> /127.0.0.1:3163, called close()
>> Socket Connection: /127.0.0.1:61616 -> /127.0.0.1:3163, called
>> closeInternal(true)
>>
>> Port 3163 in this instance is my 'server' app (one JMS client), and 3175
>> the 'client' app. I'm assuming that the socket timeout exceptions are
>> normal, and part of the Java SSL impl. as they occur at very regular
>> intervals.
>>
>> Corresponding 'server' stacktrace is:
>>
>> WARN [2006-03-28 16:00:37,484] activemq.ActiveMQConnection "Async
>> exception with no exception listener: java.io.EOFException: Peer
>> disconnected."
>> java.io.EOFException: Peer disconnected.
>> 	at
>> org.apache.activeio.command.AsyncChannelToAsyncCommandChannel$1.onPacket(AsyncChannelToAsyncCommandChannel.java:64)
>> 	at
>> org.apache.activemq.transport.activeio.PacketAggregatingAsyncChannel.onPacket(PacketAggregatingAsyncChannel.java:59)
>> 	at
>> org.apache.activeio.packet.async.FilterAsyncChannel.onPacket(FilterAsyncChannel.java:89)
>> 	at
>> org.apache.activemq.transport.activeio.ActivityMonitor.onPacket(ActivityMonitor.java:42)
>> 	at
>> org.apache.activeio.adapter.SyncToAsyncChannel.run(SyncToAsyncChannel.java:127)
>> 	at
>> edu.emory.mathcs.backport.java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:650)
>> 	at
>> edu.emory.mathcs.backport.java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:675)
>> 	at java.lang.Thread.run(Thread.java:595)
>>
>> Cheers,
>>
>> steve
>>
>>
>>
>>> BTW is there anything interesting appearing on the broker log whne
>>> your client dies?
>>>
>>> James
>>>
>>> On 3/28/06, Steve Barham <steve.barham@formicary.net> wrote:
>>>> Can you give an example of how to enable this debug logging?
>>>>
>>>> I've just tried with 4.0-SNAPSHOT rather than -M4, and get this
>>>> exception:
>>>>  WARN [2006-03-28 15:44:24,515] activemq.ActiveMQConnection "Async
>>>> exception with no exception listener: java.io.EOFException: Peer
>>>> disconnected."
>>>> java.io.EOFException: Peer disconnected.
>>>>         at
>>>> org.apache.activeio.command.AsyncChannelToAsyncCommandChannel$1.onPacket(AsyncChannelToAsyncCommandChannel.java:64)
>>>>         at
>>>> org.apache.activemq.transport.activeio.PacketAggregatingAsyncChannel.onPacket(PacketAggregatingAsyncChannel.java:59)
>>>>         at
>>>> org.apache.activeio.packet.async.FilterAsyncChannel.onPacket(FilterAsyncChannel.java:89)
>>>>         at
>>>> org.apache.activemq.transport.activeio.ActivityMonitor.onPacket(ActivityMonitor.java:42)
>>>>         at
>>>> org.apache.activeio.adapter.SyncToAsyncChannel.run(SyncToAsyncChannel.java:127)
>>>>         at
>>>> edu.emory.mathcs.backport.java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:650)
>>>>         at
>>>> edu.emory.mathcs.backport.java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:675)
>>>>         at java.lang.Thread.run(Thread.java:595)
>>>>
>>>> So very similar behaviour...
>>>>
>>>> steve
>>>> >
>>>> > I am not sure if I can help, but I think I had a similar error in my
>>>> > experiments with SSL.
>>>> > It looks like SSL doesn't work so fast or something, try to enable
>>>> the
>>>> SSL
>>>> > debug and see if you can get there a hint. My problem was at the
>>>> startup,
>>>> > that the logging took to long and I always got timeout errors.
>>>> >
>>>> >
>>>> > -----Ursprüngliche Nachricht-----
>>>> > Von: Steve Barham [mailto:steve.barham@formicary.net]
>>>> > Gesendet: Dienstag, 28. März 2006 16:16
>>>> > An: activemq-users@geronimo.apache.org
>>>> > Betreff: Final repost: SSL transport failure
>>>> >
>>>> >
>>>> >
>>>> > This is likely to be my last post unless someone has any bright
>>>> ideas
>>>> - I
>>>> > need SSL support for my application to be deployed, and I cannot
>>>> work
>>>> > around the following error.
>>>> >
>>>> > When I send a large (List of 8000 objects, each of about 10 Strings,
>>>> > around 10 characters each - serialized size is only 627k) message, I
>>>> get
>>>> > the following stacktrace:
>>>> >
>>>> > WARN [2006-03-28 15:13:46,140] activemq.ActiveMQConnection "Async
>>>> > exception with no exception listener: java.io.EOFException: Peer
>>>> > disconnected."
>>>> > java.io.EOFException: Peer disconnected.
>>>> >       at
>>>> > org.activeio.command.AsyncChannelToAsyncCommandChannel$1.onPacket(AsyncChannelToAsyncCommandChannel.java:65)
>>>> >       at
>>>> > org.apache.activemq.transport.activeio.PacketAggregatingAsyncChannel.onPacket(PacketAggregatingAsyncChannel.java:59)
>>>> >       at
>>>> org.activeio.FilterAsyncChannel.onPacket(FilterAsyncChannel.java:87)
>>>> >       at
>>>> > org.apache.activemq.transport.activeio.ActivityMonitor.onPacket(ActivityMonitor.java:42)
>>>> >       at
>>>> > org.activeio.adapter.SyncToAsyncChannel.run(SyncToAsyncChannel.java:150)
>>>> >       at
>>>> > edu.emory.mathcs.backport.java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:643)
>>>> >       at
>>>> > edu.emory.mathcs.backport.java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:668)
>>>> >       at java.lang.Thread.run(Thread.java:595)
>>>> >  WARN [2006-03-28 15:13:46,156] activemq.ActiveMQConnection "Cleanup
>>>> > failed"
>>>> > javax.jms.JMSException: Connection closed by remote host
>>>> >       at
>>>> > org.apache.activemq.util.JMSExceptionSupport.create(JMSExceptionSupport.java:57)
>>>> >       at
>>>> > org.apache.activemq.ActiveMQConnection.asyncSendPacket(ActiveMQConnection..java:1043)
>>>> >       at
>>>> > org.apache.activemq.ActiveMQConnection.cleanup(ActiveMQConnection.java:1191)
>>>> >       at
>>>> > org.apache.activemq.ActiveMQConnection.transportFailed(ActiveMQConnection..java:1585)
>>>> >       at
>>>> > org.apache.activemq.ActiveMQConnection.onException(ActiveMQConnection.java:1338)
>>>> >       at
>>>> > org.apache.activemq.transport.TransportFilter.onException(TransportFilter..java:102)
>>>> >       at
>>>> > org.apache.activemq.transport.TransportFilter.onException(TransportFilter..java:102)
>>>> >       at
>>>> > org.apache.activemq.transport.TransportFilter.onException(TransportFilter..java:102)
>>>> >       at
>>>> > org.apache.activemq.transport.InactivityMonitor.onException(InactivityMonitor.java:96)
>>>> >       at
>>>> > org.apache.activemq.transport.activeio.ActiveIOTransport$1.onError(ActiveIOTransport.java:71)
>>>> >       at
>>>> > org.activeio.command.AsyncChannelToAsyncCommandChannel$1.onPacket(AsyncChannelToAsyncCommandChannel.java:65)
>>>> >       at
>>>> > org.apache.activemq.transport.activeio.PacketAggregatingAsyncChannel.onPacket(PacketAggregatingAsyncChannel.java:59)
>>>> >       at
>>>> org.activeio.FilterAsyncChannel.onPacket(FilterAsyncChannel.java:87)
>>>> >       at
>>>> > org.apache.activemq.transport.activeio.ActivityMonitor.onPacket(ActivityMonitor.java:42)
>>>> >       at
>>>> > org.activeio.adapter.SyncToAsyncChannel.run(SyncToAsyncChannel.java:150)
>>>> >       at
>>>> > edu.emory.mathcs.backport.java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:643)
>>>> >       at
>>>> > edu.emory.mathcs.backport.java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:668)
>>>> >       at java.lang.Thread.run(Thread.java:595)
>>>> > Caused by: java.net.SocketException: Connection closed by remote
>>>> host
>>>> >       at
>>>> > com.sun.net.ssl.internal.ssl.SSLSocketImpl.checkWrite(SSLSocketImpl.java:1168)
>>>> >       at
>>>> > com.sun.net.ssl.internal.ssl.AppOutputStream.write(AppOutputStream.java:43)
>>>> >       at
>>>> > org.activeio.net.SocketStreamChannel.write(SocketStreamChannel.java:183)
>>>> >       at
>>>> > org.activeio.adapter.OutputStreamChannelToOutputStream.write(OutputStreamChannelToOutputStream.java:48)
>>>> >       at
>>>> org.activeio.packet.ByteArrayPacket.writeTo(ByteArrayPacket.java:126)
>>>> >       at
>>>> org.activeio.net.SocketSyncChannel.write(SocketSyncChannel.java:113)
>>>> >       at
>>>> > org.activeio.adapter.SyncToAsyncChannel.write(SyncToAsyncChannel.java:184)
>>>> >       at
>>>> org.activeio.FilterAsyncChannel.write(FilterAsyncChannel.java:50)
>>>> >       at
>>>> > org.apache.activemq.transport.activeio.ActivityMonitor.write(ActivityMonitor.java:47)
>>>> >       at
>>>> org.activeio.FilterAsyncChannel.write(FilterAsyncChannel.java:50)
>>>> >       at
>>>> > org.activeio.command.AsyncChannelToAsyncCommandChannel.writeCommand(AsyncChannelToAsyncCommandChannel.java:41)
>>>> >       at
>>>> > org.apache.activemq.transport.activeio.ActiveIOTransport.oneway(ActiveIOTransport.java:95)
>>>> >       at
>>>> > org.apache.activemq.transport.TransportFilter.oneway(TransportFilter.java:90)
>>>> >       at
>>>> > org.apache.activemq.transport.TransportFilter.oneway(TransportFilter.java:90)
>>>> >       at
>>>> > org.apache.activemq.transport.WireFormatNegotiator.oneway(WireFormatNegotiator.java:65)
>>>> >       at
>>>> > org.apache.activemq.transport.MutexTransport.oneway(MutexTransport.java:44)
>>>> >       at
>>>> > org.apache.activemq.transport.ResponseCorrelator.oneway(ResponseCorrelatorjava:54)
>>>> >       at
>>>> > org.apache.activemq.ActiveMQConnection.asyncSendPacket(ActiveMQConnection..java:1041)
>>>> >       ... 16 more
>>>> >
>>>> >
>>>> > I will have to move to SwiftMQ if noone has any insight or
>>>> suggestions
>>>> for
>>>> > this problem, which isn't really something I want to do!
>>>> >
>>>> > Cheers,
>>>> >
>>>> > Steve
>>>> >
>>>> >
>>>> > --
>>>> > Steve Barham                       tel: +44 (0)7973 199 471
>>>> > Systems Engineer
>>>> > Formicary - delivering quality financial technology solutions
>>>> > http://www.formicary.net/
>>>> >
>>>> >
>>>> >
>>>> > This mail has originated outside your organization,
>>>> > either from an external partner or the Global Internet.
>>>> > Keep this in mind if you answer this message.
>>>> >
>>>> > This mail has originated outside your organization, either from an
>>>> > external partner or the Global Internet. Keep this in mind if you
>>>> answer
>>>> > this message.
>>>> >
>>>>
>>>>
>>>> --
>>>> Steve Barham                       tel: +44 (0)7973 199 471
>>>> Systems Engineer
>>>> Formicary - delivering quality financial technology solutions
>>>> http://www.formicary.net/
>>>>
>>>>
>>>
>>>
>>> --
>>>
>>> James
>>> -------
>>> http://radio.weblogs.com/0112098/
>>>
>>
>>
>> --
>> Steve Barham                       tel: +44 (0)7973 199 471
>> Systems Engineer
>> Formicary - delivering quality financial technology solutions
>> http://www.formicary.net/
>>
>>
>>
>> This mail has originated outside your organization,
>> either from an external partner or the Global Internet.
>> Keep this in mind if you answer this message.
>>
>> This mail has originated outside your organization, either from an
>> external partner or the Global Internet. Keep this in mind if you answer
>> this message.
>>
>
>
> --
> Steve Barham                       tel: +44 (0)7973 199 471
> Systems Engineer
> Formicary - delivering quality financial technology solutions
> http://www.formicary.net/
>
>
>
> This mail has originated outside your organization,
> either from an external partner or the Global Internet.
> Keep this in mind if you answer this message.
>
> This mail has originated outside your organization, either from an
> external partner or the Global Internet. Keep this in mind if you answer
> this message.
>


--
Steve Barham                       tel: +44 (0)7973 199 471
Systems Engineer
Formicary - delivering quality financial technology solutions
http://www.formicary.net/



This mail has originated outside your organization,
either from an external partner or the Global Internet.
Keep this in mind if you answer this message.


This e-mail is intended only for the above addressee. It may contain
privileged information. If you are not the addressee you must not copy,
distribute, disclose or use any of the information in it. If you have
received it in error please delete it and immediately notify the sender.
Security Notice: all e-mail, sent to or from this address, may be
accessed by someone other than the recipient, for system management and
security reasons. This access is controlled under Regulation of
Investigatory Powers Act 2000, Lawful Business Practises.

Mime
View raw message