camel-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Claus Ibsen <claus.ib...@gmail.com>
Subject Threadsafety of Camel FTP
Date Mon, 25 May 2009 06:19:30 GMT
Hi

Sorry for the late reply. The lightning struck my town and I have been
without internet for the last 3 days.


On Tue, May 19, 2009 at 11:12 PM,  <oliver.hecker@capgemini-sdm.com> wrote:
> Hi Claus,
>
> there is one general question around this issue which I would like to address to you
directly:
>
> When starting to test multithreading of Camel FTP it I was hoping for some result in
the sense
> "Camel is able to handle multithreading for FTP connections, so hopefully the simpler/mainstream
things like HTTP, XSLT, JDBC, JMS.... will work in a multithreaded environment as well".
> So at the moment it seems that FTP has some limitation/issues with multithreading. I
am now wondering what to expect for the other components? Is there a general issue with multithreading
(at least as I tried to do it) or did I intuitively pick the only component which makes trouble?

Yeah you hit the component with the problem, there might be another
one. For instance LDAP was also not thread safe before hand. But this
component wasnt used very much and it was not thread safe because that
the backing LDAP implementation was not thread safe = it was not
Camels fault as such.

JMS is of course multi threaded and is used in high concurrency
environments. JDBC as well as camel is just a simple wrapper on top of
the DataSource you get from the JDBC connection pool.

We have fixed in the past a couple of multi threading issues with JAXB
and some of the other XML components.

For HTTP camel leverages Apache HTTP Client. And each http producer
creates a new http client so they have their own instance. And thus
should be thread safe.

>
> Background of my question is that we are just starting implementing an integration based
on camel. The worst scenario would be to find out in some month that there are a lot of unresolved
issues in the framework concerning nonfunctional things which affect stability and performance.
>
> It would be great if you could comment on that.
Yeah I absolutely believe you should always do your own investigations
and due dilligence testing of the candidates.
Do not trust the marketing material and sales :) And my word :) Ask
the computer, it will tell the answer!

We will work on fixing the FTP component to be thread safe. Thanks for
reporting.



And for the idea of a pool of ftp clients, that will be 2nd priority.
I am thinking to allow some API to configure this and then let 3rd
party framework provide the pooling capabilities, eg Spring have
support for that. Or use one of the Apache Pools. With the pool the
FTP connection could already be established and thus no login is
required.

But what are your requirements to this? Do you want long lasting FTP
connections to be open as a pooling solution generally is based on?
And do you use the same login for all your FTP connections?

Any thoughts?



>
> Thank you
>
> Oliver
>
> Claus Ibsen-2 wrote:
>>
>> On Tue, May 19, 2009 at 5:14 PM, Ryan Gardner <ryebrye@gmail.com> wrote:
>>> Wouldn't the prototype mean it would create a new instance each time it
>>> is
>>> retrieved?
>> Yes you are correct Ryan it would mean that it creates a new instance
>> of the endpoint at each time you use the producer.
>>
>>>
>>> I wonder if something like concurrent consumers that is on the seda
>>> pattern
>>> might be a good idea for routes frequently used concurrently - I.e. keep
>>> around up to N instances in a pool or something.
>> Ryan, an interresting thought of a pool of singleton producers, like
>> the seda. Kinda like a new URI option ?concurrentProducers=5 the
>> underlying SendToProcessor can leverage.
>>
>>>
>>> I don't know what the construction costs of the ftp endpoit are, it might
>>> not even be an issue.
>> Well this happens in real life. I know of some integration using FTP
>> that login, upload file, logout, login, upload file, logout and does
>> that constantly.
>> No wonder the throughtput is slow. But the people doing the
>> integration dont know how to script it differently.
>>
>>
>> For starters I think I will add the unit test from Oliver and then we
>> can give it a sleep and more thought how to leverage a good solution.
>> Ryan opened the door for a new possibilty that we gotta check out as well.
>>
>>
>>
>>>
>>> On May 19, 2009 9:07 AM, "Claus Ibsen" <claus.ibsen@gmail.com> wrote:
>>>
>>> Hi Oliver
>>>
>>> Thanks for reporting back so promptly and providing how you test it.
>>>
>>> Basically some endpoints in Camel are singletons and others are not
>>> (eg prototype in Spring lingo).
>>> An endpoint implements IsSingleton and provide whether its singleton or
>>> not.
>>>
>>> The problem is that we dont have a setter for controlling this so you
>>> can decide. So if you use only one FTP producer at the time you can
>>> set it to singleton. Or as you want to use concurrent ftp producers
>>> you can set it to prototype. As you would do if you configured it in
>>> Spring XML.
>>>
>>> So I am wondering if we should expose such a configuration, so you can
>>> override the default behvior for FTP which is singleton.
>>>
>>> So basically its just appending: &isSingleton=false on the URI.
>>> We could maybe rename it to something more IoC ish and use an enum to
>>> define the types
>>>
>>> So its either
>>> ?scope=singleton
>>> ?scope=prototype
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>> On Tue, May 19, 2009 at 4:02 PM, ohecker
>>>
>>> <oliver.hecker@capgemini-sdm.com> wrote: >
>>>
>>>> Hi Claus, > > we are using Camel 2.0-M1 at the moment. > The scenario
>>> consists of a simple route w...
>>>
>>
>>
>>
>> --
>> Claus Ibsen
>> Apache Camel Committer
>>
>> Open Source Integration: http://fusesource.com
>> Blog: http://davsclaus.blogspot.com/
>> Twitter: http://twitter.com/davsclaus
>>
>>
> Quoted from:
> http://www.nabble.com/Threadsafety-of-Camel-FTP-tp23615932p23618657.html
>
>



--
Claus Ibsen
Apache Camel Committer

Open Source Integration: http://fusesource.com
Blog: http://davsclaus.blogspot.com/
Twitter: http://twitter.com/davsclaus



-- 
Claus Ibsen
Apache Camel Committer

Open Source Integration: http://fusesource.com
Blog: http://davsclaus.blogspot.com/
Twitter: http://twitter.com/davsclaus

Mime
View raw message