camel-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Romain Manni-Bucau <rmannibu...@gmail.com>
Subject Re: camel for "short" programs
Date Mon, 16 Dec 2013 22:24:14 GMT
hehe, so you allocate mem only for camel?

joke apart batches can needs gigs so to avoid to buy too much machines
(to be honest I dont control this part at all) batches are scheduled
and each batch is a jvm (almost) since it keeps dev and deployment
simple but all jvms can't run at the same time (a single batch can
need 4G to have a good speed because it works on a bunch of data)

JBatch and camel are not opposed at all and IMO shoudn't be used for
the same needs. My goal was only to provide a bridge in both ways to
either start a batch from camel (in this mode camel is always running
;) or to reuse a camel component in a batch to avoid to reinvent the
wheel (useful in some cases).

Hope it is clearer

Romain Manni-Bucau
Twitter: @rmannibucau
Blog: http://rmannibucau.wordpress.com/
LinkedIn: http://fr.linkedin.com/in/rmannibucau
Github: https://github.com/rmannibucau



2013/12/16 Walzer, Thomas <thomas.walzer@integratix.net>:
> I got Camel (actually servicemix) running on a 512 mega(!)bytes Raspberry Pi and you
say you cannot spare 512 MB for a JVM? Maybe you should go with Cobol & JCL. That ran
on 8 megs of RAM as far as I can remember.
> I got servicemix running on a 64 gig memory 16 cpu 2 node cluster as well. But you can
also have much much more if you need (fabric, whatever).
> Are you talking enterprise or toy shop? You can have both with Camel. It scales beautifully
from 0 (when you stop it) to whatever you can afford.
> I know one particular serious enterprise running Camel. They transfer 10,000 orders per
day worth >1 million USD (4 cpus 4 gig). So wtf are you talking about Camel not being enterprise
ready?
>
> I did not read the jbatch spec. But from 30,000 feet I can see nothing that can´t be
done with Camel/EIPs (which Robert+Claus already pointed out).
> Maybe jbatch is great. Maybe it goes the JBI route.
>
> Good luck.
>
> Am 16.12.2013 um 18:53 schrieb Romain Manni-Bucau <rmannibucau@gmail.com>:
>
>> I dont agree, it is a common need (at least the 2 cies i did) and if the
>> answzr is keep process running it is really disappointing...ig you have too
>> much mem and cpu please share ;)...and being open/extensible (api) doesnt
>> mean being enterprise ready.
>>
>> Another thing is it makes camel not really well usable in jbatch readers
>> (not a big deal)...but same applies in spring-batch (released and worse in
>> fact since timeout is not handled). So camel should drop this code instead
>> of keeping it IMHO.
>> Le 16 déc. 2013 18:40, "kraythe ." <kraythe@gmail.com> a écrit :
>>
>>> There is nothing that you are trying to do that camel wont do. You simply
>>> don't seem want to do it. So why waste our time? You can start it up and
>>> shut it down if thats your flavor. You dont really need to do such a thing,
>>> but if you want, go for it. You can leave it running as most of us do and
>>> use quartz triggers, events on files showing up, AMQ events and so on. The
>>> only limit is your imagination. If you care looking for camel-ruby that is
>>> weird (and Ruby is ENTIRELY unmaintainable anyway). If you want to do it in
>>> something else then we all wish you good fortune but there is NOTHING
>>> preventing you from reaching your goals with camel.
>>>
>>> Always try to think "Is my problem unique to my company or domain?" if not
>>> then the odds are its solved already, you jsut have to learn the provided
>>> APIs.
>>>
>>> *Robert Simmons Jr. MSc. - Lead Java Architect @ EA*
>>> *Author of: Hardcore Java (2003) and Maintainable Java (2012)*
>>> *LinkedIn: **http://www.linkedin.com/pub/robert-simmons/40/852/a39
>>> <http://www.linkedin.com/pub/robert-simmons/40/852/a39>*
>>>
>>>
>>> On Mon, Dec 16, 2013 at 8:34 AM, Walzer, Thomas <
>>> thomas.walzer@integratix.net> wrote:
>>>
>>>> Why not keep the context+routes running?
>>>> If nothing´s there nothing gets processed.
>>>> We transfer a gazillion files and poll like crazy (5 sec) even when
>>>> nothing is expected. Who cares on a corporate network?
>>>>
>>>> Am 15.12.2013 um 20:11 schrieb Romain Manni-Bucau <rmannibucau@gmail.com
>>>> :
>>>>
>>>>> Not really true since it prevents from using camel consumers which
>>>>> doesn't support it so it makes camel not as useful as it could.
>>>>> Romain Manni-Bucau
>>>>> Twitter: @rmannibucau
>>>>> Blog: http://rmannibucau.wordpress.com/
>>>>> LinkedIn: http://fr.linkedin.com/in/rmannibucau
>>>>> Github: https://github.com/rmannibucau
>>>>>
>>>>>
>>>>>
>>>>> 2013/12/15 Claus Ibsen <claus.ibsen@gmail.com>:
>>>>>> You can use route policy / event notifier or what not to know when
>>>>>> there is nothing more to process and then signal to the main thread
to
>>>>>> stop camel and terminate the jvm.
>>>>>>
>>>>>> No hacks is needed just use the API there is already there.
>>>>>>
>>>>>> On Sun, Dec 15, 2013 at 6:57 PM, Romain Manni-Bucau
>>>>>> <rmannibucau@gmail.com> wrote:
>>>>>>> If you look camel architecture it is great but not usable for
batches
>>>>>>> *out of the box*. What I find "sad" is the code needed to support
>>> this
>>>>>>> (common) use case shouldn't be that important:
>>>>>>>
>>>>>>> CamelContext ctx = new ...();
>>>>>>> ctx.setSingleShort(true); // or singleExecution(); maybe
>>>>>>>
>>>>>>> This would set the same boolean on the consumers which would
not wait
>>>>>>> to get data if nothing is available anymore and would stop the
route.
>>>>>>> Once all routes are stopped the context would be stopped too.
>>>>>>>
>>>>>>> This way it would be easy to write cron-ned mains with camel
without
>>>> app hacks.
>>>>>>>
>>>>>>> Romain Manni-Bucau
>>>>>>> Twitter: @rmannibucau
>>>>>>> Blog: http://rmannibucau.wordpress.com/
>>>>>>> LinkedIn: http://fr.linkedin.com/in/rmannibucau
>>>>>>> Github: https://github.com/rmannibucau
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> 2013/12/15 John D. Ament <john.d.ament@gmail.com>:
>>>>>>>> Romain,
>>>>>>>>
>>>>>>>> What do you mean?
>>>>>>>>
>>>>>>>> On Sat, Dec 14, 2013 at 4:00 PM, Romain Manni-Bucau
>>>>>>>> <rmannibucau@gmail.com> wrote:
>>>>>>>>> Hmm, so if I understand you camel will not solve it.
I find it sad
>>>> cause
>>>>>>>>> camel pipeline and the numerous components are 2 tempting
things
>>> for
>>>>>>>>> batches but the fact to be able to process what is here
when
>>>> starting and
>>>>>>>>> dont wait another messge is no more is present is mandatory
to be
>>>> usable.
>>>>>>>>>
>>>>>>>>> I know it is hackable but I dont think it is clean if
not in camel
>>>> itself.
>>>>>>>>> Context should get an option propagated to consumer for
it imo.
>>>>>>>>> Le 14 déc. 2013 16:32, "kraythe ." <kraythe@gmail.com>
a écrit :
>>>>>>>>>
>>>>>>>>>> Indeed. Though you could use it to start up and shut
down, nothing
>>>> stopping
>>>>>>>>>> you. I would not opt for that choice if I had some
sort of
>>>> deployment
>>>>>>>>>> system where I could keep the routes running.
>>>>>>>>>>
>>>>>>>>>> *Robert Simmons Jr. MSc. - Lead Java Architect @
EA*
>>>>>>>>>> *Author of: Hardcore Java (2003) and Maintainable
Java (2012)*
>>>>>>>>>> *LinkedIn: **
>>> http://www.linkedin.com/pub/robert-simmons/40/852/a39
>>>>>>>>>> <http://www.linkedin.com/pub/robert-simmons/40/852/a39>*
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Sat, Dec 14, 2013 at 9:09 AM, John D. Ament <
>>>> john.d.ament@gmail.com
>>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> Why not use a polling consumer?
>>>>>>>>>>>
>>>>>>>>>>> On Sat, Dec 14, 2013 at 6:25 AM, Romain Manni-Bucau
>>>>>>>>>>> <rmannibucau@gmail.com> wrote:
>>>>>>>>>>>> Hi
>>>>>>>>>>>>
>>>>>>>>>>>> any opinion on how to make consumers consume
all what is
>>> possible
>>>> when
>>>>>>>>>>>> program is running then shutdown the route
once processed?
>>>>>>>>>>>>
>>>>>>>>>>>> It is basically needed for BatchEE camel
extension (
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>
>>> https://git-wip-us.apache.org/repos/asf?p=incubator-batchee.git;a=blob;f=extensions/camel/src/main/java/org/apache/batchee/camel/CamelItemReader.java;h=bf4d289a8fea4a18f783353c3cb25d1aa9050018;hb=HEAD
>>>>>>>>>>>> ) + I wondered it for some batches I wrote
some months ago
>>> without
>>>>>>>>>>>> camel because the infra needed for it was
too heavy (route
>>> policy
>>>> +
>>>>>>>>>>>> few other things) compared to the gain.
>>>>>>>>>>>>
>>>>>>>>>>>> ATM batchee relies on timeout but surely
not the best way to do.
>>>>>>>>>>>>
>>>>>>>>>>>> Romain Manni-Bucau
>>>>>>>>>>>> Twitter: @rmannibucau
>>>>>>>>>>>> Blog: http://rmannibucau.wordpress.com/
>>>>>>>>>>>> LinkedIn: http://fr.linkedin.com/in/rmannibucau
>>>>>>>>>>>> Github: https://github.com/rmannibucau
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Claus Ibsen
>>>>>> -----------------
>>>>>> Red Hat, Inc.
>>>>>> Email: cibsen@redhat.com
>>>>>> Twitter: davsclaus
>>>>>> Blog: http://davsclaus.com
>>>>>> Author of Camel in Action: http://www.manning.com/ibsen
>>>>>> Make your Camel applications look hawt, try: http://hawt.io
>>>>
>>>>
>>>
>

Mime
View raw message