camel-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Claus Ibsen <claus.ib...@gmail.com>
Subject Re: Skipping empty files, problem with simple language
Date Fri, 03 Sep 2010 04:46:02 GMT
On Fri, Sep 3, 2010 at 6:45 AM, Claus Ibsen <claus.ibsen@gmail.com> wrote:
> On Thu, Sep 2, 2010 at 7:43 PM, Bengt Rodehav <bengt@rodehav.com> wrote:
>> Even better - looking forward to it. I must say that Camel is impressively
>> flexible.
>>
>> BTW we recently put the first version of our Camel/Karaf based integration
>> platform into production and it works very well indeed. I'm very pleased!
>>
>
> That is great news. Glad that you made it into production. Getting the
> first app into production is usually the biggest obstacle.
> If it performs well, then its usually easier to get the next apps in
> production as well.
>

BTW: Feel free to add a user story if you want to help spread the word
where Camel is used in production
http://camel.apache.org/user-stories.html



>
>> /Bengt
>>
>> 2010/9/2 Claus Ibsen <claus.ibsen@gmail.com>
>>
>>> On Thu, Sep 2, 2010 at 4:57 PM, Bengt Rodehav <bengt@rodehav.com> wrote:
>>> > Just to sum up. Recipient list worked perfectly. I now use the following
>>> > interceptor:
>>> >
>>> >        interceptFrom().when(
>>> >            PredicateBuilder.toPredicate(SimpleLanguage
>>> >                .simple("${file:length} == null"))).setHeader("recipient")
>>> >            .simple("file://${file:parent}/skipped/${date:now:yyyyMMdd}")
>>> >            .recipientList(header("recipient")).stop();
>>> >
>>> > All empty files will then be moved to the "skipped/<yyyyMMdd>" folder
>>> under
>>> > the folder where the polled file resided. Beautiful solution.
>>> >
>>>
>>> Will be more beautiful in Camel 2.5, with simple directly in the
>>> RouteBuilder.
>>>
>>> And you can move the ugly Predicate before the intercept
>>>
>>> Predicate emptyFile =
>>> PredicateBuilder.toPredicate(SimpleLanguage.simple("${file:length} ==
>>> null"));
>>>
>>>
>>> interceptFrom().when(emptyFile).recipientList(simple("file://${file:parent}/skipped/${date:now:yyyyMMdd}")).stop();
>>>
>>>
>>> And you can set the expression directly in the recipientList, you dont
>>> have to use a header.
>>>
>>>
>>> > Thanks Claus and Willem for your help,
>>> >
>>> > /Bengt
>>> >
>>> >
>>> > 2010/9/2 Bengt Rodehav <bengt@rodehav.com>
>>> >
>>> >> Thanks again Claus - will have a look at recipient list.
>>> >>
>>> >> BTW seems like camel.apache.org is not accessible for the moment -
>>> >> problems?
>>> >>
>>> >> /Bengt
>>> >>
>>> >> 2010/9/2 Claus Ibsen <claus.ibsen@gmail.com>
>>> >>
>>> >>> Use dynamic recipient list to construct the endpoint uri on-the-fly
>>> >>> http://camel.apache.org/recipient-list.html
>>> >>>
>>> >>>
>>> >>> On Thu, Sep 2, 2010 at 1:07 PM, Bengt Rodehav <bengt@rodehav.com>
>>> wrote:
>>> >>> > Thanks Claus,
>>> >>> >
>>> >>> > stop() worked perfectly - I had confused it with end() but
stand
>>> >>> corrected.
>>> >>> >
>>> >>> > However, the Properties component does not seem to address
my
>>> specific
>>> >>> > problem. I need to get hold of properties from the exchange
itself -
>>> not
>>> >>> > from an external properties file.
>>> >>> >
>>> >>> > If the file is polled from the folder "C:\in-box" then I want
the
>>> empty
>>> >>> > files to end up in the folder "C:\in-box\skipped\<date>"
where <date>
>>> is
>>> >>> the
>>> >>> > date the exchange was processed. I cannot  use property files
for the
>>> >>> base
>>> >>> > directory either since this is a general component that can
be
>>> >>> configured to
>>> >>> > read files from many different places. No matter what folder
the
>>> input
>>> >>> file
>>> >>> > is polled from, I want skipped files in the "skipped" subfolder
of
>>> that
>>> >>> > original folder.
>>> >>> >
>>> >>> > It's very similar to the  functionality that I use when archiving
>>> >>> successful
>>> >>> > and failed exchanges. I use the following uri options for that:
>>> >>> >
>>> >>> > move=archive/${date:now:yyyyMMdd}/${file:name}
>>> >>> >
>>> >>>
>>> moveFailed=failed/${file:name.noext}-${date:now:yyyyMMddHHmmssSSS}.${file:ext}
>>> >>> >
>>> >>> > I now want to use the simple (and file) language to specify
where
>>> >>> skipped
>>> >>> > files are archived. I haven't investigated how the move/moveFailed
>>> >>> options
>>> >>> > are actually implemented but I guess I need to do it in a similar
way
>>> >>> unless
>>> >>> > there is an easy way to use simple/file language directly in
an
>>> endpoint
>>> >>> > uri.
>>> >>> >
>>> >>> > /Bengt
>>> >>> >
>>> >>> > 2010/9/2 Claus Ibsen <claus.ibsen@gmail.com>
>>> >>> >
>>> >>> >> Ad 1)
>>> >>> >> See the properties component
>>> >>> >> http://camel.apache.org/properties
>>> >>> >>
>>> >>> >> Ad 2)
>>> >>> >> Use stop()
>>> >>> >>
>>> >>> >> On Thu, Sep 2, 2010 at 11:17 AM, Bengt Rodehav <bengt@rodehav.com>
>>> >>> wrote:
>>> >>> >> > Thanks Willem it worked perfectly.
>>> >>> >> >
>>> >>> >> > However, I now have two other problems (if you bear
with me...):
>>> >>> >> >
>>> >>> >> > I want to use this interceptor:
>>> >>> >> >
>>> >>> >> >
>>> >>> >>
>>> >>>
>>> *interceptFrom().when(PredicateBuilder.toPredicate(SimpleLanguage.simple("${file:length}
>>> >>> >> > ==
>>> >>> >>
>>> null"))).to("file://${file:path}/${date:now:yyyyMMdd}/skipped").end();*
>>> >>> >> >
>>> >>> >> > 1. I get the following exception:
>>> >>> >> >
>>> >>> >> > *java.lang.IllegalArgumentException: Invalid directory:
>>> >>> >> > skipped/${date:now:yyyyMMdd}. Dynamic expressions
with ${ }
>>> >>> placeholders
>>> >>> >> is
>>> >>> >> > not allowed. Use the fileName option to set the dynamic
>>> expression.*
>>> >>> >> >
>>> >>> >> > How can I direct the file to a directory that I need
properties
>>> (file
>>> >>> and
>>> >>> >> > date) to calculate?
>>> >>> >> >
>>> >>> >> > 2. The interceptor does not "skip" the file. It sends
it both to
>>> the
>>> >>> >> > original endpoint and to my "skipped" folder. How
can I make it
>>> NOT
>>> >>> send
>>> >>> >> the
>>> >>> >> > file to the original endpoint. I've seen the
>>> >>> >> >
>>> >>> >> >  *interceptSendToEndpoint("...").skipSendToOriginalEndpoint();*
>>> >>> >> >
>>> >>> >> > but in this case I want to use the interceptFrom()
since I want to
>>> >>> >> intercept
>>> >>> >> > the route at the earliest possible stage.
>>> >>> >> >
>>> >>> >> > /Bengt
>>> >>> >> >
>>> >>> >> >
>>> >>> >> >
>>> >>> >> > 2010/9/2 Claus Ibsen <claus.ibsen@gmail.com>
>>> >>> >> >
>>> >>> >> >> On Thu, Sep 2, 2010 at 9:20 AM, Bengt Rodehav
<bengt@rodehav.com
>>> >
>>> >>> >> wrote:
>>> >>> >> >> > Hi Willem,
>>> >>> >> >> >
>>> >>> >> >> > Thanks for the info. I assume then that as
long as I'm on Camel
>>> >>> 2.4
>>> >>> >> I'll
>>> >>> >> >> > check for null but when I upgrade to Camel
2.5 I'll start
>>> checking
>>> >>> for
>>> >>> >> >> zero
>>> >>> >> >> > instead.
>>> >>> >> >> >
>>> >>> >> >> > However, that issue aside, I still get the
same problems that I
>>> >>> wrote
>>> >>> >> >> about.
>>> >>> >> >> > Can you confirm that the following syntax
is correct?
>>> >>> >> >> >
>>> >>> >> >> > interceptFrom().when(simple("${file:length}
==
>>> >>> 0")).to(skipped).end();
>>> >>> >> >> >
>>> >>> >> >> > OR (as long as I stay on Camel 2.4)
>>> >>> >> >> >
>>> >>> >> >> > interceptFrom().when(simple("${file:length}
==
>>> >>> >> null")).to(skipped).end();
>>> >>> >> >> >
>>> >>> >> >> > Both of the above give me a compilation error
since the
>>> "when()"
>>> >>> >> method
>>> >>> >> >> > expects a "Predicate" while "simple()" returns
a String. If I
>>> >>> instead
>>> >>> >> use
>>> >>> >> >> > "simple(..)..isEqualTo(0)" then the compiler
accepts it (since
>>> >>> >> >> "isEqualTo()"
>>> >>> >> >> > returns a Predicate), but then I get the
runtime exception
>>> >>> instead.
>>> >>> >> >> >
>>> >>> >> >>
>>> >>> >> >> That is also part of Camel 2.5 that simple is
easier to use in
>>> the
>>> >>> >> >> RouteBuilder out of the box.
>>> >>> >> >>
>>> >>> >> >> Just use a
>>> PredicateBuilder.toPredicate(SimpleLanguage.simple("foo")
>>> >>> >> >> to construct the predicate.
>>> >>> >> >>
>>> >>> >> >>
>>> >>> >> >> > I'v double checked my dependencies and it
turns out that I'm
>>> using
>>> >>> a
>>> >>> >> >> > snapshot version of Camel 2.4 based on revision
958950. Maybe
>>> my
>>> >>> >> problems
>>> >>> >> >> > stem from that... I will check of course.
>>> >>> >> >> >
>>> >>> >> >> > Do you know if the "when()" method will accept
a String as
>>> >>> parameter
>>> >>> >> in
>>> >>> >> >> > Camel 2.4?
>>> >>> >> >> >
>>> >>> >> >> > /Bengt
>>> >>> >> >> >
>>> >>> >> >> >
>>> >>> >> >> > 2010/9/2 Willem Jiang <willem.jiang@gmail.com>
>>> >>> >> >> >
>>> >>> >> >> >> Hi Bengt,
>>> >>> >> >> >>
>>> >>> >> >> >> I tried to reproduce you issue in Camel
trunk, and found this
>>> >>> bug[1].
>>> >>> >> >> >> BTW, If you just want to filter the zero
length file, you can
>>> >>> take a
>>> >>> >> >> look
>>> >>> >> >> >> this unit test file[2] and change the
simple expression to
>>> >>> >> >> "${file:length}
>>> >>> >> >> >> == null" as in Camel 2.4.0, ${file:length}
is null if the
>>> >>> file.length
>>> >>> >> is
>>> >>> >> >> 0.
>>> >>> >> >> >>
>>> >>> >> >> >>
>>> >>> >> >> >> [1]https://issues.apache.org/activemq/browse/CAMEL-3100
>>> >>> >> >> >> [2]
>>> >>> >> >> >>
>>> >>> >> >>
>>> >>> >>
>>> >>>
>>> https://svn.apache.org/repos/asf/camel/trunk/camel-core/src/test/java/org/apache/camel/component/file/FileConsumerIntercepEmptyFileTest.java
>>> >>> >> >> >>
>>> >>> >> >> >> Willem
>>> >>> >> >> >>
>>> >>> >> >> >>
>>> >>> >> >> >> Bengt Rodehav wrote:
>>> >>> >> >> >>
>>> >>> >> >> >>> I've developed a file transfer service
(for
>>> file/ftp/ftps/sftp)
>>> >>> that
>>> >>> >> >> uses
>>> >>> >> >> >>> Camel 2.4. I need to be able to skip
transferring empty files
>>> >>> and
>>> >>> >> was
>>> >>> >> >> >>> hoping
>>> >>> >> >> >>> to use an interceptor to accomplish
this. Below is the
>>> relevant
>>> >>> >> code:
>>> >>> >> >> >>>
>>> >>> >> >> >>>  String skippedUri =
>>> >>> >> >> "file://skipped/${date:now:yyyyMMdd}/${file:name}";
>>> >>> >> >> >>>
>>> >>> >> >> >>>
>>> >>> >> >>
>>> >>> >>
>>> >>>
>>>  interceptFrom().when(simple("${file:length}").isEqualTo(0)).to(skippedUri).end();
>>> >>> >> >> >>>
>>> >>> >> >> >>> This compiles but in runtime a get
the following exception
>>> when
>>> >>> >> trying
>>> >>> >> >> to
>>> >>> >> >> >>> start the route:
>>> >>> >> >> >>>
>>> >>> >> >> >>> *java.lang.NoSuchMethodError:
>>> >>> >> >> >>>
>>> >>> >> >> >>>
>>> >>> >> >>
>>> >>> >>
>>> >>>
>>> se.digia.connect.services.skandia.filetransfer.FileTransferService$1.simple(Ljava/lang/String;)Lorg/apache/camel/builder/ValueBuilder;
>>> >>> >> >> >>> *
>>> >>> >> >> >>> * **at
>>> >>> >> >> >>>
>>> >>> >> >> >>>
>>> >>> >> >>
>>> >>> >>
>>> >>>
>>> se.digia.connect.services.skandia.filetransfer.FileTransferService$1.configure(FileTransferService.java:279)
>>> >>> >> >> >>> *
>>> >>> >> >> >>> * **at
>>> >>> >> >> >>>
>>> >>> >> >> >>>
>>> >>> >> >>
>>> >>> >>
>>> >>>
>>> org.apache.camel.builder.RouteBuilder.checkInitialized(RouteBuilder.java:295)
>>> >>> >> >> >>> *
>>> >>> >> >> >>> * **at
>>> >>> >> >> >>>
>>> >>> >> >> >>>
>>> >>> >> >>
>>> >>> >>
>>> >>>
>>> org.apache.camel.builder.RouteBuilder.configureRoutes(RouteBuilder.java:250)
>>> >>> >> >> >>> *
>>> >>> >> >> >>> * **at
>>> >>> >> >> >>>
>>> >>> >> >> >>>
>>> >>> >> >>
>>> >>> >>
>>> >>>
>>> org.apache.camel.builder.RouteBuilder.addRoutesToCamelContext(RouteBuilder.java:236)
>>> >>> >> >> >>> *
>>> >>> >> >> >>> * **at
>>> >>> >> >> >>>
>>> >>> >> >> >>>
>>> >>> >> >>
>>> >>> >>
>>> >>>
>>> org.apache.camel.impl.DefaultCamelContext.addRoutes(DefaultCamelContext.java:498)
>>> >>> >> >> >>> *
>>> >>> >> >> >>> * **at
>>> >>> >> >> >>>
>>> >>> >> >> >>>
>>> >>> >> >>
>>> >>> >>
>>> >>>
>>> se.digia.connect.core.service.RouteServiceBase.doStart(RouteServiceBase.java:42)
>>> >>> >> >> >>> *
>>> >>> >> >> >>> * **at
>>> >>> >> >> >>>
>>> >>> >>
>>> se.digia.connect.core.service.ServiceBase.start(ServiceBase.java:49)*
>>> >>> >> >> >>> * **at
>>> >>> >> >> >>>
>>> >>> >> >> >>>
>>> >>> >> >>
>>> >>> >>
>>> >>>
>>> se.digia.connect.services.skandia.filetransfer.FileTransferService.__start(FileTransferService.java:60)
>>> >>> >> >> >>> *
>>> >>> >> >> >>> * **at
>>> >>> >> >> >>>
>>> >>> >> >> >>>
>>> >>> >> >>
>>> >>> >>
>>> >>>
>>> se.digia.connect.services.skandia.filetransfer.FileTransferService.start(FileTransferService.java)
>>> >>> >> >> >>> *
>>> >>> >> >> >>> * **at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
>>> >>> Method)*
>>> >>> >> >> >>> * **at
>>> >>> >> >> >>>
>>> >>> >> >> >>>
>>> >>> >> >>
>>> >>> >>
>>> >>>
>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>>> >>> >> >> >>> *
>>> >>> >> >> >>> * **at
>>> >>> >> >> >>>
>>> >>> >> >> >>>
>>> >>> >> >>
>>> >>> >>
>>> >>>
>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>> >>> >> >> >>> *
>>> >>> >> >> >>> * **at java.lang.reflect.Method.invoke(Method.java:597)*
>>> >>> >> >> >>> * **at
>>> >>> org.apache.felix.ipojo.util.Callback.call(Callback.java:235)*
>>> >>> >> >> >>> * **at
>>> >>> org.apache.felix.ipojo.util.Callback.call(Callback.java:191)*
>>> >>> >> >> >>> * **at
>>> >>> >> >> >>>
>>> >>> >> >> >>>
>>> >>> >> >>
>>> >>> >>
>>> >>>
>>> org.apache.felix.ipojo.handlers.lifecycle.callback.LifecycleCallback.call(LifecycleCallback.java:86)
>>> >>> >> >> >>> *
>>> >>> >> >> >>> * **at
>>> >>> >> >> >>>
>>> >>> >> >> >>>
>>> >>> >> >>
>>> >>> >>
>>> >>>
>>> org.apache.felix.ipojo.handlers.lifecycle.callback.LifecycleCallbackHandler.__stateChanged(LifecycleCallbackHandler.java:162)
>>> >>> >> >> >>> *
>>> >>> >> >> >>> * **at
>>> >>> >> >> >>>
>>> >>> >> >> >>>
>>> >>> >> >>
>>> >>> >>
>>> >>>
>>> org.apache.felix.ipojo.handlers.lifecycle.callback.LifecycleCallbackHandler.stateChanged(LifecycleCallbackHandler.java)
>>> >>> >> >> >>> *
>>> >>> >> >> >>> * **at
>>> >>> >> >> >>>
>>> >>> >> >>
>>> >>> >>
>>> >>>
>>> org.apache.felix.ipojo.InstanceManager.setState(InstanceManager.java:440)*
>>> >>> >> >> >>> * **at
>>> >>> >> >> >>>
>>> >>> >>
>>> org.apache.felix.ipojo.InstanceManager.start(InstanceManager.java:321)*
>>> >>> >> >> >>> * **at
>>> >>> >> >> >>>
>>> >>> >> >> >>>
>>> >>> >> >>
>>> >>> >>
>>> >>>
>>> org.apache.felix.ipojo.ComponentFactory.createInstance(ComponentFactory.java:155)
>>> >>> >> >> >>> *
>>> >>> >> >> >>> * **at
>>> >>> >> >> >>>
>>> >>> >> >> >>>
>>> >>> >> >>
>>> >>> >>
>>> >>>
>>> org.apache.felix.ipojo.IPojoFactory.createComponentInstance(IPojoFactory.java:298)
>>> >>> >> >> >>> *
>>> >>> >> >> >>> * **at
>>> >>> >> >> >>>
>>> >>> >> >> >>>
>>> >>> >> >>
>>> >>> >>
>>> >>>
>>> org.apache.felix.ipojo.IPojoFactory.createComponentInstance(IPojoFactory.java:235)
>>> >>> >> >> >>> *
>>> >>> >> >> >>> * **at
>>> >>> >> >>
>>> org.apache.felix.ipojo.IPojoFactory.updated(IPojoFactory.java:603)*
>>> >>> >> >> >>> * **at
>>> >>> >> >> >>>
>>> >>> >> >> >>>
>>> >>> >> >>
>>> >>> >>
>>> >>>
>>> org.apache.felix.cm.impl.ConfigurationManager$ManagedServiceFactoryUpdate.run(ConfigurationManager.java:1279)
>>> >>> >> >> >>> *
>>> >>> >> >> >>> * **at
>>> >>> >> org.apache.felix.cm.impl.UpdateThread.run(UpdateThread.java:88)*
>>> >>> >> >> >>>
>>> >>> >> >> >>> I've also tried the following:
>>> >>> >> >> >>>
>>> >>> >> >> >>>        interceptFrom().when(simple("${file:length}
==
>>> >>> >> >> >>> 0")).to(skipped).end();
>>> >>> >> >> >>>
>>> >>> >> >> >>> But it gives the following compilation
error:
>>> >>> >> >> >>>
>>> >>> >> >> >>> *The method when(Predicate) in the
type InterceptDefinition
>>> is
>>> >>> not
>>> >>> >> >> >>> applicable for the arguments (ValueBuilder)*
>>> >>> >> >> >>>
>>> >>> >> >> >>> What am I missing? This is the first
time I'm using simple
>>> >>> language
>>> >>> >> and
>>> >>> >> >> >>> the
>>> >>> >> >> >>> first time I'm using interceptors.
I should also mention that
>>> I
>>> >>> >> deploy
>>> >>> >> >> the
>>> >>> >> >> >>> camel route in Karaf 1.6.0 which
means that OSGI could play a
>>> >>> part I
>>> >>> >> >> >>> guess.
>>> >>> >> >> >>>
>>> >>> >> >> >>> /Bengt
>>> >>> >> >> >>>
>>> >>> >> >> >>>
>>> >>> >> >> >>
>>> >>> >> >> >
>>> >>> >> >>
>>> >>> >> >>
>>> >>> >> >>
>>> >>> >> >> --
>>> >>> >> >> Claus Ibsen
>>> >>> >> >> Apache Camel Committer
>>> >>> >> >>
>>> >>> >> >> Author of Camel in Action: http://www.manning.com/ibsen/
>>> >>> >> >> Open Source Integration: http://fusesource.com
>>> >>> >> >> Blog: http://davsclaus.blogspot.com/
>>> >>> >> >> Twitter: http://twitter.com/davsclaus
>>> >>> >> >>
>>> >>> >> >
>>> >>> >>
>>> >>> >>
>>> >>> >>
>>> >>> >> --
>>> >>> >> Claus Ibsen
>>> >>> >> Apache Camel Committer
>>> >>> >>
>>> >>> >> Author of Camel in Action: http://www.manning.com/ibsen/
>>> >>> >> Open Source Integration: http://fusesource.com
>>> >>> >> Blog: http://davsclaus.blogspot.com/
>>> >>> >> Twitter: http://twitter.com/davsclaus
>>> >>> >>
>>> >>> >
>>> >>>
>>> >>>
>>> >>>
>>> >>> --
>>> >>> Claus Ibsen
>>> >>> Apache Camel Committer
>>> >>>
>>> >>> Author of Camel in Action: http://www.manning.com/ibsen/
>>> >>> Open Source Integration: http://fusesource.com
>>> >>> Blog: http://davsclaus.blogspot.com/
>>> >>> Twitter: http://twitter.com/davsclaus
>>> >>>
>>> >>
>>> >>
>>> >
>>>
>>>
>>>
>>> --
>>> Claus Ibsen
>>> Apache Camel Committer
>>>
>>> Author of Camel in Action: http://www.manning.com/ibsen/
>>> Open Source Integration: http://fusesource.com
>>> Blog: http://davsclaus.blogspot.com/
>>> Twitter: http://twitter.com/davsclaus
>>>
>>
>
>
>
> --
> Claus Ibsen
> Apache Camel Committer
>
> Author of Camel in Action: http://www.manning.com/ibsen/
> Open Source Integration: http://fusesource.com
> Blog: http://davsclaus.blogspot.com/
> Twitter: http://twitter.com/davsclaus
>



-- 
Claus Ibsen
Apache Camel Committer

Author of Camel in Action: http://www.manning.com/ibsen/
Open Source Integration: http://fusesource.com
Blog: http://davsclaus.blogspot.com/
Twitter: http://twitter.com/davsclaus

Mime
View raw message