camel-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Claus Ibsen <claus.ib...@gmail.com>
Subject Re: Skipping empty files, problem with simple language
Date Fri, 03 Sep 2010 04:45:26 GMT
On Thu, Sep 2, 2010 at 7:43 PM, Bengt Rodehav <bengt@rodehav.com> wrote:
> Even better - looking forward to it. I must say that Camel is impressively
> flexible.
>
> BTW we recently put the first version of our Camel/Karaf based integration
> platform into production and it works very well indeed. I'm very pleased!
>

That is great news. Glad that you made it into production. Getting the
first app into production is usually the biggest obstacle.
If it performs well, then its usually easier to get the next apps in
production as well.


> /Bengt
>
> 2010/9/2 Claus Ibsen <claus.ibsen@gmail.com>
>
>> On Thu, Sep 2, 2010 at 4:57 PM, Bengt Rodehav <bengt@rodehav.com> wrote:
>> > Just to sum up. Recipient list worked perfectly. I now use the following
>> > interceptor:
>> >
>> >        interceptFrom().when(
>> >            PredicateBuilder.toPredicate(SimpleLanguage
>> >                .simple("${file:length} == null"))).setHeader("recipient")
>> >            .simple("file://${file:parent}/skipped/${date:now:yyyyMMdd}")
>> >            .recipientList(header("recipient")).stop();
>> >
>> > All empty files will then be moved to the "skipped/<yyyyMMdd>" folder
>> under
>> > the folder where the polled file resided. Beautiful solution.
>> >
>>
>> Will be more beautiful in Camel 2.5, with simple directly in the
>> RouteBuilder.
>>
>> And you can move the ugly Predicate before the intercept
>>
>> Predicate emptyFile =
>> PredicateBuilder.toPredicate(SimpleLanguage.simple("${file:length} ==
>> null"));
>>
>>
>> interceptFrom().when(emptyFile).recipientList(simple("file://${file:parent}/skipped/${date:now:yyyyMMdd}")).stop();
>>
>>
>> And you can set the expression directly in the recipientList, you dont
>> have to use a header.
>>
>>
>> > Thanks Claus and Willem for your help,
>> >
>> > /Bengt
>> >
>> >
>> > 2010/9/2 Bengt Rodehav <bengt@rodehav.com>
>> >
>> >> Thanks again Claus - will have a look at recipient list.
>> >>
>> >> BTW seems like camel.apache.org is not accessible for the moment -
>> >> problems?
>> >>
>> >> /Bengt
>> >>
>> >> 2010/9/2 Claus Ibsen <claus.ibsen@gmail.com>
>> >>
>> >>> Use dynamic recipient list to construct the endpoint uri on-the-fly
>> >>> http://camel.apache.org/recipient-list.html
>> >>>
>> >>>
>> >>> On Thu, Sep 2, 2010 at 1:07 PM, Bengt Rodehav <bengt@rodehav.com>
>> wrote:
>> >>> > Thanks Claus,
>> >>> >
>> >>> > stop() worked perfectly - I had confused it with end() but stand
>> >>> corrected.
>> >>> >
>> >>> > However, the Properties component does not seem to address my
>> specific
>> >>> > problem. I need to get hold of properties from the exchange itself
-
>> not
>> >>> > from an external properties file.
>> >>> >
>> >>> > If the file is polled from the folder "C:\in-box" then I want the
>> empty
>> >>> > files to end up in the folder "C:\in-box\skipped\<date>"
where <date>
>> is
>> >>> the
>> >>> > date the exchange was processed. I cannot  use property files
for the
>> >>> base
>> >>> > directory either since this is a general component that can be
>> >>> configured to
>> >>> > read files from many different places. No matter what folder the
>> input
>> >>> file
>> >>> > is polled from, I want skipped files in the "skipped" subfolder
of
>> that
>> >>> > original folder.
>> >>> >
>> >>> > It's very similar to the  functionality that I use when archiving
>> >>> successful
>> >>> > and failed exchanges. I use the following uri options for that:
>> >>> >
>> >>> > move=archive/${date:now:yyyyMMdd}/${file:name}
>> >>> >
>> >>>
>> moveFailed=failed/${file:name.noext}-${date:now:yyyyMMddHHmmssSSS}.${file:ext}
>> >>> >
>> >>> > I now want to use the simple (and file) language to specify where
>> >>> skipped
>> >>> > files are archived. I haven't investigated how the move/moveFailed
>> >>> options
>> >>> > are actually implemented but I guess I need to do it in a similar
way
>> >>> unless
>> >>> > there is an easy way to use simple/file language directly in an
>> endpoint
>> >>> > uri.
>> >>> >
>> >>> > /Bengt
>> >>> >
>> >>> > 2010/9/2 Claus Ibsen <claus.ibsen@gmail.com>
>> >>> >
>> >>> >> Ad 1)
>> >>> >> See the properties component
>> >>> >> http://camel.apache.org/properties
>> >>> >>
>> >>> >> Ad 2)
>> >>> >> Use stop()
>> >>> >>
>> >>> >> On Thu, Sep 2, 2010 at 11:17 AM, Bengt Rodehav <bengt@rodehav.com>
>> >>> wrote:
>> >>> >> > Thanks Willem it worked perfectly.
>> >>> >> >
>> >>> >> > However, I now have two other problems (if you bear with
me...):
>> >>> >> >
>> >>> >> > I want to use this interceptor:
>> >>> >> >
>> >>> >> >
>> >>> >>
>> >>>
>> *interceptFrom().when(PredicateBuilder.toPredicate(SimpleLanguage.simple("${file:length}
>> >>> >> > ==
>> >>> >>
>> null"))).to("file://${file:path}/${date:now:yyyyMMdd}/skipped").end();*
>> >>> >> >
>> >>> >> > 1. I get the following exception:
>> >>> >> >
>> >>> >> > *java.lang.IllegalArgumentException: Invalid directory:
>> >>> >> > skipped/${date:now:yyyyMMdd}. Dynamic expressions with
${ }
>> >>> placeholders
>> >>> >> is
>> >>> >> > not allowed. Use the fileName option to set the dynamic
>> expression.*
>> >>> >> >
>> >>> >> > How can I direct the file to a directory that I need properties
>> (file
>> >>> and
>> >>> >> > date) to calculate?
>> >>> >> >
>> >>> >> > 2. The interceptor does not "skip" the file. It sends
it both to
>> the
>> >>> >> > original endpoint and to my "skipped" folder. How can
I make it
>> NOT
>> >>> send
>> >>> >> the
>> >>> >> > file to the original endpoint. I've seen the
>> >>> >> >
>> >>> >> >  *interceptSendToEndpoint("...").skipSendToOriginalEndpoint();*
>> >>> >> >
>> >>> >> > but in this case I want to use the interceptFrom() since
I want to
>> >>> >> intercept
>> >>> >> > the route at the earliest possible stage.
>> >>> >> >
>> >>> >> > /Bengt
>> >>> >> >
>> >>> >> >
>> >>> >> >
>> >>> >> > 2010/9/2 Claus Ibsen <claus.ibsen@gmail.com>
>> >>> >> >
>> >>> >> >> On Thu, Sep 2, 2010 at 9:20 AM, Bengt Rodehav <bengt@rodehav.com
>> >
>> >>> >> wrote:
>> >>> >> >> > Hi Willem,
>> >>> >> >> >
>> >>> >> >> > Thanks for the info. I assume then that as long
as I'm on Camel
>> >>> 2.4
>> >>> >> I'll
>> >>> >> >> > check for null but when I upgrade to Camel 2.5
I'll start
>> checking
>> >>> for
>> >>> >> >> zero
>> >>> >> >> > instead.
>> >>> >> >> >
>> >>> >> >> > However, that issue aside, I still get the same
problems that I
>> >>> wrote
>> >>> >> >> about.
>> >>> >> >> > Can you confirm that the following syntax is
correct?
>> >>> >> >> >
>> >>> >> >> > interceptFrom().when(simple("${file:length} ==
>> >>> 0")).to(skipped).end();
>> >>> >> >> >
>> >>> >> >> > OR (as long as I stay on Camel 2.4)
>> >>> >> >> >
>> >>> >> >> > interceptFrom().when(simple("${file:length} ==
>> >>> >> null")).to(skipped).end();
>> >>> >> >> >
>> >>> >> >> > Both of the above give me a compilation error
since the
>> "when()"
>> >>> >> method
>> >>> >> >> > expects a "Predicate" while "simple()" returns
a String. If I
>> >>> instead
>> >>> >> use
>> >>> >> >> > "simple(..)..isEqualTo(0)" then the compiler
accepts it (since
>> >>> >> >> "isEqualTo()"
>> >>> >> >> > returns a Predicate), but then I get the runtime
exception
>> >>> instead.
>> >>> >> >> >
>> >>> >> >>
>> >>> >> >> That is also part of Camel 2.5 that simple is easier
to use in
>> the
>> >>> >> >> RouteBuilder out of the box.
>> >>> >> >>
>> >>> >> >> Just use a
>> PredicateBuilder.toPredicate(SimpleLanguage.simple("foo")
>> >>> >> >> to construct the predicate.
>> >>> >> >>
>> >>> >> >>
>> >>> >> >> > I'v double checked my dependencies and it turns
out that I'm
>> using
>> >>> a
>> >>> >> >> > snapshot version of Camel 2.4 based on revision
958950. Maybe
>> my
>> >>> >> problems
>> >>> >> >> > stem from that... I will check of course.
>> >>> >> >> >
>> >>> >> >> > Do you know if the "when()" method will accept
a String as
>> >>> parameter
>> >>> >> in
>> >>> >> >> > Camel 2.4?
>> >>> >> >> >
>> >>> >> >> > /Bengt
>> >>> >> >> >
>> >>> >> >> >
>> >>> >> >> > 2010/9/2 Willem Jiang <willem.jiang@gmail.com>
>> >>> >> >> >
>> >>> >> >> >> Hi Bengt,
>> >>> >> >> >>
>> >>> >> >> >> I tried to reproduce you issue in Camel trunk,
and found this
>> >>> bug[1].
>> >>> >> >> >> BTW, If you just want to filter the zero
length file, you can
>> >>> take a
>> >>> >> >> look
>> >>> >> >> >> this unit test file[2] and change the simple
expression to
>> >>> >> >> "${file:length}
>> >>> >> >> >> == null" as in Camel 2.4.0, ${file:length}
is null if the
>> >>> file.length
>> >>> >> is
>> >>> >> >> 0.
>> >>> >> >> >>
>> >>> >> >> >>
>> >>> >> >> >> [1]https://issues.apache.org/activemq/browse/CAMEL-3100
>> >>> >> >> >> [2]
>> >>> >> >> >>
>> >>> >> >>
>> >>> >>
>> >>>
>> https://svn.apache.org/repos/asf/camel/trunk/camel-core/src/test/java/org/apache/camel/component/file/FileConsumerIntercepEmptyFileTest.java
>> >>> >> >> >>
>> >>> >> >> >> Willem
>> >>> >> >> >>
>> >>> >> >> >>
>> >>> >> >> >> Bengt Rodehav wrote:
>> >>> >> >> >>
>> >>> >> >> >>> I've developed a file transfer service
(for
>> file/ftp/ftps/sftp)
>> >>> that
>> >>> >> >> uses
>> >>> >> >> >>> Camel 2.4. I need to be able to skip
transferring empty files
>> >>> and
>> >>> >> was
>> >>> >> >> >>> hoping
>> >>> >> >> >>> to use an interceptor to accomplish this.
Below is the
>> relevant
>> >>> >> code:
>> >>> >> >> >>>
>> >>> >> >> >>>  String skippedUri =
>> >>> >> >> "file://skipped/${date:now:yyyyMMdd}/${file:name}";
>> >>> >> >> >>>
>> >>> >> >> >>>
>> >>> >> >>
>> >>> >>
>> >>>
>>  interceptFrom().when(simple("${file:length}").isEqualTo(0)).to(skippedUri).end();
>> >>> >> >> >>>
>> >>> >> >> >>> This compiles but in runtime a get the
following exception
>> when
>> >>> >> trying
>> >>> >> >> to
>> >>> >> >> >>> start the route:
>> >>> >> >> >>>
>> >>> >> >> >>> *java.lang.NoSuchMethodError:
>> >>> >> >> >>>
>> >>> >> >> >>>
>> >>> >> >>
>> >>> >>
>> >>>
>> se.digia.connect.services.skandia.filetransfer.FileTransferService$1.simple(Ljava/lang/String;)Lorg/apache/camel/builder/ValueBuilder;
>> >>> >> >> >>> *
>> >>> >> >> >>> * **at
>> >>> >> >> >>>
>> >>> >> >> >>>
>> >>> >> >>
>> >>> >>
>> >>>
>> se.digia.connect.services.skandia.filetransfer.FileTransferService$1.configure(FileTransferService.java:279)
>> >>> >> >> >>> *
>> >>> >> >> >>> * **at
>> >>> >> >> >>>
>> >>> >> >> >>>
>> >>> >> >>
>> >>> >>
>> >>>
>> org.apache.camel.builder.RouteBuilder.checkInitialized(RouteBuilder.java:295)
>> >>> >> >> >>> *
>> >>> >> >> >>> * **at
>> >>> >> >> >>>
>> >>> >> >> >>>
>> >>> >> >>
>> >>> >>
>> >>>
>> org.apache.camel.builder.RouteBuilder.configureRoutes(RouteBuilder.java:250)
>> >>> >> >> >>> *
>> >>> >> >> >>> * **at
>> >>> >> >> >>>
>> >>> >> >> >>>
>> >>> >> >>
>> >>> >>
>> >>>
>> org.apache.camel.builder.RouteBuilder.addRoutesToCamelContext(RouteBuilder.java:236)
>> >>> >> >> >>> *
>> >>> >> >> >>> * **at
>> >>> >> >> >>>
>> >>> >> >> >>>
>> >>> >> >>
>> >>> >>
>> >>>
>> org.apache.camel.impl.DefaultCamelContext.addRoutes(DefaultCamelContext.java:498)
>> >>> >> >> >>> *
>> >>> >> >> >>> * **at
>> >>> >> >> >>>
>> >>> >> >> >>>
>> >>> >> >>
>> >>> >>
>> >>>
>> se.digia.connect.core.service.RouteServiceBase.doStart(RouteServiceBase.java:42)
>> >>> >> >> >>> *
>> >>> >> >> >>> * **at
>> >>> >> >> >>>
>> >>> >>
>> se.digia.connect.core.service.ServiceBase.start(ServiceBase.java:49)*
>> >>> >> >> >>> * **at
>> >>> >> >> >>>
>> >>> >> >> >>>
>> >>> >> >>
>> >>> >>
>> >>>
>> se.digia.connect.services.skandia.filetransfer.FileTransferService.__start(FileTransferService.java:60)
>> >>> >> >> >>> *
>> >>> >> >> >>> * **at
>> >>> >> >> >>>
>> >>> >> >> >>>
>> >>> >> >>
>> >>> >>
>> >>>
>> se.digia.connect.services.skandia.filetransfer.FileTransferService.start(FileTransferService.java)
>> >>> >> >> >>> *
>> >>> >> >> >>> * **at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
>> >>> Method)*
>> >>> >> >> >>> * **at
>> >>> >> >> >>>
>> >>> >> >> >>>
>> >>> >> >>
>> >>> >>
>> >>>
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>> >>> >> >> >>> *
>> >>> >> >> >>> * **at
>> >>> >> >> >>>
>> >>> >> >> >>>
>> >>> >> >>
>> >>> >>
>> >>>
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>> >>> >> >> >>> *
>> >>> >> >> >>> * **at java.lang.reflect.Method.invoke(Method.java:597)*
>> >>> >> >> >>> * **at
>> >>> org.apache.felix.ipojo.util.Callback.call(Callback.java:235)*
>> >>> >> >> >>> * **at
>> >>> org.apache.felix.ipojo.util.Callback.call(Callback.java:191)*
>> >>> >> >> >>> * **at
>> >>> >> >> >>>
>> >>> >> >> >>>
>> >>> >> >>
>> >>> >>
>> >>>
>> org.apache.felix.ipojo.handlers.lifecycle.callback.LifecycleCallback.call(LifecycleCallback.java:86)
>> >>> >> >> >>> *
>> >>> >> >> >>> * **at
>> >>> >> >> >>>
>> >>> >> >> >>>
>> >>> >> >>
>> >>> >>
>> >>>
>> org.apache.felix.ipojo.handlers.lifecycle.callback.LifecycleCallbackHandler.__stateChanged(LifecycleCallbackHandler.java:162)
>> >>> >> >> >>> *
>> >>> >> >> >>> * **at
>> >>> >> >> >>>
>> >>> >> >> >>>
>> >>> >> >>
>> >>> >>
>> >>>
>> org.apache.felix.ipojo.handlers.lifecycle.callback.LifecycleCallbackHandler.stateChanged(LifecycleCallbackHandler.java)
>> >>> >> >> >>> *
>> >>> >> >> >>> * **at
>> >>> >> >> >>>
>> >>> >> >>
>> >>> >>
>> >>>
>> org.apache.felix.ipojo.InstanceManager.setState(InstanceManager.java:440)*
>> >>> >> >> >>> * **at
>> >>> >> >> >>>
>> >>> >>
>> org.apache.felix.ipojo.InstanceManager.start(InstanceManager.java:321)*
>> >>> >> >> >>> * **at
>> >>> >> >> >>>
>> >>> >> >> >>>
>> >>> >> >>
>> >>> >>
>> >>>
>> org.apache.felix.ipojo.ComponentFactory.createInstance(ComponentFactory.java:155)
>> >>> >> >> >>> *
>> >>> >> >> >>> * **at
>> >>> >> >> >>>
>> >>> >> >> >>>
>> >>> >> >>
>> >>> >>
>> >>>
>> org.apache.felix.ipojo.IPojoFactory.createComponentInstance(IPojoFactory.java:298)
>> >>> >> >> >>> *
>> >>> >> >> >>> * **at
>> >>> >> >> >>>
>> >>> >> >> >>>
>> >>> >> >>
>> >>> >>
>> >>>
>> org.apache.felix.ipojo.IPojoFactory.createComponentInstance(IPojoFactory.java:235)
>> >>> >> >> >>> *
>> >>> >> >> >>> * **at
>> >>> >> >>
>> org.apache.felix.ipojo.IPojoFactory.updated(IPojoFactory.java:603)*
>> >>> >> >> >>> * **at
>> >>> >> >> >>>
>> >>> >> >> >>>
>> >>> >> >>
>> >>> >>
>> >>>
>> org.apache.felix.cm.impl.ConfigurationManager$ManagedServiceFactoryUpdate.run(ConfigurationManager.java:1279)
>> >>> >> >> >>> *
>> >>> >> >> >>> * **at
>> >>> >> org.apache.felix.cm.impl.UpdateThread.run(UpdateThread.java:88)*
>> >>> >> >> >>>
>> >>> >> >> >>> I've also tried the following:
>> >>> >> >> >>>
>> >>> >> >> >>>        interceptFrom().when(simple("${file:length}
==
>> >>> >> >> >>> 0")).to(skipped).end();
>> >>> >> >> >>>
>> >>> >> >> >>> But it gives the following compilation
error:
>> >>> >> >> >>>
>> >>> >> >> >>> *The method when(Predicate) in the type
InterceptDefinition
>> is
>> >>> not
>> >>> >> >> >>> applicable for the arguments (ValueBuilder)*
>> >>> >> >> >>>
>> >>> >> >> >>> What am I missing? This is the first
time I'm using simple
>> >>> language
>> >>> >> and
>> >>> >> >> >>> the
>> >>> >> >> >>> first time I'm using interceptors. I
should also mention that
>> I
>> >>> >> deploy
>> >>> >> >> the
>> >>> >> >> >>> camel route in Karaf 1.6.0 which means
that OSGI could play a
>> >>> part I
>> >>> >> >> >>> guess.
>> >>> >> >> >>>
>> >>> >> >> >>> /Bengt
>> >>> >> >> >>>
>> >>> >> >> >>>
>> >>> >> >> >>
>> >>> >> >> >
>> >>> >> >>
>> >>> >> >>
>> >>> >> >>
>> >>> >> >> --
>> >>> >> >> Claus Ibsen
>> >>> >> >> Apache Camel Committer
>> >>> >> >>
>> >>> >> >> Author of Camel in Action: http://www.manning.com/ibsen/
>> >>> >> >> Open Source Integration: http://fusesource.com
>> >>> >> >> Blog: http://davsclaus.blogspot.com/
>> >>> >> >> Twitter: http://twitter.com/davsclaus
>> >>> >> >>
>> >>> >> >
>> >>> >>
>> >>> >>
>> >>> >>
>> >>> >> --
>> >>> >> Claus Ibsen
>> >>> >> Apache Camel Committer
>> >>> >>
>> >>> >> Author of Camel in Action: http://www.manning.com/ibsen/
>> >>> >> Open Source Integration: http://fusesource.com
>> >>> >> Blog: http://davsclaus.blogspot.com/
>> >>> >> Twitter: http://twitter.com/davsclaus
>> >>> >>
>> >>> >
>> >>>
>> >>>
>> >>>
>> >>> --
>> >>> Claus Ibsen
>> >>> Apache Camel Committer
>> >>>
>> >>> Author of Camel in Action: http://www.manning.com/ibsen/
>> >>> Open Source Integration: http://fusesource.com
>> >>> Blog: http://davsclaus.blogspot.com/
>> >>> Twitter: http://twitter.com/davsclaus
>> >>>
>> >>
>> >>
>> >
>>
>>
>>
>> --
>> Claus Ibsen
>> Apache Camel Committer
>>
>> Author of Camel in Action: http://www.manning.com/ibsen/
>> Open Source Integration: http://fusesource.com
>> Blog: http://davsclaus.blogspot.com/
>> Twitter: http://twitter.com/davsclaus
>>
>



-- 
Claus Ibsen
Apache Camel Committer

Author of Camel in Action: http://www.manning.com/ibsen/
Open Source Integration: http://fusesource.com
Blog: http://davsclaus.blogspot.com/
Twitter: http://twitter.com/davsclaus

Mime
View raw message