camel-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jean Francois LE BESCONT <jflebesc...@gmail.com>
Subject Re: stream:file closeOnDone not close all time
Date Mon, 20 May 2013 16:51:30 GMT
Hi !

Please h o w can I pass by this limitation ?
Le 19 mai 2013 22:58, "Jean Francois LE BESCONT" <jflebescont@gmail.com> a
écrit :

> If it is possible !
>
>
> 2013/5/19 Jean Francois LE BESCONT <jflebescont@gmail.com>
>
>> Hi,
>>
>> Could you please tell me Claus how to use the addOnCompletion to close
>> the stream ?
>>
>> THanks
>>
>>
>> Jeff
>>
>>
>> 2013/5/16 Jean Francois LE BESCONT <jflebescont@gmail.com>
>>
>>> By the way, the problem appears not on if there are two files. It
>>> appears if the last exchange is not passed to the endpoint.
>>>
>>> Example with only one file :
>>>
>>> from("file://C:/Temp/camel/rep1/?noop=true")
>>>      .split()
>>>      .tokenize("\n")
>>>      .streaming()
>>>           .process(new Processor() {
>>>
>>>                public void process(Exchange exchange) throws Exception {
>>>                     // After 2 lines, next lines are rejected via an
>>>                     // exchange property
>>>                     i++;
>>>                     if (i > 2) {
>>>                          exchange.setProperty("FILE_1", false);
>>>                     } else {
>>>                          exchange.setProperty("FILE_1", true);
>>>                     }
>>>                }
>>>           })
>>>           .choice()
>>>           .when(property("FILE_1").isEqualTo(Boolean.TRUE))
>>>
>>> .to("stream:file?fileName=C:/Temp/camel/rep1/out/out_1.csv&closeOnDone=true")
>>>           .end()
>>>       .end()
>>> .end();
>>>
>>>
>>> Jeff
>>>
>>>
>>> 2013/5/16 Jean Francois LE BESCONT <jflebescont@gmail.com>
>>>
>>>> Hi,
>>>>
>>>> Thanks Claus for the answer. I have logged a jira (
>>>> https://issues.apache.org/jira/browse/CAMEL-6367)
>>>>
>>>> I have not really understand how to implements your recommandation
>>>> about exchange.addOnCompletion. After downloaded the camel code to
>>>> understand how package org.apache.camel.converter.stream works, I
>>>> don't know how to access the fileInputStreamCache from the processor :
>>>>
>>>>
>>>> .process(new Processor() {
>>>>
>>>> public void process(Exchange exchange) throws Exception {
>>>>  exchange.addOnCompletion(new SynchronizationAdapter() {
>>>> @Override
>>>>  public void onDone(Exchange exchange) {
>>>>  FileInputStreamCache fileInputStreamCache= ?????????? ;
>>>>  try {
>>>> if (fileInputStreamCache != null) {
>>>> fileInputStreamCache.close();
>>>>  }
>>>> close();
>>>> } catch (Exception e) {
>>>>  LOG.warn("Error deleting temporary cache file: " + tempFile, e);
>>>> }
>>>> }
>>>>  });
>>>> }
>>>>
>>>> Is it possible  ?
>>>>
>>>>
>>>> Thanks
>>>>
>>>> Jeff
>>>>
>>>>
>>>> 2013/5/16 Claus Ibsen <claus.ibsen@gmail.com>
>>>>
>>>>> I have updated the wiki docs.
>>>>>
>>>>> Though we could improve the stream component to auto close the stream
>>>>> when the exchange is done at the end of routing as a fallback. Then
>>>>> your use-case with writing to 2+ files can be supported, as all the
>>>>> streams is closed when the routing is done at the end.
>>>>>
>>>>> This requires to add an onCompletion to the exchange
>>>>>
>>>>> exchange.addOnCompletion
>>>>>
>>>>> And then add logic there to close the stream
>>>>>
>>>>> Though a little challenge could be concurrency if you close the
>>>>> stream, and then another exchange is using it currently.
>>>>>
>>>>> Seems like the logic in stream producer should sync (lock) and not per
>>>>> method which is wrong.
>>>>> Or maybe better yet do not have a shared output stream.
>>>>>
>>>>> Fell free to log a JIRA ticket
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On Thu, May 16, 2013 at 8:07 AM, Claus Ibsen <claus.ibsen@gmail.com>
>>>>> wrote:
>>>>> > Hi
>>>>> >
>>>>> > That is because you do write to 2 files. The option was intended
for
>>>>> > writing to the same file.
>>>>> >
>>>>> >
>>>>> >
>>>>> >
>>>>> > On Thu, May 16, 2013 at 7:43 AM, Jean Francois LE BESCONT
>>>>> > <jflebescont@gmail.com> wrote:
>>>>> >> Hi,
>>>>> >>
>>>>> >> what should I do ?
>>>>> >>
>>>>> >> Thanks
>>>>> >>
>>>>> >> Jeff
>>>>> >>
>>>>> >>
>>>>> >> 2013/5/15 Jean Francois LE BESCONT <jflebescont@gmail.com>
>>>>> >>
>>>>> >>> Hey !
>>>>> >>>
>>>>> >>> A new option is available for the stream:file to close it
when done
>>>>> >>> (closeOnDone). This option is appeared in 2.11 after my
jira (
>>>>> >>> https://issues.apache.org/jira/browse/CAMEL-6147)
>>>>> >>>
>>>>> >>> File unlock (or released or closed) looks to doesn't works
fine if
>>>>> last
>>>>> >>> line is not passed to the endpoint
>>>>> >>>
>>>>> >>> Example :
>>>>> >>>
>>>>> >>> We have a CSV with X line. We want to write a part of it
in a file
>>>>> >>> out_1.csv and a second part in a file out_2.csv according
to a
>>>>> business
>>>>> >>> rule, in my example the rule is after two lines readed.
>>>>> >>>
>>>>> >>> An example is :
>>>>> >>>
>>>>> >>> from("file://C:/Temp/camel/rep1/?noop=true")
>>>>> >>>     .log("start process file => ${file:name}")
>>>>> >>>     .split()
>>>>> >>>     .tokenize("\n")
>>>>> >>>         .streaming()
>>>>> >>>         .process(new Processor() {
>>>>> >>>
>>>>> >>>             public void process(Exchange exchange) throws
>>>>> Exception {
>>>>> >>>                // After 2 lines, next lines are rejected
via an
>>>>> exchange
>>>>> >>> property
>>>>> >>>                i++ ;
>>>>> >>>                if( i  > 2)  {
>>>>> >>>                    exchange.setProperty("FILE_1", true );
>>>>> >>>                } else {
>>>>> >>>                    exchange.setProperty("FILE_1", false);
>>>>> >>>                }
>>>>> >>>             }
>>>>> >>>        })
>>>>> >>>        .choice()
>>>>> >>>        .when(property("FILE_1").isEqualTo(Boolean.TRUE))
>>>>> >>>
>>>>> >>>
>>>>>  .to("stream:file?fileName=C:/Temp/camel/rep1/out/out_1.csv&closeOnDone=true")
>>>>> >>>        .when(property("FILE_2").isEqualTo(Boolean.TRUE))
>>>>> >>>
>>>>> >>>
>>>>> .to("stream:file?fileName=C:/Temp/camel/rep1/out/out_2.csv&closeOnDone=true")
>>>>> >>>        .end()
>>>>> >>>     .end()
>>>>> >>> .log("end process file => ${file:name}")
>>>>> >>> .end()
>>>>> >>> ;
>>>>> >>>
>>>>> >>> It create two files, and out_1.csv is still locked.
>>>>> >>>
>>>>> >>> Should I update the jira or open a new ?
>>>>> >>>
>>>>> >>> Thanks
>>>>> >>>
>>>>> >>> Jeff
>>>>> >>>
>>>>> >>>
>>>>> >
>>>>> >
>>>>> >
>>>>> > --
>>>>> > Claus Ibsen
>>>>> > -----------------
>>>>> > www.camelone.org: The open source integration conference.
>>>>> >
>>>>> > Red Hat, Inc.
>>>>> > FuseSource is now part of Red Hat
>>>>> > Email: cibsen@redhat.com
>>>>> > Web: http://fusesource.com
>>>>> > Twitter: davsclaus
>>>>> > Blog: http://davsclaus.com
>>>>> > Author of Camel in Action: http://www.manning.com/ibsen
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Claus Ibsen
>>>>> -----------------
>>>>> www.camelone.org: The open source integration conference.
>>>>>
>>>>> Red Hat, Inc.
>>>>> FuseSource is now part of Red Hat
>>>>> Email: cibsen@redhat.com
>>>>> Web: http://fusesource.com
>>>>> Twitter: davsclaus
>>>>> Blog: http://davsclaus.com
>>>>> Author of Camel in Action: http://www.manning.com/ibsen
>>>>>
>>>>
>>>>
>>>
>>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message