camel-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jean Francois LE BESCONT <jflebesc...@gmail.com>
Subject Re: Simple split / tokenize problem
Date Tue, 08 Apr 2014 09:22:17 GMT
Hi

Thanks Claus, you are right it is an encoding issue !

And a simple :

from("file://[...]&charset=iso-8859-1")

Solve it.

I am never used  java.util.Scanner to program file parsing (opencsv, old
bufferreaader etc .) but I have never add this kind of encoding problem ...


By the way, thank (again) you save my day :)

JF



2014-04-08 10:49 GMT+02:00 Claus Ibsen <claus.ibsen@gmail.com>:

> Hi
>
> You may need to set a charset to support reading the file with a given
> charset that supports those extended charachters
> http://camel.apache.org/file2
>
> Or set the jvm -D file.encoding option or something.
>
> Also java.util.Scanner parsing with those special chars is likely a
> common problem, so you can find detail on the internet how to deal
> with that.
>
> On Mon, Apr 7, 2014 at 9:39 PM, Jean Francois LE BESCONT
> <jflebescont@gmail.com> wrote:
> > Hey,
> >
> > I forget to specify that this problem can be reproduce this other special
> > caracter like ¤
> >
> > Thanks
> >
> > JF
> >
> >
> >
> >
> > 2014-04-07 21:30 GMT+02:00 Jean Francois LE BESCONT <
> jflebescont@gmail.com>:
> >
> >> Thanks claus !
> >>
> >> Oh I didn't realized that "Camel versions 2.11.x and older are no longer
> >> actively developed.".
> >>
> >> So I retried with the fresh homemade  version 2.13.0 :)
> >>
> >> And my  simple :
> >>
> >>
> >>
> from("file://C:/Users/EJEALEB/Desktop/POUBELLE/2/camel-spring-test/src/main/resources/input?noop=true")
> >> .log("start process ${file:name}")
> >> .split()
> >> .tokenize("\n")
> >>  .to("log:bugz")
> >>  .end()
> >> .log("end process ${file:name}")
> >> .end();
> >>
> >> With a file :
> >>
> >>
> >>
> 1;RFN;2;;2;1;1;1;1;0;E;;XXXXXXX;XXXXXXX;;;20140325105550;131;0;0;0;gernx800;;250143309824;0;1;RRRR;0;;0;0;0;0;0;;;;;0
> >>
> >>
> 3;RFN;2;;2;1;1;1;1;0;E;;XXXXXXX;XXXXXXX;;;20140325105550;131;0;0;0;gernx800;;250143309824;0;1;RRRR;0;;0;0;0;0;0;;;;;0
> >>
> >>
> 4;RFN;2;;2;1;1;1;1;0;E;;XXXXXXX;XXXXXXX;;;20140325105550;131;0;0;0;gernx°800;;250143309824;0;1;RRRR;0;;0;0;0;0;0;;;;;0
> >>
> >>
> 5;RFN;2;;2;1;1;1;1;0;E;;XXXXXXX;XXXXXXX;;;20140325105550;131;0;0;0;gernx800;;250143309824;0;1;RRRR;0;;0;0;0;0;0;;;;;0
> >>
> >>
> 7;RFN;2;;2;1;1;1;1;0;E;;XXXXXXX;XXXXXXX;;YYYYYYYY;20140325105609;7;0;0;0;n800;;688431300608;0;1;ZZZZZ;0;;0;0;0;0;0;;;;;0
> >>
> >> Doesn't stop strangly like 2.11.0 but throw an exception :
> >>
> >> 2014-04-07 21:16:17 - [-test/src/main/resources/input]
> DefaultErrorHandler
> >>            ERROR Failed delivery for (MessageId:
> >> ID-E7B499BAEC2534-61680-1396898175486-0-1 on ExchangeId:
> >> ID-E7B499BAEC2534-61680-1396898175486-0-2). Exhausted after delivery
> >> attempt: 1 caught: org.apache.camel.RuntimeCamelException: Scanner
> aborted
> >> because of an IOException!
> >>
> >> And if I delete the ° character of the sequence "rnx°800"  the route
> >> works fine.
> >>
> >> So to summarize :
> >>
> >> with 2.11 ( version in production of our fuse esb ) : line ignored
> >> with 2.13 ( version in production of our fuse esb ) : exception
> >>
> >> I have uploaded a runnable test in the repo :
> >>
> >> https://github.com/0loky0/github-camel-spring-test
> >>
> >> I don't know if it is a bad utilization of the tokenize or if it is a
> real
> >> bug (jirable)
> >>
> >> Thanks all !
> >>
> >> JF
> >>
> >>
> >> Le lundi 7 avril 2014, Claus Ibsen <claus.ibsen@gmail.com> a écrit :
> >>
> >> > Hi
> >> >
> >> > As we say on this page. Try with newer Camel version to see which (if
> >> > any) it has been fixed
> >> > http://camel.apache.org/support
> >> >
> >> > On Mon, Apr 7, 2014 at 3:16 PM, Jean Francois LE BESCONT
> >> > <jflebescont@gmail.com> wrote:
> >> >> Hey !
> >> >>
> >> >> I am a little bit disappointed about an apache camel simple example.
> >> >>
> >> >> The configuration is :
> >> >>
> >> >> project.build.sourceEncoding : UTF-8
> >> >> project.reporting.outputEncoding : UTF-8
> >> >> org.apache.camel.version: 2.11.0
> >> >> org.spring.version :3.0.7.RELEASE
> >> >>
> >> >> The source code of this spring camel project is here :
> >> >> https://github.com/0loky0/camel-spring-test
> >> >> ( FUSE IDE project)
> >> >> Only one route while only parse data and print it.
> >> >>
> >> >>
> >> >> from("file://C:/dir/camel-spring-test/input_dir/?noop=true")
> >> >>   .split()
> >> >>        .tokenize("\n")
> >> >>       .to("log:bugz")
> >> >> .end();
> >> >>
> >> >> With the file in the project
> >> >> the input_dir/VDE20140326184821.MGC1.05.000000026.csv
> >> >>
> >> >> The process stop after 15 lines  !!!
> >> >>
> >> >>                       main] SpringCamelContext             INFO
>  Route:
> >> >> route1 started and consuming from:
> >> >>
> >>
> Endpoint[file://C:/Users/EJEALEB/Desktop/POUBELLE/New%20folder/camel-spring-test/input_dir/?noop=true]
> >> >> 2014-04-07 15:09:17 - [                          main]
> >> >> ultManagementLifecycleStrategy INFO  Load performance statistics
> >> enabled.
> >> >> 2014-04-07 15:09:17 - [                          main]
> >> SpringCamelContext
> >> >>           INFO  Total 1 routes, of which 1 is started.
> >> >> 2014-04-07 15:09:17 - [                          main]
> >> SpringCamelContext
> >> >>           INFO  Apache Camel 2.11.0 (CamelContext: camel-1) started
> in
> >> >> 0.815 seconds
> >> >> 2014-04-07 15:09:18 - [r/camel-spring-test/input_dir/] bugz
> >> >>           INFO  Exchange[ExchangePattern:InOnly, BodyType:String,
> >> >>
> >>
> Body:2238850;RFN;2;;2;1;1;1;1;0;E;;385255353;385255303;;811702703;20140325105550;131;0;0;0;UGAP
> >> >> nx800;;250143309824;0;1;427020458;0;;0;0;0;0;0;;;;;0]
> >> >> 2014-04-07 15:09:18 - [r/camel-spring-test/input_dir/] bugz
> >> >>           INFO  Exchange[ExchangePattern:InOnly, BodyType:String,
> >> >>
> >>
> Body:2238843;RFN;2;;2;1;1;2;1;0;E;;557400831;;;805801802;20140325105546;106;0;0;0;ELOQUANT;;907709644800;0;1;457381415;0;;0;0;0;0;0;;;B000;;0]
> >> >> 2014-04-07 15:09:18 - [r/camel-spring-test/input_dir/] bugz
> >> >>           INFO  Exchange[ExchangePattern:InOnly, BodyType:String,
> >> >>
> >>
> Body:2238844;RFN;2;;2;1;1;2;1;0;E;;546850551;;;820902500;20140325105539;179;0;0;0;LYRA
> >> >> NETWO;;229054480384;0;1;171301007;0;;0;0;0;0;0;;;;;0]
> >> >> 2014-04-07 15:09:18 - [r/camel-spring-test/input_dir/] bugz
> >> >>           INFO  Exchange[ExchangePattern:InOnly, BodyType:String,
> >> >>
> >>
> Body:2238845;RFN;2;;2;1;1;2;1;0;E;;468478708;;;820902500;20140325105542;168;0;0;0;LYRA
> >> >> NETWO;;963040772096;0;1;171301007;0;;0;0;0;0;0;;ACNC;;;0]
> >> >> 2014-04-07 15:09:18 - [r/camel-spring-test/input_dir/] bugz
> >> >>           INFO  Exchange[ExchangePattern:InOnly, BodyType:String,
> >> >>
> >>
> Body:2238846;RFN;2;;2;1;1;2;1;0;E;;555361506;555361506;;810333683;20140325105551;70;0;0;0;ELECTRICIT;;1035954683904;170;1;369186100;0;;0;0;0;0;0;;10070;;;0]
> >> >> 2014-04-07 15:09:18 - [r/camel-spring-test/input_dir/] bugz
> >> >>           INFO  Exchange[ExchangePattern:InOnly, BodyType:String,
> >> >>
> >>
> Body:2238847;RFN;2;;2;1;1;1;1;0;E;;479320017;;;811367617;20140325105223;2182;0;0;0;SFR
> >> >> - MANA;;797835591680;0;1;185060411;0;;0;0;0;0;0;;;;;0]
> >> >> 2014-04-07 15:09:18 - [r/camel-spring-test/input_dir/] bugz
> >> >>           INFO  Exchange[ExchangePattern:InOnly, BodyType:String,
> >> >>
> >>
> Body:2238848;RFN;2;;2;1;1;2;1;0;E;;479004426;;;820902500;20140325105544;174;0;0;0;LYRA
> >> >> NETWO;;675764633600;0;1;171301007;0;;0;0;0;0;--
> >>
> >> > Claus Ibsen
> >> > -----------------
> >> > Red Hat, Inc.
> >> > Email: cibsen@redhat.com
> >> > Twitter: davsclaus
> >> > Blog: http://davsclaus.com
> >> > Author of Camel in Action: http://www.manning.com/ibsen
> >> > Make your Camel applications look hawt, try: http://hawt.io
> >> >
> >>
>
>
>
> --
> Claus Ibsen
> -----------------
> Red Hat, Inc.
> Email: cibsen@redhat.com
> Twitter: davsclaus
> Blog: http://davsclaus.com
> Author of Camel in Action: http://www.manning.com/ibsen
> Make your Camel applications look hawt, try: http://hawt.io
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message