camel-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jan Matèrne (jhm) <apa...@materne.de>
Subject AW: AW: Problem with ETL example
Date Fri, 13 Jun 2014 12:20:54 GMT
Thanks for explanation. I updated the wiki ;)

Jan


> -----Ursprüngliche Nachricht-----
> Von: Babak Vahdat [mailto:babak.vahdat@swissonline.ch]
> Gesendet: Freitag, 13. Juni 2014 13:15
> An: users@camel.apache.org
> Betreff: Re: AW: Problem with ETL example
> 
> No, the file consumer makes use of the memory based Idempotent Consumer
> here.
> No matter if you make use of noop true/false it will consume each file
> *only* once. Change noop to false or remove it and you will not see any
> change in the behavior of the second route. The first route will
> *always* consume *only* 2 times (row1.xml & row2.xml) no matter what
> the noop option value is.
> 
> To verify this one can also make use of hawtio, so just run:
> 
> ~/dev/workspace/camel/examples/camel-example-etl>mvn
> io.hawt:hawtio-maven-plugin:1.4.4:camel
> 
> Then refresh you browser and jump to the following URL:
> 
> http://localhost:8080/hawtio/#/jmx/attributes?tab=camel&nid=root-
> org.apache.camel-camel-routes
> 
> And you should see something like  Screen_Shot_2014-06-13_at_13.png
> <http://camel.465427.n5.nabble.com/file/n5752263/Screen_Shot_2014-06-
> 13_at_13.png>
> 
> As you see the the first route has the “Completed #” column count equal
> to 2 and the number doesn’t increase anymore, however the second route
> counts up continuously.
> 
> Now change consumerDelete option by the second route and run hawtio
> maven plugin again as above and verify the change in the behavior of
> the second route.
> 
> Babak
> 
> 
> Jan Matèrne (jhm) wrote
> > from("file:src/data?noop=true")
> >             .convertBodyTo(PersonDocument.class)
> >             .to("jpa:org.apache.camel.example.etl.CustomerEntity");
> >
> >         // the following will dump the database to files
> >
> >
> from("jpa:org.apache.camel.example.etl.CustomerEntity?consumeDelete=fa
> > lse&de
> > lay=3000&consumeLockEntity=false")
> >             .setHeader(Exchange.FILE_NAME,
> el("${in.body.userName}.xml"))
> >             .to("file:target/customers");
> >
> > I think it is because the first route doesnt move the processed files
> > to somewhere else ...
> > http://camel.apache.org/file2.html
> >
> >
> > Jan
> >
> >
> >> -----Ursprüngliche Nachricht-----
> >> Von: Babak Vahdat [mailto:
> 
> > babak.vahdat@
> 
> > ]
> >> Gesendet: Freitag, 13. Juni 2014 12:18
> >> An:
> 
> > users@.apache
> 
> >> Betreff: Re: Problem with ETL example
> >>
> >> Hi
> >>
> >> The reason for the same logs you see is because of the consumeDelete
> >> option being set to false:
> >>
> >> http://camel.apache.org/jpa.html#JPA-Options
> >>
> >> So that the JPA consumer polls the *same* 2 entities again and
> again.
> >> This is just for the demonstration purpose so that you can
> >> continuously see something is running.
> >>
> >> You can set that option to true or remove it (default is true) and
> >> check the change in the behavior here:
> >>
> >> https://github.com/apache/camel/blob/master/examples/camel-example-
> >> etl/src/main/java/org/apache/camel/example/etl/EtlRoutes.java#L36
> >>
> >> And if not already done, maybe you want to go through the
> >> documentation of this example as well:
> >>
> >> http://camel.apache.org/etl-example.html
> >>
> >> Babak
> >>
> >> Licia wrote
> >> > Hi everyone,
> >> >
> >> > I'm all new to Camel and after getting documentation, I went
> >> > directly to the ETL example because it really looks like what I'm
> >> > trying to
> >> do.
> >> >
> >> > So I just did mvn compile, which worked (everything did in the
> >> > directory where the POM is).
> >> > And then I tried mvn camel:run and I get a problem : it looks like
> >> the
> >> > same message get displayed over and over again.
> >> >
> >> >
> >> >
> >> >
> >> >
> >> > "
> >> *
> >> > 2014-06-13 11:07:52,094 [.CustomerEntity] INFO  Tracer
> >> > -
> >> >  ID-ITEM-93866-58463-1402650414339-0-78 >>> (route2)
> >> > from(jpa://org.apache.camel
> >> >
> >>
> .example.etl.CustomerEntity?consumeDelete=false&consumeLockEntity=fal
> >> s
> >> > e&delay=30
> >> > 00) --> setHeader[CamelFileName] <<< Pattern:InOnly,
> >> > Headers:{CamelEntityManager
> >> > =org.apache.openjpa.persistence.EntityManagerImpl@5d84d74b,
> >> > breadcrumbId=ID-ITEM
> >> > -93866-58463-1402650414339-0-77},
> >> > BodyType:org.apache.camel.example.etl.Customer
> >> > Entity, Body:<?xml version="1.0" encoding="UTF-8"
> >> > standalone="yes"?>
> >> >
> > <customer id="1">
> >> >
> >> >
> > <userName>
> >> > james
> >> >
> > </userName>
> >> >
> >> >
> > <firstName>
> >> > James
> >> >
> > </firstName>
> >> >
> >> >
> > <surname>
> >> > Strachan
> >> >
> > </surname>
> >> >
> >> >
> > <city>
> >> > London
> >> >
> > </city>
> >> >
> > </customer>
> >> *
> >> >
> >> > 2014-06-13 11:07:52,096 [.CustomerEntity] INFO  Tracer
> >> > -
> >> >  ID-ITEM-93866-58463-1402650414339-0-78 >>> (route2)
> >> > setHeader[CamelFileName] --
> >> >> file://target/customers <<< Pattern:InOnly,
> >> >> Headers:{breadcrumbId=ID-ITEM-9386
> >> > 6-58463-1402650414339-0-77, CamelFileName=james.xml,
> >> > CamelEntityManager=org.apac
> >> > he.openjpa.persistence.EntityManagerImpl@5d84d74b},
> >> > BodyType:org.apache.camel.ex
> >> > ample.etl.CustomerEntity, Body:<?xml version="1.0" encoding="UTF-
> 8"
> >> > standalone="
> >> > yes"?>
> >> >
> > <customer id="1">
> >> >
> >> >
> > <userName>
> >> > james
> >> >
> > </userName>
> >> >
> >> >
> > <firstName>
> >> > James
> >> >
> > </firstName>
> >> >
> >> >
> > <surname>
> >> > Strachan
> >> >
> > </surname>
> >> >
> >> >
> > <city>
> >> > London
> >> >
> > </city>
> >> >
> > </customer>
> >> > 2014-06-13 11:07:52,136 [.CustomerEntity] INFO  Tracer
> >> > -
> >> >  ID-ITEM-93866-58463-1402650414339-0-80 >>> (route2)
> >> > from(jpa://org.apache.camel
> >> >
> >>
> .example.etl.CustomerEntity?consumeDelete=false&consumeLockEntity=fal
> >> s
> >> > e&delay=30
> >> > 00) --> setHeader[CamelFileName] <<< Pattern:InOnly,
> >> > Headers:{CamelEntityManager
> >> > =org.apache.openjpa.persistence.EntityManagerImpl@5d84d74b,
> >> > breadcrumbId=ID-ITEM
> >> > -93866-58463-1402650414339-0-79},
> >> > BodyType:org.apache.camel.example.etl.Customer
> >> > Entity, Body:<?xml version="1.0" encoding="UTF-8"
> >> > standalone="yes"?>
> >> >
> > <customer id="2">
> >> >
> >> >
> > <userName>
> >> > hiram
> >> >
> > </userName>
> >> >
> >> >
> > <firstName>
> >> > Hiram
> >> >
> > </firstName>
> >> >
> >> >
> > <surname>
> >> > Chirino
> >> >
> > </surname>
> >> >
> >> >
> > <city>
> >> > Tampa
> >> >
> > </city>
> >> >
> > </customer>
> >> > 2014-06-13 11:07:52,137 [.CustomerEntity] INFO  Tracer
> >> > -
> >> >  ID-ITEM-93866-58463-1402650414339-0-80 >>> (route2)
> >> > setHeader[CamelFileName] --
> >> >> file://target/customers <<< Pattern:InOnly,
> >> >> Headers:{CamelEntityManager=org.ap
> >> > ache.openjpa.persistence.EntityManagerImpl@5d84d74b,
> >> > breadcrumbId=ID-ITEM-93866-
> >> > 58463-1402650414339-0-79, CamelFileName=hiram.xml},
> >> > BodyType:org.apache.camel.ex ample.etl.CustomerEntity, Body:<?xml
> >> > version="1.0" encoding="UTF-8"
> >> > standalone="
> >> > yes"?>
> >> >
> > <customer id="2">
> >> >
> >> >
> > <userName>
> >> > hiram
> >> >
> > </userName>
> >> >
> >> >
> > <firstName>
> >> > Hiram
> >> >
> > </firstName>
> >> >
> >> >
> > <surname>
> >> > Chirino
> >> >
> > </surname>
> >> >
> >> >
> > <city>
> >> > Tampa
> >> >
> > </city>
> >> >
> > </customer>
> >> > 60513  camel  TRACE  [Camel (camel) thread #1 -
> >> > jpa://org.apache.camel.example.e tl.CustomerEntity]
> >> > openjpa.jdbc.SQL
> >> -
> >> >
> > <t 53213167, conn 227180630>
> >   executing pre pstmnt 882159474 SELECT
> >> > t0.id, t0.city, t0.firstName, t0.phone, t0.street, t0.surname,
> >> >         t0.userName, t0.zip
> >> >     FROM customer t0
> >> >
> >> > 60514  camel  TRACE  [Camel (camel) thread #1 -
> >> > jpa://org.apache.camel.example.e tl.CustomerEntity]
> >> > openjpa.jdbc.SQL
> >> -
> >> >
> > <t 53213167, conn 227180630>
> >   [0 ms] spent
> >> *
> >> > 2014-06-13 11:07:55,165 [.CustomerEntity] INFO  Tracer
> >> > -
> >> >  ID-ITEM-93866-58463-1402650414339-0-82 >>> (route2)
> >> > from(jpa://org.apache.camel
> >> >
> >>
> .example.etl.CustomerEntity?consumeDelete=false&consumeLockEntity=fal
> >> s
> >> > e&delay=30
> >> > 00) --> setHeader[CamelFileName] <<< Pattern:InOnly,
> >> > Headers:{breadcrumbId=ID-IT EM-93866-58463-1402650414339-0-81,
> >> > CamelEntityManager=org.apache.openjpa.persist
> >> > ence.EntityManagerImpl@5d84d74b},
> >> > BodyType:org.apache.camel.example.etl.Customer
> >> > Entity, Body:<?xml version="1.0" encoding="UTF-8"
> >> > standalone="yes"?>
> >> >
> > <customer id="1">
> >> >
> >> >
> > <userName>
> >> > james
> >> >
> > </userName>
> >> >
> >> >
> > <firstName>
> >> > James
> >> >
> > </firstName>
> >> >
> >> >
> > <surname>
> >> > Strachan
> >> >
> > </surname>
> >> >
> >> >
> > <city>
> >> > London
> >> >
> > </city>
> >> >
> > </customer>
> >> *
> >> >
> >> > 2014-06-13 11:07:55,167 [.CustomerEntity] INFO  Tracer
> >> > -
> >> >  ID-ITEM-93866-58463-1402650414339-0-82 >>> (route2)
> >> > setHeader[CamelFileName] --
> >> >> file://target/customers <<< Pattern:InOnly,
> >> >> Headers:{breadcrumbId=ID-ITEM-9386
> >> > 6-58463-1402650414339-0-81, CamelFileName=james.xml,
> >> > CamelEntityManager=org.apac
> >> > he.openjpa.persistence.EntityManagerImpl@5d84d74b},
> >> > BodyType:org.apache.camel.ex
> >> > ample.etl.CustomerEntity, Body:<?xml version="1.0" encoding="UTF-
> 8"
> >> > standalone="
> >> > yes"?>
> >> >
> > <customer id="1">
> >> >
> >> >
> > <userName>
> >> > james
> >> >
> > </userName>
> >> >
> >> >
> > <firstName>
> >> > James
> >> >
> > </firstName>
> >> >
> >> >
> > <surname>
> >> > Strachan
> >> >
> > </surname>
> >> >
> >> >
> > <city>
> >> > London
> >> >
> > </city>
> >> >
> > </customer>
> >> > 2014-06-13 11:07:55,196 [.CustomerEntity] INFO  Tracer
> >> > -
> >> >  ID-ITEM-93866-58463-1402650414339-0-84 >>> (route2)
> >> > from(jpa://org.apache.camel
> >> >
> >>
> .example.etl.CustomerEntity?consumeDelete=false&consumeLockEntity=fal
> >> s
> >> > e&delay=30
> >> > 00) --> setHeader[CamelFileName] <<< Pattern:InOnly,
> >> > Headers:{breadcrumbId=ID-IT EM-93866-58463-1402650414339-0-83,
> >> > CamelEntityManager=org.apache.openjpa.persist
> >> > ence.EntityManagerImpl@5d84d74b},
> >> > BodyType:org.apache.camel.example.etl.Customer
> >> > Entity, Body:<?xml version="1.0" encoding="UTF-8"
> >> > standalone="yes"?>
> >> >
> > <customer id="2">
> >> >
> >> >
> > <userName>
> >> > hiram
> >> >
> > </userName>
> >> >
> >> >
> > <firstName>
> >> > Hiram
> >> >
> > </firstName>
> >> >
> >> >
> > <surname>
> >> > Chirino
> >> >
> > </surname>
> >> >
> >> >
> > <city>
> >> > Tampa
> >> >
> > </city>
> >> >
> > </customer>
> >> > 2014-06-13 11:07:55,198 [.CustomerEntity] INFO  Tracer
> >> > -
> >> >  ID-ITEM-93866-58463-1402650414339-0-84 >>> (route2)
> >> > setHeader[CamelFileName] --
> >> >> file://target/customers <<< Pattern:InOnly,
> >> >> Headers:{CamelEntityManager=org.ap
> >> > ache.openjpa.persistence.EntityManagerImpl@5d84d74b,
> >> > CamelFileName=hiram.xml, br
> >> > eadcrumbId=ID-ITEM-93866-58463-1402650414339-0-83},
> >> > BodyType:org.apache.camel.ex
> >> > ample.etl.CustomerEntity, Body:<?xml version="1.0" encoding="UTF-
> 8"
> >> > standalone="
> >> > yes"?>
> >> >
> > <customer id="2">
> >> >
> >> >
> > <userName>
> >> > hiram
> >> >
> > </userName>
> >> >
> >> >
> > <firstName>
> >> > Hiram
> >> >
> > </firstName>
> >> >
> >> >
> > <surname>
> >> > Chirino
> >> >
> > </surname>
> >> >
> >> >
> > <city>
> >> > Tampa
> >> >
> > </city>
> >> >
> > </customer>
> >> > "
> >> >
> >> >
> >> >
> >> >
> >> >
> >> >
> >> >
> >> >
> >> >
> >> >
> >> > The two bold parts are the exact same ones.
> >> >
> >> > Plus I'm really confused about what the example is supposed to do.
> >> > Isn't it supposed to create the java classes ? Or am I completely
> >> wrong ?
> >> > Else, isn't it supposed to create the persons in my database ? How
> >> > do I see if it has been done ?
> >> >
> >> > Thanks a lot for your answers.
> >>
> >>
> >>
> >>
> >>
> >> --
> >> View this message in context:
> >> http://camel.465427.n5.nabble.com/Problem-with-ETL-example-
> >> tp5752256p5752259.html
> >> Sent from the Camel - Users mailing list archive at Nabble.com.
> 
> 
> 
> 
> 
> --
> View this message in context:
> http://camel.465427.n5.nabble.com/Problem-with-ETL-example-
> tp5752256p5752263.html
> Sent from the Camel - Users mailing list archive at Nabble.com.


Mime
View raw message