camel-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Claus Ibsen <claus.ib...@gmail.com>
Subject Re: Sql Component StreamList Poll
Date Wed, 05 Apr 2017 07:41:04 GMT
Put the count down logic someplace else such as in your aggregation strategy

On Wed, Apr 5, 2017 at 6:50 AM, andre.gilliam@icg360.com
<andre.gilliam@icg360.com> wrote:
> I have a route similar to the following that uses the SQL Component:
>
> from("sql:" + QUERY + "?outputType=StreamList")
>                 .autoStartup(false)
>                 .routeId(getRouteId())
>                 .to("log:stream")
>                 .split(body(), new AggregationStrategy())
>                 .executorService(getThreadPool())
>                 .streaming()
>                 .parallelProcessing()
>                 .to("log:postProcess")
>                 .end()
>                 .process(new Processor() {
>             @Override
>             public void process(Exchange exchange) throws Exception {
>                 // stop Camel by signaling to the latch
>                 getCountDownLatch().countDown();
>             }
>         });
>
> I'm running into a problem and need direction. If the last streaming row is
> successfully processed by the route everything is fine. If the query returns
> 0 results or If the last streaming row has an exception while in the route
> it, the sql component will poll and never get into the route to hit the last
> process (which notifies the countDownLatch await and stops the process).
> This is the error that it produces when in that state:
>
> 00:33:37.150 [Camel (camel-1) thread #3 -
> sql://SELECT%20Id,%20QuoteId%20%20FROM%20%5Bdbo%5D.%5BIPSImporter%5D%20WHERE%20Status%20=%20'NEW']
> WARN  o.s.j.support.SQLErrorCodesFactory - Error while extracting database
> name - falling back to empty error codes
> org.springframework.jdbc.support.MetaDataAccessException: Error while
> extracting DatabaseMetaData; nested exception is java.sql.SQLException:
> Invalid state, the Connection object is closed.
>         at
> org.springframework.jdbc.support.JdbcUtils.extractDatabaseMetaData(JdbcUtils.java:342)
>         at
> org.springframework.jdbc.support.JdbcUtils.extractDatabaseMetaData(JdbcUtils.java:366)
>         at
> org.springframework.jdbc.support.SQLErrorCodesFactory.getErrorCodes(SQLErrorCodesFactory.java:212)
>         at
> org.springframework.jdbc.support.SQLErrorCodeSQLExceptionTranslator.setDataSource(SQLErrorCodeSQLExceptionTranslator.java:134)
>         at
> org.springframework.jdbc.support.SQLErrorCodeSQLExceptionTranslator.<init>(SQLErrorCodeSQLExceptionTranslator.java:97)
>         at
> org.springframework.jdbc.support.JdbcAccessor.getExceptionTranslator(JdbcAccessor.java:99)
>         at
> org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:649)
>         at
> org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:662)
>         at org.apache.camel.component.sql.SqlConsumer.poll(SqlConsumer.java:168)
>         at
> org.apache.camel.impl.ScheduledPollConsumer.doRun(ScheduledPollConsumer.java:175)
>         at
> org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:102)
>         at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
>         at java.util.concurrent.FutureTask.runAndReset(Unknown Source)
>         at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(Unknown
> Source)
>         at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown
> Source)
>         at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
>         at java.lang.Thread.run(Unknown Source)
> Caused by: java.sql.SQLException: Invalid state, the Connection object is
> closed.
>         at
> net.sourceforge.jtds.jdbc.JtdsConnection.checkOpen(JtdsConnection.java:1744)
>         at
> net.sourceforge.jtds.jdbc.JtdsConnection.getMetaData(JtdsConnection.java:2350)
>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>         at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
>         at java.lang.reflect.Method.invoke(Unknown Source)
>         at
> org.apache.tomcat.jdbc.pool.ProxyConnection.invoke(ProxyConnection.java:126)
>         at
> org.apache.tomcat.jdbc.pool.JdbcInterceptor.invoke(JdbcInterceptor.java:108)
>         at
> org.apache.tomcat.jdbc.pool.DisposableConnectionFacade.invoke(DisposableConnectionFacade.java:81)
>         at com.sun.proxy.$Proxy96.getMetaData(Unknown Source)
>         at
> org.springframework.jdbc.support.JdbcUtils.extractDatabaseMetaData(JdbcUtils.java:331)
>         ... 17 common frames omitted
> 00:33:37.152 [Camel (camel-1) thread #3 -
> sql://SELECT%20Id,%20QuoteId%20%20FROM%20%5Bdbo%5D.%5BIPSImporter%5D%20WHERE%20Status%20=%20'NEW']
> WARN  o.a.camel.component.sql.SqlConsumer - Consumer
> Consumer[sql://SELECT%20Id,%20QuoteId%20%20FROM%20%5Bdbo%5D.%5BIPSImporter%5D%20WHERE%20Status%20=%20'NEW'?noop=true&outputType=StreamList]
> failed polling endpoint:
> sql://SELECT%20Id,%20QuoteId%20%20FROM%20%5Bdbo%5D.%5BIPSImporter%5D%20WHERE%20Status%20=%20'NEW'?noop=true&outputType=StreamList.
> Will try again at next poll. Caused by:
> [org.springframework.jdbc.UncategorizedSQLException -
> PreparedStatementCallback; uncategorized SQLException for SQL [SELECT Id,
> QuoteId  FROM [dbo].[IPSImporter] WHERE Status = 'NEW']; SQL state [HY010];
> error code [0]; Invalid state, the Connection object is closed.; nested
> exception is java.sql.SQLException: Invalid state, the Connection object is
> closed.]
> org.springframework.jdbc.UncategorizedSQLException:
> PreparedStatementCallback; uncategorized SQLException for SQL [SELECT Id,
> QuoteId  FROM [dbo].[IPSImporter] WHERE Status = 'NEW']; SQL state [HY010];
> error code [0]; Invalid state, the Connection object is closed.; nested
> exception is java.sql.SQLException: Invalid state, the Connection object is
> closed.
>         at
> org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:84)
>         at
> org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:81)
>         at
> org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:81)
>         at
> org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:649)
>         at
> org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:662)
>         at org.apache.camel.component.sql.SqlConsumer.poll(SqlConsumer.java:168)
>         at
> org.apache.camel.impl.ScheduledPollConsumer.doRun(ScheduledPollConsumer.java:175)
>         at
> org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:102)
>         at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
>         at java.util.concurrent.FutureTask.runAndReset(Unknown Source)
>         at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(Unknown
> Source)
>         at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown
> Source)
>         at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
>         at java.lang.Thread.run(Unknown Source)
> Caused by: java.sql.SQLException: Invalid state, the Connection object is
> closed.
>         at
> net.sourceforge.jtds.jdbc.JtdsConnection.checkOpen(JtdsConnection.java:1744)
>         at
> net.sourceforge.jtds.jdbc.JtdsConnection.prepareStatement(JtdsConnection.java:2448)
>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>         at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
>         at java.lang.reflect.Method.invoke(Unknown Source)
>         at
> org.apache.tomcat.jdbc.pool.ProxyConnection.invoke(ProxyConnection.java:126)
>         at
> org.apache.tomcat.jdbc.pool.JdbcInterceptor.invoke(JdbcInterceptor.java:108)
>         at
> org.apache.tomcat.jdbc.pool.DisposableConnectionFacade.invoke(DisposableConnectionFacade.java:81)
>         at com.sun.proxy.$Proxy96.prepareStatement(Unknown Source)
>         at
> org.springframework.jdbc.core.JdbcTemplate$SimplePreparedStatementCreator.createPreparedStatement(JdbcTemplate.java:1521)
>         at
> org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:627)
>         ... 11 common frames omitted
> 00:33:37.653 [Camel (camel-1) thread #3 -
> sql://SELECT%20Id,%20QuoteId%20%20FROM%20%5Bdbo%5D.%5BIPSImporter%5D%20WHERE%20Status%20=%20'NEW']
> WARN  o.a.camel.component.sql.SqlConsumer - Consumer
> Consumer[sql://SELECT%20Id,%20QuoteId%20%20FROM%20%5Bdbo%5D.%5BIPSImporter%5D%20WHERE%20Status%20=%20'NEW'?noop=true&outputType=StreamList]
> failed polling endpoint:
> sql://SELECT%20Id,%20QuoteId%20%20FROM%20%5Bdbo%5D.%5BIPSImporter%5D%20WHERE%20Status%20=%20'NEW'?noop=true&outputType=StreamList.
> Will try again at next poll. Caused by:
> [org.springframework.jdbc.UncategorizedSQLException -
> PreparedStatementCallback; uncategorized SQLException for SQL [SELECT Id,
> QuoteId  FROM [dbo].[IPSImporter] WHERE Status = 'NEW']; SQL state [HY010];
> error code [0]; Invalid state, the Connection object is closed.; nested
> exception is java.sql.SQLException: Invalid state, the Connection object is
> closed.]
> org.springframework.jdbc.UncategorizedSQLException:
> PreparedStatementCallback; uncategorized SQLException for SQL [SELECT Id,
> QuoteId  FROM [dbo].[IPSImporter] WHERE Status = 'NEW']; SQL state [HY010];
> error code [0]; Invalid state, the Connection object is closed.; nested
> exception is java.sql.SQLException: Invalid state, the Connection object is
> closed.
>         at
> org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:84)
>         at
> org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:81)
>         at
> org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:81)
>         at
> org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:649)
>         at
> org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:662)
>         at org.apache.camel.component.sql.SqlConsumer.poll(SqlConsumer.java:168)
>         at
> org.apache.camel.impl.ScheduledPollConsumer.doRun(ScheduledPollConsumer.java:175)
>         at
> org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:102)
>         at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
>         at java.util.concurrent.FutureTask.runAndReset(Unknown Source)
>         at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(Unknown
> Source)
>         at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown
> Source)
>         at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
>         at java.lang.Thread.run(Unknown Source)
>
>
>
> --
> View this message in context: http://camel.465427.n5.nabble.com/Sql-Component-StreamList-Poll-tp5796939.html
> Sent from the Camel - Users mailing list archive at Nabble.com.



-- 
Claus Ibsen
-----------------
http://davsclaus.com @davsclaus
Camel in Action 2: https://www.manning.com/ibsen2

Mime
View raw message