camel-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Shing Hing Man <mat...@yahoo.com.INVALID>
Subject Re: Timeout Exception in S3 download
Date Mon, 12 Jan 2015 21:07:57 GMT
I've  added the delay and maxMessagePerPoll, as you suggested,  to the path, but I still
get the same timeout exception.
Shing 
 

     On Monday, 12 January 2015, 2:10, Willem Jiang <willem.jiang@gmail.com> wrote:
   

 
Can you increase the delay option and maxMessagePerPoll to see if the error is still there?

from("aws-s3://MyBucket?amazonS3Client=#client&delay=5000&maxMessagesPerPoll=5")
.to("mock:result”);

--  
Willem Jiang

Red Hat, Inc.
Web: http://www.redhat.com
Blog: http://willemjiang.blogspot.com (English)
http://jnn.iteye.com (Chinese)
Twitter: willemjiang  
Weibo: 姜宁willem



On January 10, 2015 at 10:44:44 PM, Shing Hing Man (matmsh@yahoo.com.invalid) wrote:
>  
> I am using Apache Camel 2.14.1. I have set up the following S3 route to download from
a S3  
> Bucket.
> String bucketName = "myBucket"; String prefix = "out"; String awsURLTemplate="aws-s3://%s?accessKey=%s&secretKey=RAW(%s)®ion=eu-west-1&prefix=%s&deleteAfterRead=false"; 

> final String awsURL = String.format(awsURLTemplate,bucketName, accessKey,secretKey,prefix); 

> DefaultCamelContext context = new DefaultCamelContext(); context.addRoutes(new  
> RouteBuilder() { public void configure() { // Only download {myBucket}/out/**.txt}. 

> from(awsURL).filter().simple("${header.camelawss3key} regex '.*\\.txt'") .setHeader(Exchange.FILE_NAME, 

> simple("${header.camelawss3key}")).to("file:camel/in"); } }); context.start();  
> Thread.sleep(600000);
> When the above is run, all the txt files in S3 myBucket/out are downloaded. But about
1  
> minute later (without adding any files to the bucket), I get the following timeout exception.[ead 

> #0 - aws-s3://camel-lombok] AmazonHttpClient INFO Unable to execute HTTP request:  
> Timeout waiting for connection from poolorg.apache.http.conn.ConnectionPoolTimeoutException: 

> Timeout waiting for connection from pool at org.apache.http.impl.conn.PoolingClientConnectionManager.leaseConnection(PoolingClientConnectionManager.java:226)[httpclient-4.3.4.jar:4.3.4] 

> at org.apache.http.impl.conn.PoolingClientConnectionManager$1.getConnection(PoolingClientConnectionManager.java:195)[httpclient-4.3.4.jar:4.3.4] 

> at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)[:1.7.0_40] 

> at java.lang.reflect.Method.invoke(Method.java:606)[:1.7.0_40] at com.amazonaws.http.conn.ClientConnectionRequestFactory$Handler.invoke(ClientConnectionRequestFactory.java:70)[aws-java-sdk-1.8.9.1.jar:] 

> at com.amazonaws.http.conn.$Proxy13.getConnection(Unknown Source)[:] at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:423)[httpclient-4.3.4.jar:4.3.4] 

> at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:863)[httpclient-4.3.4.jar:4.3.4] 

> at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)[httpclient-4.3.4.jar:4.3.4] 

> at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:57)[httpclient-4.3.4.jar:4.3.4] 

> at com.amazonaws.http.AmazonHttpClient.executeHelper(AmazonHttpClient.java:464)[aws-java-sdk-1.8.9.1.jar:] 

> at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:273)[aws-java-sdk-1.8.9.1.jar:] 

> at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:3660)[aws-java-sdk-1.8.9.1.jar:] 

> at com.amazonaws.services.s3.AmazonS3Client.getObject(AmazonS3Client.java:1133)[aws-java-sdk-1.8.9.1.jar:] 

> at com.amazonaws.services.s3.AmazonS3Client.getObject(AmazonS3Client.java:1005)[aws-java-sdk-1.8.9.1.jar:] 

> at org.apache.camel.component.aws.s3.S3Consumer.createExchanges(S3Consumer.java:104)[camel-aws-2.14.1.jar:2.14.1] 

> at org.apache.camel.component.aws.s3.S3Consumer.poll(S3Consumer.java:85)[camel-aws-2.14.1.jar:2.14.1] 

> at org.apache.camel.impl.ScheduledPollConsumer.doRun(ScheduledPollConsumer.java:187)[camel-core-2.14.1.jar:2.14.1] 

> at org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:114)[camel-core-2.14.1.jar:2.14.1] 

> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)[:1.7.0_40] 

> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)[:1.7.0_40]  
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)[:1.7.0_40] 

> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)[:1.7.0_40] 

> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)[:1.7.0_40] 

> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)[:1.7.0_40] 

> at java.lang.Thread.run(Thread.java:724)[:1.7.0_40]
> Thanks in advance for any assistance ! Shing
>  
>  


   
Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message