cxf-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Daniel Kulp <dk...@apache.org>
Subject Re: File Descriptor Leaks
Date Sun, 12 Jul 2009 02:53:32 GMT

Just a "what is this" description for the cxf devs.....

George and I have been having a "twitter discussion" trying to figure out if 
it's CXF or something else that is causing all the file descriptors to get 
used up.   This is a stack trace from that.   

Dan

On Sat July 11 2009 10:44:22 pm George Reese wrote:
> This is an application that is constantly polling Amazon S3 using the
> jets3t Java wrapper.
>
> Attached is a stack trace of one of the recent failures.
>
> -George
>
> [WARN ] [12 Jul 2009 00:53:49.358] [RestUtils] [Retried connection 6
> times, which exceeds the maximum retry count of 5]
> [ERROR] [12 Jul 2009 00:53:49.358] [AmazonS3]
> [org.jets3t.service.S3ServiceException: S3 GET connection failed for
> '/']
> org.jets3t.service.S3ServiceException: S3 GET connection failed for '/'
> 	at
> org
> .jets3t
> .service
> .impl.rest.httpclient.RestS3Service.performRequest(RestS3Service.java:
> 516)
> 	at
> org
> .jets3t
> .service
> .impl.rest.httpclient.RestS3Service.performRestGet(RestS3Service.java:
> 752)
> 	at
> org
> .jets3t
> .service
> .impl
> .rest.httpclient.RestS3Service.listAllBucketsImpl(RestS3Service.java:
> 1041)
> 	at org.jets3t.service.S3Service.listAllBuckets(S3Service.java:1348)
> 	at com.enstratus.provider.aws.AmazonS3.listFiles(AmazonS3.java:406)
> 	at
> com
> .enstratus
> .provisioning.storage.StorageFactory.checkState(StorageFactory.java:55)
> 	at com.enstratus.monitor.CloudMonitor$1.call(CloudMonitor.java:277)
> 	at com.enstratus.monitor.CloudMonitor$1.call(CloudMonitor.java:276)
> 	at com.enstratus.util.Retry.retry(Retry.java:32)
> 	at com.enstratus.util.Retry.retry(Retry.java:19)
> 	at com.enstratus.monitor.CloudMonitor.checkStatus(CloudMonitor.java:
> 282)
> 	at com.enstratus.monitor.CloudMonitor.access$000(CloudMonitor.java:41)
> 	at com.enstratus.monitor.CloudMonitor$2$1.run(CloudMonitor.java:468)
> 	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:
> 441)
> 	at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> 	at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> 	at java.util.concurrent.ThreadPoolExecutor
> $Worker.runTask(ThreadPoolExecutor.java:886)
> 	at java.util.concurrent.ThreadPoolExecutor
> $Worker.run(ThreadPoolExecutor.java:908)
> 	at java.lang.Thread.run(Thread.java:619)
> Caused by: java.net.SocketException: Too many open files
> 	at java.net.Socket.createImpl(Socket.java:388)
> 	at java.net.Socket.getImpl(Socket.java:451)
> 	at java.net.Socket.bind(Socket.java:565)
> 	at
> com
> .sun
> .net.ssl.internal.ssl.BaseSSLSocketImpl.bind(BaseSSLSocketImpl.java:95)
> 	at com.sun.net.ssl.internal.ssl.SSLSocketImpl.bind(SSLSocketImpl.java:
> 45)
> 	at sun.reflect.GeneratedMethodAccessor31.invoke(Unknown Source)
> 	at
> sun
> .reflect
> .DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:
> 25)
> 	at java.lang.reflect.Method.invoke(Method.java:597)
> 	at
> org
> .apache
> .commons
> .httpclient
> .protocol
> .ReflectionSocketFactory.createSocket(ReflectionSocketFactory.java:139)
> 	at
> org
> .apache
> .commons
> .httpclient
> .protocol
> .SSLProtocolSocketFactory.createSocket(SSLProtocolSocketFactory.java:
> 130)
> 	at
> org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:
> 707)
> 	at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager
> $HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:
> 1361)
> 	at
> org
> .apache
> .commons
> .httpclient
> .HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387)
> 	at
> org
> .apache
> .commons
> .httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:
> 171)
> 	at
> org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:
> 397)
> 	at
> org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:
> 323)
> 	at
> org
> .jets3t
> .service
> .impl.rest.httpclient.RestS3Service.performRequest(RestS3Service.java:
> 342)
> 	... 18 more
> [WARN ] [12 Jul 2009 00:53:49.361] [Retry]
> [com.enstratus.provider.api.CloudException: S3 GET connection failed
> for '/']
> [WARN ] [12 Jul 2009 00:53:51.568] [RestUtils] [Retried connection 6
> times, which exceeds the maximum retry count of 5]
> [ERROR] [12 Jul 2009 00:53:51.568] [AmazonS3]
> [org.jets3t.service.S3ServiceException: S3 GET connection failed for
> '/']
> org.jets3t.service.S3ServiceException: S3 GET connection failed for '/'
> 	at
> org
> .jets3t
> .service
> .impl.rest.httpclient.RestS3Service.performRequest(RestS3Service.java:
> 516)
> 	at
> org
> .jets3t
> .service
> .impl.rest.httpclient.RestS3Service.performRestGet(RestS3Service.java:
> 752)
> 	at
> org
> .jets3t
> .service
> .impl
> .rest
> .httpclient.RestS3Service.getBucketLocationImpl(RestS3Service.java:1635)
> 	at org.jets3t.service.S3Service.getBucketLocation(S3Service.java:2316)
> 	at com.enstratus.provider.aws.AmazonS3.listFiles(AmazonS3.java:407)
> 	at
> com
> .enstratus
> .provisioning.storage.StorageFactory.checkState(StorageFactory.java:55)
> 	at com.enstratus.monitor.CloudMonitor$1.call(CloudMonitor.java:277)
> 	at com.enstratus.monitor.CloudMonitor$1.call(CloudMonitor.java:276)
> 	at com.enstratus.util.Retry.retry(Retry.java:32)
> 	at com.enstratus.util.Retry.retry(Retry.java:19)
> 	at com.enstratus.monitor.CloudMonitor.checkStatus(CloudMonitor.java:
> 282)
> 	at com.enstratus.monitor.CloudMonitor.access$000(CloudMonitor.java:41)
> 	at com.enstratus.monitor.CloudMonitor$2$1.run(CloudMonitor.java:468)
> 	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:
> 441)
> 	at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> 	at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> 	at java.util.concurrent.ThreadPoolExecutor
> $Worker.runTask(ThreadPoolExecutor.java:886)
> 	at java.util.concurrent.ThreadPoolExecutor
> $Worker.run(ThreadPoolExecutor.java:908)
> 	at java.lang.Thread.run(Thread.java:619)
> Caused by: java.net.SocketException: Too many open files
> 	at java.net.Socket.createImpl(Socket.java:388)
> 	at java.net.Socket.getImpl(Socket.java:451)
> 	at java.net.Socket.bind(Socket.java:565)
> 	at
> com
> .sun
> .net.ssl.internal.ssl.BaseSSLSocketImpl.bind(BaseSSLSocketImpl.java:95)
> 	at com.sun.net.ssl.internal.ssl.SSLSocketImpl.bind(SSLSocketImpl.java:
> 45)
> 	at sun.reflect.GeneratedMethodAccessor31.invoke(Unknown Source)
> 	at
> sun
> .reflect
> .DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:
> 25)
> 	at java.lang.reflect.Method.invoke(Method.java:597)
> 	at
> org
> .apache
> .commons
> .httpclient
> .protocol
> .ReflectionSocketFactory.createSocket(ReflectionSocketFactory.java:139)
> 	at
> org
> .apache
> .commons
> .httpclient
> .protocol
> .SSLProtocolSocketFactory.createSocket(SSLProtocolSocketFactory.java:
> 130)
> 	at
> org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:
> 707)
> 	at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager
> $HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:
> 1361)
> 	at
> org
> .apache
> .commons
> .httpclient
> .HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387)
> 	at
> org
> .apache
> .commons
> .httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:
> 171)
> 	at
> org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:
> 397)
> 	at
> org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:
> 323)
> 	at
> org
> .jets3t
> .service
> .impl.rest.httpclient.RestS3Service.performRequest(RestS3Service.java:
> 342)
> 	... 18 more
> [WARN ] [12 Jul 2009 00:53:51.571] [Retry]
> [com.enstratus.provider.api.CloudException: S3 GET connection failed
> for '/']
> [WARN ] [12 Jul 2009 00:53:55.587] [RestUtils] [Retried connection 6
> times, which exceeds the maximum retry count of 5]
> [ERROR] [12 Jul 2009 00:53:55.587] [AmazonS3]
> [org.jets3t.service.S3ServiceException: S3 GET connection failed for
> '/']
> org.jets3t.service.S3ServiceException: S3 GET connection failed for '/'
> 	at
> org
> .jets3t
> .service
> .impl.rest.httpclient.RestS3Service.performRequest(RestS3Service.java:
> 516)
> 	at
> org
> .jets3t
> .service
> .impl.rest.httpclient.RestS3Service.performRestGet(RestS3Service.java:
> 752)
> 	at
> org
> .jets3t
> .service
> .impl
> .rest.httpclient.RestS3Service.listAllBucketsImpl(RestS3Service.java:
> 1041)
> 	at org.jets3t.service.S3Service.listAllBuckets(S3Service.java:1348)
> 	at com.enstratus.provider.aws.AmazonS3.listFiles(AmazonS3.java:406)
> 	at
> com
> .enstratus
> .provisioning.storage.StorageFactory.checkState(StorageFactory.java:55)
> 	at com.enstratus.monitor.CloudMonitor$1.call(CloudMonitor.java:277)
> 	at com.enstratus.monitor.CloudMonitor$1.call(CloudMonitor.java:276)
> 	at com.enstratus.util.Retry.retry(Retry.java:32)
> 	at com.enstratus.util.Retry.retry(Retry.java:19)
> 	at com.enstratus.monitor.CloudMonitor.checkStatus(CloudMonitor.java:
> 282)
> 	at com.enstratus.monitor.CloudMonitor.access$000(CloudMonitor.java:41)
> 	at com.enstratus.monitor.CloudMonitor$2$1.run(CloudMonitor.java:468)
> 	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:
> 441)
> 	at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> 	at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> 	at java.util.concurrent.ThreadPoolExecutor
> $Worker.runTask(ThreadPoolExecutor.java:886)
> 	at java.util.concurrent.ThreadPoolExecutor
> $Worker.run(ThreadPoolExecutor.java:908)
> 	at java.lang.Thread.run(Thread.java:619)
> Caused by: java.net.SocketException: Too many open files
> 	at java.net.Socket.createImpl(Socket.java:388)
> 	at java.net.Socket.getImpl(Socket.java:451)
> 	at java.net.Socket.bind(Socket.java:565)
> 	at
> com
> .sun
> .net.ssl.internal.ssl.BaseSSLSocketImpl.bind(BaseSSLSocketImpl.java:95)
> 	at com.sun.net.ssl.internal.ssl.SSLSocketImpl.bind(SSLSocketImpl.java:
> 45)
> 	at
> com.sun.net.ssl.internal.ssl.SSLSocketImpl.<init>(SSLSocketImpl.java:
> 392)
> 	at
> com
> .sun
> .net
> .ssl
> .internal
> .ssl.SSLSocketFactoryImpl.createSocket(SSLSocketFactoryImpl.java:123)
> 	at
> org
> .apache
> .commons
> .httpclient
> .protocol
> .SSLProtocolSocketFactory.createSocket(SSLProtocolSocketFactory.java:82)
> 	at
> org.apache.commons.httpclient.protocol.ControllerThreadSocketFactory
> $1.doit(ControllerThreadSocketFactory.java:91)
> 	at
> org.apache.commons.httpclient.protocol.ControllerThreadSocketFactory
> $SocketTask.run(ControllerThreadSocketFactory.java:158)
> 	... 1 more
>
>
> --
> George Reese (george.reese@enstratus.com - Twitter: @GeorgeReese)
> Chief Technology Officer, enStratus Networks LLC
> p: +1.612.355.5272    m: +1.612.991.4446    f: +1.612.338.5041
> enStratus - Confidence in the Cloud - http://www.enstratus.com

-- 
Daniel Kulp
dkulp@apache.org
http://www.dankulp.com/blog

Mime
View raw message