commons-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From David Griffiths <dgriffi...@boats.com>
Subject [NET] java.net.SocketException: Too many open files
Date Wed, 22 Feb 2006 23:14:39 GMT
I've searched the web, and through the archives, looking for similar problems. The most common
cause seems to be too few file 
descriptors.

We seem to have lots of file descriptors:

cat /proc/sys/fs/file-max
406054

And not many are being used (this is with the code running):

cat /proc/sys/fs/file-nr
2730    0       406054

This happens in a long-running program (it processes images, and keeps going till there are
no more - there are usually a few
thousand in the queue waiting to be downloaded).

A group of images is retrieved via http (wget), resized (via image-magik-convert), and then
ftp'd up to other servers.

The exception always occurs in the ftp "storeFile" portion of the process.

The code is,

try
{
	File remoteFile = new File(strRemoteDirectory + f.getName());
	boolean bOneSucceeded = false;
	for (int iServerCount = 0; iServerCount < ftpConnections.length; iServerCount++)
         {
                 FTPClient ftpClient = ftpConnections[iServerCount];

                 if ( !ftpClient.changeWorkingDirectory(strRemoteDirectory) )
                 {
                 	m_sLogger.info("\t" + new java.util.Date() + " ListingExchangeImageProcessor:
run: could not change directories
(on ftp server) to " + strRemoteDirectory);
                         continue;
		}

                 fInputStream = new FileInputStream(f);
		ftpClient.setFileType(FTP.BINARY_FILE_TYPE);

		if ( ftpClient.storeFile(f.getName(), fInputStream) )
                 {
                 	bOneSucceeded = true;
		}

	}

         if ( !bOneSucceeded )
         {
         	m_sLogger.info("\t" + new java.util.Date() + " ListingExchangeImageProcessor: run:
unable to upload to a single ftp
server...");

		return;
	}
}
catch (Exception e)
{
	m_sLogger.info("\t" + new java.util.Date() + " ListingExchangeImageProcessor: run: exception
uploading file: " + e.toString());
         e.printStackTrace();
         return;
}
finally
{
	try
         {
         	fInputStream.close();
	}
         catch (Exception e) {}
}

The stack trace for the exception is,

java.net.SocketException: Too many open files
         at java.net.PlainSocketImpl.socketAccept(Native Method)
         at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:384)
         at java.net.ServerSocket.implAccept(ServerSocket.java:450)
         at java.net.ServerSocket.accept(ServerSocket.java:421)
         at org.apache.commons.net.ftp.FTPClient._openDataConnection_(FTPClient.java:502)
         at org.apache.commons.net.ftp.FTPClient.__storeFile(FTPClient.java:388)
         at org.apache.commons.net.ftp.FTPClient.storeFile(FTPClient.java:1388)
         at listingexchange.ListingExchangeImageProcessor.run(ListingExchangeImageProcessor.java:1143)
         at listingexchange.ListingExchangeImageProcessor.main(ListingExchangeImageProcessor.java:204)


Any insight would be helpful.

David

---------------------------------------------------------------------
To unsubscribe, e-mail: commons-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: commons-user-help@jakarta.apache.org


Mime
View raw message