commons-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bibs L <bibsotherem...@yahoo.com>
Subject RE: java.lang.OutOfMemoryError - IOUtils.copy - When file size larger than 7 MB
Date Thu, 26 Apr 2007 16:51:09 GMT
Hi, Thank you so much for your help!!!
   
  Turns out we are running JDK 1.4 not good enought to use the setChunkedStreamingMode method,
but I was able to fix the issue by increasing the JVM heap size to 512 MB.  Not sure what
kind of risks I may be introducing by allocating more memory there...
   
  Thanks!

"Artamonov, Juri" <jartamonov@fusionone.com> wrote:
  Hi!

This is how I fixed uploading big files in Cargo. Perhaps this is what
you are looking for.

// When trying to upload large amount of data the internal
connection buffer can become
// too large and exceed the heap size, leading to a
java.lang.OutOfMemoryError.
// This was fixed in JDK 1.5 by introducing a new
setChunkedStreamingMode() method.
// As Cargo should also work with JDK versions lesser than
1.5 we use reflection to call
// setChunkedStreamingMode(). If it fails, we assume we're
running an older version of
// the JDK. In that case the solution for the user is to
increase it's heap size.
// For reference, see the following discussions about this:
//
http://www.velocityreviews.com/forums/t149076-leaking-memory-when-writin
g-to-url.html
//
http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=5026745
//
http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4212479
try
{

connection.getClass().getMethod("setChunkedStreamingMode",
new Class[] {Integer.TYPE}).invoke(connection, new
Object[] {new Integer(0)});
}
catch (Exception e)
{
// We assume we're on an older JDK version, do nothing.
getLogger().debug("Not calling setChunkedStreamingMode()
method as JVM ["
+ System.getProperty("java.version") + "] doesn't
support it.",
getClass().getName());
}

Best regards,
Juri.

-----Original Message-----
From: Bibs L [mailto:bibsotheremail@yahoo.com] 
Sent: Thursday, April 26, 2007 2:01 AM
To: commons-dev@jakarta.apache.org; commons-user@jakarta.apache.org
Subject: java.lang.OutOfMemoryError - IOUtils.copy - When file size
larger than 7 MB

Hi,
I am stuck with a java.lang.OutOfMemoryError when I try to copy a file
of about 7 MB from a window server. I am using the
org.apache.commons.io.IOUtils, 
IOUtils.copy(fileIn, response.getOutputStream()) with Survlet response
to get the download diaglog to pop up.

This is working fine when I run it locally - (download file larger
than 7MB), but when I deploy the code to Shaed Environment- (download
the SAME file larger than 7MB), it is throwing
java.lang.OutOfMemoryError Exception. And evertime it throw the
OutOfMemoryError, it creates a heapdump.phd file and a
javacore File on the shared environemnt's profile
location. 

Does anyone know what I need to do to fix this issue? 

The only thing I could think of is that the memory is low on the
server, do we need to increase the JVM heap? What's the down side to
that? 
Is it something I am doing wring in my Code below? I could download
file size less than 7MB.

A few notes about our shared envirnment:
- We are running WebSphere 6.0 on Window server
- It is a Virtual Machine
- 4 GBs on server, but is split to 4 server instance, so we have about
1 GB
- The code below is executed on WebSphere, but the file that need to
be downloaded is on a different window server - shared server -
"//serverName/somedir/file.doc" 

I would greatly appreciate any help you could offer!!!!
THANKS!!!!!!!!!!!!!!!!!

Here's my code:

FileInputStream fileIn = null;
File file = null;
URL url = null;
URLConnection urlConn = null;
InputStream in = null;
PrintWriter pw = null; 
try{
file = new File("\\serverPath\download\documents\test.doc"); 

if(file.exists() == true){ 
LOG.debug("file exists..."); 

//generate URL for file 
url = file.toURL(); 
urlConn = url.openConnection(); 
in = url.openStream();

fileIn = new FileInputStream(file); 

response.setContentType(urlConn.getContentType()); 
response.setHeader("Content-Disposition","inline; filename=\"" +
file.getName() + "\"");

LOG.debug("urlConn.getContentType() = " + urlConn.getContentType());

long startTime = System.currentTimeMillis(); 

IOUtils.copy(fileIn, response.getOutputStream());
response.setStatus(response.SC_OK);
response.flushBuffer(); 

LOG.debug("Total Time Taken to download: "+(System.currentTimeMillis()
- startTime) + " ms"); renderFileFlag = true; 

}else{ 
LOG.debug("file not found..."); 
} 
} catch (FileNotFoundExceptione) { 
LOG.error("FileNotFoundExceptione: download - " + e.getMessage(), e);

} catch (IOExceptione) { 
LOG.error("IOException: download - " + e.getMessage(), e); 

} catch (Exception e) { 
LOG.error("Exception : download - " + e.getMessage(), e);

} finally { 
if (facesContext != null) facesContext = null; 
if (fileIn != null) fileIn.close(); 
if (file != null) file = null; 
if (url != null) url = null;
if (urlConn != null) urlConn = null;
if (in != null) in.close(); 
if (pw != null) pw.close();
} 


---------------------------------
Ahhh...imagining that irresistible "new car" smell?
Check out new cars at Yahoo! Autos. 


---------------------------------
Ahhh...imagining that irresistible "new car" smell?
Check outnew cars at Yahoo! Autos.

---------------------------------------------------------------------
To unsubscribe, e-mail: commons-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: commons-user-help@jakarta.apache.org



       
---------------------------------
Ahhh...imagining that irresistible "new car" smell?
 Check outnew cars at Yahoo! Autos.
Mime
  • Unnamed multipart/alternative (inline, 8-Bit, 0 bytes)
View raw message