db-derby-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Bryan Pendleton (JIRA)" <derby-...@db.apache.org>
Subject [jira] Updated: (DERBY-1472) Avoid expanding object to memory before sending it to server.
Date Fri, 07 Jul 2006 01:37:30 GMT
     [ http://issues.apache.org/jira/browse/DERBY-1472?page=all ]

Bryan Pendleton updated DERBY-1472:
-----------------------------------

    Attachment: test.java

I do not know the answer to your question. I experimented with sending BLOB columns from a
network client program (see attached 'test.java'), and when I ran the program I observed the
following:

1) On the client side, the BLOB does not seem to be materialized into memory when the input
stream is used. I was able to run the client using a very small amount of memory (appx 10
Mb) even when I sent very large files as BLOB.
2) On the server side, the BLOB does seem to be materialized into memory, possibly multiple
times.

Using this test program, I was able to send a 235 Megabyte file as a BLOB from the network
client to the server. The Network Client memory never got higher than 10 Mb. However, I had
to give the server a very large memory setting (-Xmx1024m) in order to load the 235 Megabyte
BLOB into the table.

I also tried sending a 600 Megabyte file as a BLOB from the client to the server. Again, the
client memory usage was very small, but the server memory usage was extreme. I was not able
to set the memory setting high enough to be successful; even when I ran the server with -Xmx1536m,
which is as high as I could get on my machine, the server still got OutOfMemoryError.

I also stepped through the code in org.apache.derby.client.net.Request's writeScalarStream()
method, and that code appears to be written very carefully such that it does NOT materialize
the entire BLOB on the client side, but rather reads the stream in 32K chunks and writes them
directly out to the network.

So, my conclusion is that I *cannot* reproduce DERBY-550 and I am *not* able to verify a memory
usage problem on the client side; only on the server side do I observe a problem.

I hope this is helpful to you.


> Avoid expanding object to memory before sending it to server.
> -------------------------------------------------------------
>
>          Key: DERBY-1472
>          URL: http://issues.apache.org/jira/browse/DERBY-1472
>      Project: Derby
>         Type: Improvement

>   Components: Network Client
>     Reporter: Tomohito Nakayama
>     Assignee: Tomohito Nakayama
>  Attachments: test.java
>
> Object , such as blob/clob, is expanded to memory before sending it to server.
> This behavior easily causes OutOfMemoryError, and it is desirable to remove this process
of expanding .

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
   http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
   http://www.atlassian.com/software/jira


Mime
View raw message