Mailing-List: contact derby-dev-help@db.apache.org; run by ezmlm
Precedence: bulk
Reply-To: <derby-dev@db.apache.org>
Message-ID: <22835991.1152236250684.JavaMail.jira@brutus>
Date: Fri, 7 Jul 2006 01:37:30 +0000 (GMT+00:00)
From: "Bryan Pendleton (JIRA)" <derby-dev@db.apache.org>
To: derby-dev@db.apache.org
Subject: [jira] Updated: (DERBY-1472) Avoid expanding object to memory
 before sending it to server.
In-Reply-To: <2090505.1151731291667.JavaMail.jira@brutus>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit

     [ http://issues.apache.org/jira/browse/DERBY-1472?page=all ]

Bryan Pendleton updated DERBY-1472:
-----------------------------------

    Attachment: test.java

I do not know the answer to your question. I experimented with sending BLOB columns from a network client program (see attached 'test.java'), and when I ran the program I observed the following:

1) On the client side, the BLOB does not seem to be materialized into memory when the input stream is used. I was able to run the client using a very small amount of memory (appx 10 Mb) even when I sent very large files as BLOB.
2) On the server side, the BLOB does seem to be materialized into memory, possibly multiple times.

Using this test program, I was able to send a 235 Megabyte file as a BLOB from the network client to the server. The Network Client memory never got higher than 10 Mb. However, I had to give the server a very large memory setting (-Xmx1024m) in order to load the 235 Megabyte BLOB into the table.

I also tried sending a 600 Megabyte file as a BLOB from the client to the server. Again, the client memory usage was very small, but the server memory usage was extreme. I was not able to set the memory setting high enough to be successful; even when I ran the server with -Xmx1536m, which is as high as I could get on my machine, the server still got OutOfMemoryError.

I also stepped through the code in org.apache.derby.client.net.Request's writeScalarStream() method, and that code appears to be written very carefully such that it does NOT materialize the entire BLOB on the client side, but rather reads the stream in 32K chunks and writes them directly out to the network.

So, my conclusion is that I *cannot* reproduce DERBY-550 and I am *not* able to verify a memory usage problem on the client side; only on the server side do I observe a problem.

I hope this is helpful to you.


> Avoid expanding object to memory before sending it to server.
> -------------------------------------------------------------
>
>          Key: DERBY-1472
>          URL: http://issues.apache.org/jira/browse/DERBY-1472
>      Project: Derby
>         Type: Improvement

>   Components: Network Client
>     Reporter: Tomohito Nakayama
>     Assignee: Tomohito Nakayama
>  Attachments: test.java
>
> Object , such as blob/clob, is expanded to memory before sending it to server.
> This behavior easily causes OutOfMemoryError, and it is desirable to remove this process of expanding .

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
   http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
   http://www.atlassian.com/software/jira