db-derby-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Kristian Waagan <Kristian.Waa...@Sun.COM>
Subject Re: [jira] Commented: (DERBY-550) BLOB : java.lang.OutOfMemoryError with network JDBC driver (org.apache.derby.jdbc.ClientDriver)
Date Mon, 10 Jul 2006 14:53:00 GMT
Andreas Korneliussen wrote:
> Kathey Marsden wrote:
>> Tomohito Nakayama (JIRA) wrote:
>>
>>>    [ 
>>> http://issues.apache.org/jira/browse/DERBY-550?page=comments#action_12419714

>>> ]
>>> Tomohito Nakayama commented on DERBY-550:
>>> -----------------------------------------
>>>
>>> I intended to resolve this issue as "Cannot Reproduce" as next url.
>>> http://issues.apache.org/jira/browse/DERBY-1472#action_12419712
>>>
>>> However, I reconsidered.
>>>
>>> I won't resolve DERBY-550, "BLOB : java.lang.OutOfMemoryError with 
>>> network JDBC driver (org.apache.derby.jdbc.ClientDriver)",
>>> because it seems to be true that OutOfMemoryError happens "in server 
>>> side".
>>>
>>> Title of DERBY-550 does not say that Error happens in client side.
>>> Then, this issue is not mistaken completely.
>>>
>>> Description of the issue "writting all the file into memory (RAM) 
>>> before sending ",  is mistaken.
>>>
>>>  
>>>
>> Sorry for  my own propagation of confusion over  this issue.  When 
>> filing  DERBY-326  and seeing similar code in the client I suspected 
>> there might be streaming  issues  in the client as well.  I  then took 
>> DERBY-550 as confirmation of that  when I saw it filed against the 
>> client and described in this way.
>>
>> In addition to changing the component to Network Server, changing the  
>> DERBY-550 description as follows might clarify things sufficiently.
>> from
>>
>> "Using the org.apache.derby.jdbc.ClientDriver driver to access the
>> Derby database through network, the driver is writting all the file 
>> into memory (RAM) before sending
>> it to the database. "
>>
>> to more of a a description of the functional problem:
>>
>> "Using the org.apache.derby.jdbc.ClientDriver driver to access the
>> Derby database through network, sending a large amount of parameter 
>> data with setBinaryStream will cause Network Server  to throw an 
>> OutOfMemoryException.
>>
>> I am not sure if setCharacterStream is also at issue. If so it could 
>> be filed as a separate issue.
>>
> 
> I think additionally one could make an issue for the client:
> 
> "Using the org.apache.derby.jdbc.ClientDriver driver to access the Derby 
> database through network, receiving a large amount of resultset data
> with rs.next() may cause Network Client to go out of memory"
> 
> I think derby works fine in the "sending" end of the streams, however in 
> the receiving end, there are memory problems.

I can second that, as I had a look at the code for ResultSet when 
looking at another issue. For instance, the 
NetCursor.getBlobColumn-method seems to materialize the whole Blob:
(data is a byte array)

         if (data != null) {
             // data found
             // set data offset based on the presence of a null indicator
             if (!nullable_[index]) {
                 dataOffset = 0;
             } else {
                 dataOffset = 1;
             }

             blob = new Blob(data, agent, dataOffset);
         } else {

The contents of 'data' come from the copyEXTDTA-method in 
NetStatementReply.java.

     protected void copyEXTDTA(NetCursor netCursor) throws 
DisconnectException {
         try {
             parseLengthAndMatchCodePoint(CodePoint.EXTDTA);
             byte[] data = null;
             if (longValueForDecryption_ == null) {
                 data = (getData(null)).toByteArray();
             } else {
                 data = longValueForDecryption_;
                 dssLength_ = 0;
                 longValueForDecryption_ = null;
             }
             netCursor.extdtaData_.add(data);
         } catch (java.lang.OutOfMemoryError e) {
             agent_.accumulateChainBreakingReadExceptionAndThrow(new 
DisconnectException(agent_,
                 new 
ClientMessageId(SQLState.NET_LOB_DATA_TOO_LARGE_FOR_JVM),
                 e));
         }
     }



I know layerB streaming in DRDA is not yet implemented (DERBY-1471) for 
streaming from the client to the server (to be used for streams with 
unknown length).
Do we have any support for streaming between server and client, which 
could be used to implement LOB streaming?
Do we support streaming in the embedded driver?



-- 
Kristian

> 
> Andreas
> 


Mime
View raw message