db-derby-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Kristian Waagan <Kristian.Waa...@Sun.COM>
Subject Re: Network Client/Server BLOB
Date Wed, 28 Mar 2007 20:30:13 GMT
Raymond Kroeker skrev:
> I did run in Java 6 and have a stack, but the stack is without debug 
> info.  The derby.log is also kinda big so extracting the OOME will 
> require a script and a little time.
>
> My confusion stems from the fact that the OOME manifests as a slow 
> leak as opposed to direct implementation flaw in the server.  I can 
> work fine for a while then run into the error as opposed to hitting 
> the error consistently with every LOB transaction.
>
> I'll reply again when I have extracted the error.

Raymond,

Thank you for taking the time to investigate this :)

Depending on the what we figure out for this problem, it might be best 
to create a Jira issue to track it. We can get back to this later if 
necessary.

If you have a repro, it would be perfect. If you don't, but have a brief 
description of the sequence leading to the OOME, I think we have the 
required tools in the test-framework to easily write one. If no-one else 
feels the itch, I can have a look at it.


regards,
-- 
Kristian

>
> On 3/28/07, *Kristian Waagan* <Kristian.Waagan@sun.com 
> <mailto:Kristian.Waagan@sun.com>> wrote:
>
>     Raymond Kroeker wrote:
>     > Hello Kristian,
>     >
>     >   Thank you for your reply.  In client/server mode the derby server
>     > currently uses no jvm memory settings, however the derby client
>     > application uses -Xms512m -Xmx1024m.
>
>     Okay. This is to be expected, as LOBs are currently materialized
>     in the
>     client when they are retrieved (and in some cases when they are
>     sent to
>     the server).
>     The LOCATOR-work will improve the clients ability to handle large LOBs
>     on the server.
>
>     >
>     >   The same blob is being written to and retreived from the
>     database in
>     > both the client and server software.  I have had successfull
>     attempts at
>     > storing 5 x 100MB blobs then had an out of memory error within
>     the derby
>     > network server when retreiving a 12MB blob.
>     >
>     >   1.  My blobs vary in size.  Up to 100MB has worked, 12MB had a
>     failure
>     > in the network server.
>     >   2.  I insert them using  ps.setBinaryStream(1, stream,
>     length); and
>     > retreive them using rs.getBlob("CONTENT").getBinaryStream().  Both
>     > workflows are issued from the database client.
>
>     Using the setBinaryStream with a length argument is good, and should
>     allow you to stream any Blob to the server.
>
>     >   3.  The heap size for the derby server is default, the heap
>     size of
>     > the derb client is -Xms512m -Xmx1024m.
>     >
>     >   I too have had no issue with respect to the embedded client,
>     and my
>     > out of memory error occured within the network server so I am
>     confused a
>     > little.
>
>     Did you run with Java SE 6 and get a stack trace when the OOME
>     happened?
>     If so, I'm sure it would be interesting to see where it happened
>     for the
>     people working on these issues. If you want stack traces with line
>     numbers in them, you must download the debug-distribution (from the
>     db.apache.org/derby <http://db.apache.org/derby> site).
>
>     The Derby network server is a component/layer on top of the embedded
>     driver. It is possible that even though the embedded driver is
>     capable
>     of handling Blobs up to 2GB-1, the network server might not be.
>
>
>
>     thanks,
>     --
>     Kristian
>
>     >
>     > Raymond
>     >
>     > On 3/27/07, *Kristian Waagan* < Kristian.Waagan@sun.com
>     <mailto:Kristian.Waagan@sun.com>
>     > <mailto:Kristian.Waagan@sun.com
>     <mailto:Kristian.Waagan@sun.com>>> wrote:
>     >
>     >     Raymond Kroeker wrote:
>     >      > I have encountered some entries in the mailing list
>     suggesting that
>     >      > large BLOB content is not supported when using the
>     >      > network client/server.  By not supported I mean either
>     the client
>     >     or the
>     >      > server run out of memory when the content is
>     >      > sufficiently large.
>     >      >
>     >      > My own tests on version 10.2.2.0 <http://10.2.2.0>
>     <http://10.2.2.0>
>     >     < http://10.2.2.0> suggest that this is
>     >      > not the case; however I wanted to know if anyone can
>     provide a
>     >      > confirmation one way or another of this.  Thanks.
>     >
>     >     Hello Raymond,
>     >
>     >      From your mail I understand that you are able to use
>     sufficiently
>     >     large
>     >     BLOBs for you purpose with Derby 10.2.2.0 <http://10.2.2.0>
>     <http://10.2.2.0 <http://10.2.2.0>> in
>     >     client/server mode.
>     >     Would you mind telling us a little more about your use of Derby?
>     >
>     >     Quite some work has been done to remove problems with LOB
>     handling in
>     >     Derby, and more work is being done as we speek (implementing
>     >     LOCATOR-based handling of LOBs).
>     >     That said, I'm not sure we are exactly where we want to be
>     in Derby
>     >     10.2.2.0 <http://10.2.2.0> <http://10.2.2.0 <http://10.2.2.0>>.
>     >
>     >     A few questions;
>     >        1) How big are your BLOBs?
>     >        2) Do you both insert and retrieve them from/on the client?
>     >        3) What are your JVM settings? (heap size)
>     >
>     >
>     >     As far as confirmation goes, I have tested embedded earlier
>     with 2GB-1
>     >     LOBs without out-of-memory problems. I am under the
>     impression that
>     >     stream data still has to be materialized on the client
>     before it is sent
>     >     to the server.
>     >     Can anyone else confirm this?
>     >     Will the LOCATOR-work fix the issue?
>     >
>     >
>     >
>     >     regards,
>     >     --
>     >     Kristian
>     >
>     >      >
>     >      > --
>     >      > ---------------------------------------------------------
>     >      > Raymond Kroeker
>     >
>     >
>     >
>     >
>     > --
>     > ---------------------------------------------------------
>     > Raymond Kroeker
>
>
>
>
> -- 
> ---------------------------------------------------------
> Raymond Kroeker 


Mime
View raw message