db-derby-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Raymond Kroeker" <raym...@raykroeker.com>
Subject Re: Network Client/Server BLOB
Date Fri, 30 Mar 2007 22:38:39 GMT
Error Description:
  I was running my test harness code to create a new user.  I had a number
of poorly coded sql statements causing exceptions, and eventually i came to
a point where the derby database server gave up.

  The only LOB work I was doing was retreiving user vcard info (CLOB) and
storing/retreiving application event info (CLOB) both of which are small in
size (< 1KB).

Stack Trace:
80{770}), Failed Statement is: select P.PRODUCT_ID,P.PRODUCT_NAME from
TPSD_PRODUCT P where P.PRODUCT_NAME=?
java.lang.OutOfMemoryError: Java heap space
        at java.io.ByteArrayOutputStream.<init>(ByteArrayOutputStream.java
:60)
        at org.apache.derby.iapi.services.io.AccessibleByteArrayOutputStream.<init>(Unknown
Source)
        at org.apache.derby.iapi.services.classfile.ClassFormatOutput.<init>(Unknown
Source)
        at org.apache.derby.iapi.services.classfile.ClassFormatOutput.<init>(Unknown
Source)
        at org.apache.derby.impl.services.bytecode.CodeChunk.<init>(Unknown
Source)
        at
org.apache.derby.impl.services.bytecode.BCClass.newFieldWithAccessors(Unknown
Source)
        at
org.apache.derby.impl.sql.compile.ActivationClassBuilder.finishExecuteMethod(Unknown
Source)
        at org.apache.derby.impl.sql.compile.StatementNode.generate(Unknown
Source)
        at org.apache.derby.impl.sql.GenericStatement.prepMinion(Unknown
Source)
        at org.apache.derby.impl.sql.GenericStatement.prepare(Unknown
Source)
        at
org.apache.derby.impl.sql.conn.GenericLanguageConnectionContext.prepareInternalStatement(Unknown
Source)
        at org.apache.derby.impl.jdbc.EmbedPreparedStatement.<init>(Unknown
Source)
        at org.apache.derby.impl.jdbc.EmbedPreparedStatement20.<init>(Unknown
Source)
        at org.apache.derby.impl.jdbc.EmbedPreparedStatement30.<init>(Unknown
Source)
        at org.apache.derby.impl.jdbc.EmbedPreparedStatement40.<init>(Unknown
Source)
        at org.apache.derby.jdbc.Driver40.newEmbedPreparedStatement(Unknown
Source)
        at org.apache.derby.impl.jdbc.EmbedConnection.prepareStatement(Unknown
Source)
        at org.apache.derby.impl.jdbc.EmbedConnection.prepareStatement(Unknown
Source)
        at org.apache.derby.impl.drda.DRDAStatement.prepareStatementJDBC3(Unknown
Source)
        at org.apache.derby.impl.drda.DRDAStatement.prepare(Unknown Source)
        at org.apache.derby.impl.drda.DRDAStatement.explicitPrepare(Unknown
Source)
        at org.apache.derby.impl.drda.DRDAConnThread.parsePRPSQLSTT(Unknown
Source)
        at org.apache.derby.impl.drda.DRDAConnThread.processCommands(Unknown
Source)
        at org.apache.derby.impl.drda.DRDAConnThread.run(Unknown Source)
Cleanup action completed


On 3/28/07, Kristian Waagan <Kristian.Waagan@sun.com> wrote:
>
> Raymond Kroeker skrev:
> > I did run in Java 6 and have a stack, but the stack is without debug
> > info.  The derby.log is also kinda big so extracting the OOME will
> > require a script and a little time.
> >
> > My confusion stems from the fact that the OOME manifests as a slow
> > leak as opposed to direct implementation flaw in the server.  I can
> > work fine for a while then run into the error as opposed to hitting
> > the error consistently with every LOB transaction.
> >
> > I'll reply again when I have extracted the error.
>
> Raymond,
>
> Thank you for taking the time to investigate this :)
>
> Depending on the what we figure out for this problem, it might be best
> to create a Jira issue to track it. We can get back to this later if
> necessary.
>
> If you have a repro, it would be perfect. If you don't, but have a brief
> description of the sequence leading to the OOME, I think we have the
> required tools in the test-framework to easily write one. If no-one else
> feels the itch, I can have a look at it.
>
>
> regards,
> --
> Kristian
>
> >
> > On 3/28/07, *Kristian Waagan* <Kristian.Waagan@sun.com
> > <mailto:Kristian.Waagan@sun.com>> wrote:
> >
> >     Raymond Kroeker wrote:
> >     > Hello Kristian,
> >     >
> >     >   Thank you for your reply.  In client/server mode the derby
> server
> >     > currently uses no jvm memory settings, however the derby client
> >     > application uses -Xms512m -Xmx1024m.
> >
> >     Okay. This is to be expected, as LOBs are currently materialized
> >     in the
> >     client when they are retrieved (and in some cases when they are
> >     sent to
> >     the server).
> >     The LOCATOR-work will improve the clients ability to handle large
> LOBs
> >     on the server.
> >
> >     >
> >     >   The same blob is being written to and retreived from the
> >     database in
> >     > both the client and server software.  I have had successfull
> >     attempts at
> >     > storing 5 x 100MB blobs then had an out of memory error within
> >     the derby
> >     > network server when retreiving a 12MB blob.
> >     >
> >     >   1.  My blobs vary in size.  Up to 100MB has worked, 12MB had a
> >     failure
> >     > in the network server.
> >     >   2.  I insert them using  ps.setBinaryStream(1, stream,
> >     length); and
> >     > retreive them using rs.getBlob("CONTENT").getBinaryStream().  Both
> >     > workflows are issued from the database client.
> >
> >     Using the setBinaryStream with a length argument is good, and should
> >     allow you to stream any Blob to the server.
> >
> >     >   3.  The heap size for the derby server is default, the heap
> >     size of
> >     > the derb client is -Xms512m -Xmx1024m.
> >     >
> >     >   I too have had no issue with respect to the embedded client,
> >     and my
> >     > out of memory error occured within the network server so I am
> >     confused a
> >     > little.
> >
> >     Did you run with Java SE 6 and get a stack trace when the OOME
> >     happened?
> >     If so, I'm sure it would be interesting to see where it happened
> >     for the
> >     people working on these issues. If you want stack traces with line
> >     numbers in them, you must download the debug-distribution (from the
> >     db.apache.org/derby <http://db.apache.org/derby> site).
> >
> >     The Derby network server is a component/layer on top of the embedded
> >     driver. It is possible that even though the embedded driver is
> >     capable
> >     of handling Blobs up to 2GB-1, the network server might not be.
> >
> >
> >
> >     thanks,
> >     --
> >     Kristian
> >
> >     >
> >     > Raymond
> >     >
> >     > On 3/27/07, *Kristian Waagan* < Kristian.Waagan@sun.com
> >     <mailto:Kristian.Waagan@sun.com>
> >     > <mailto:Kristian.Waagan@sun.com
> >     <mailto:Kristian.Waagan@sun.com>>> wrote:
> >     >
> >     >     Raymond Kroeker wrote:
> >     >      > I have encountered some entries in the mailing list
> >     suggesting that
> >     >      > large BLOB content is not supported when using the
> >     >      > network client/server.  By not supported I mean either
> >     the client
> >     >     or the
> >     >      > server run out of memory when the content is
> >     >      > sufficiently large.
> >     >      >
> >     >      > My own tests on version 10.2.2.0 <http://10.2.2.0>
> >     <http://10.2.2.0>
> >     >     < http://10.2.2.0> suggest that this is
> >     >      > not the case; however I wanted to know if anyone can
> >     provide a
> >     >      > confirmation one way or another of this.  Thanks.
> >     >
> >     >     Hello Raymond,
> >     >
> >     >      From your mail I understand that you are able to use
> >     sufficiently
> >     >     large
> >     >     BLOBs for you purpose with Derby 10.2.2.0 <http://10.2.2.0>
> >     <http://10.2.2.0 <http://10.2.2.0>> in
> >     >     client/server mode.
> >     >     Would you mind telling us a little more about your use of
> Derby?
> >     >
> >     >     Quite some work has been done to remove problems with LOB
> >     handling in
> >     >     Derby, and more work is being done as we speek (implementing
> >     >     LOCATOR-based handling of LOBs).
> >     >     That said, I'm not sure we are exactly where we want to be
> >     in Derby
> >     >     10.2.2.0 <http://10.2.2.0> <http://10.2.2.0 <http://10.2.2.0
> >>.
> >     >
> >     >     A few questions;
> >     >        1) How big are your BLOBs?
> >     >        2) Do you both insert and retrieve them from/on the client?
> >     >        3) What are your JVM settings? (heap size)
> >     >
> >     >
> >     >     As far as confirmation goes, I have tested embedded earlier
> >     with 2GB-1
> >     >     LOBs without out-of-memory problems. I am under the
> >     impression that
> >     >     stream data still has to be materialized on the client
> >     before it is sent
> >     >     to the server.
> >     >     Can anyone else confirm this?
> >     >     Will the LOCATOR-work fix the issue?
> >     >
> >     >
> >     >
> >     >     regards,
> >     >     --
> >     >     Kristian
> >     >
> >     >      >
> >     >      > --
> >     >      > ---------------------------------------------------------
> >     >      > Raymond Kroeker
> >     >
> >     >
> >     >
> >     >
> >     > --
> >     > ---------------------------------------------------------
> >     > Raymond Kroeker
> >
> >
> >
> >
> > --
> > ---------------------------------------------------------
> > Raymond Kroeker
>
>


-- 
---------------------------------------------------------
Raymond Kroeker

Mime
View raw message