Return-Path: X-Original-To: apmail-db-derby-user-archive@www.apache.org Delivered-To: apmail-db-derby-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 69531D164 for ; Wed, 5 Sep 2012 02:39:52 +0000 (UTC) Received: (qmail 3697 invoked by uid 500); 5 Sep 2012 02:39:51 -0000 Delivered-To: apmail-db-derby-user-archive@db.apache.org Received: (qmail 3650 invoked by uid 500); 5 Sep 2012 02:39:51 -0000 Mailing-List: contact derby-user-help@db.apache.org; run by ezmlm Precedence: bulk list-help: list-unsubscribe: List-Post: List-Id: Reply-To: "Derby Discussion" Delivered-To: mailing list derby-user@db.apache.org Received: (qmail 3639 invoked by uid 99); 5 Sep 2012 02:39:51 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 05 Sep 2012 02:39:51 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=FSL_RCVD_USER,HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of brett.wooldridge@gmail.com designates 209.85.223.172 as permitted sender) Received: from [209.85.223.172] (HELO mail-ie0-f172.google.com) (209.85.223.172) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 05 Sep 2012 02:39:43 +0000 Received: by ieak13 with SMTP id k13so40799iea.31 for ; Tue, 04 Sep 2012 19:39:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=SIkGa8Wb4kgiTd+7LI8MnmPd3Jhn/0CuDHRN05A7Pzs=; b=GpFYB+l1B9TfomiDP5wkivUvoz5NZW0Oe5/Z9K2FtirJa2M0CFwQser7M1vuSou3dE YpQns0IaQu2QsamACT35diXfxkNTyd1Ax2olqA93QjyjikNDScKCd07VYjGU8xJKeuPa pU8HvNk4dPbkqZIhTP/j3skIK65BzYKBS7PcO2flERHahYxjglLLCoSdg0V4MvmhN2Vw Jxq5M2LCH4SLYhi8AEr9+oHlIVBFktbGfThQ435MUrT4bwKGtXCGi/qDLArDYTRy1CiJ twuWHmYwHVxPRtJ59mDqJ5fbBF+DEwP2sxv13EC/P/mVSC0xS5F/BBQ6hWcGpUfN7ha7 RNSw== MIME-Version: 1.0 Received: by 10.50.16.202 with SMTP id i10mr16414671igd.19.1346812762758; Tue, 04 Sep 2012 19:39:22 -0700 (PDT) Received: by 10.231.24.79 with HTTP; Tue, 4 Sep 2012 19:39:22 -0700 (PDT) In-Reply-To: <000001cd8ad9$6f6cccb0$4e466610$@lichtasweb.com> References: <000001cd8ad9$6f6cccb0$4e466610$@lichtasweb.com> Date: Wed, 5 Sep 2012 11:39:22 +0900 Message-ID: Subject: Re: NPE when compressing large table From: Brett Wooldridge To: Derby Discussion Content-Type: multipart/alternative; boundary=f46d044303004d9dc904c8eb484b --f46d044303004d9dc904c8eb484b Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable This looks like a bug to me, I recommend you open one. You might also try SYSCS_UTIL.SYSCS_INPLACE_COMPRESS_TABLE. If you can copy the DB to your dev environment, reproduce the issue, and run a derby 10.8.2.2 built with line numbers, it would probably help the devs pinpoint the issue. Brett On Wed, Sep 5, 2012 at 5:11 AM, Erick Lichtas wrot= e: > Hi Everyone,**** > > ** ** > > I am having an issue with the SYSCS_UTIL.SYSCS_COMPRESS_TABLE operation o= n > a large table in Derby 10.8.2.2.**** > > ** ** > > Statement stmt =3D *null*;**** > > CallableStatement cs =3D *null*;**** > > *try* {**** > > String sql =3D "select schemaname, tablename from sys.sysschemas s= , " > **** > > + "sys.systables t where s.schemaid=3Dt.schemaid and > t.tabletype=3D'T'";**** > > stmt =3D con.createStatement();**** > > ResultSet rs =3D stmt.executeQuery(sql);**** > > cs =3D con.prepareCall("call SYSCS_UTIL.SYSCS_COMPRESS_TABLE(?, ?, > ?)");**** > > *while* (rs.next()) {**** > > String schema =3D rs.getString(1).trim();**** > > String table =3D rs.getString(2).trim();**** > > log("Compressing " + schema + "." + table + "...");**** > > cs.setString(1, schema);**** > > cs.setString(2, table);**** > > cs.setShort(3, (*short*) 1);**** > > cs.execute();**** > > }**** > > }**** > > *finally* {**** > > *if* (stmt !=3D *null*) {**** > > stmt.close();**** > > }**** > > *if* (cs !=3D *null*) {**** > > cs.close();**** > > }**** > > }**** > > ** ** > > The above code, successfully runs over 3 of 20 some tables, then hits the= 4 > th table with over 2 million records. After working on this table for > about 5 minutes, the process terminates with an SQLException wrapping a > NPE. **** > > This happened in a production instance and I have not yet set up a > reproduction in a development environment. I=92m wondering if anyone has= any > thoughts based on the stacktrace below?**** > > ** ** > > com.linoma.gaservices.upgrader.UpgradeException: Java exception: ': > java.lang.NullPointerException'.**** > > at > com.linoma.gaservices.upgrader.DefaultUpgrader.upgrade(DefaultUpgrader.ja= va:227) > **** > > at > com.linoma.gaservices.upgrader.UnixUpgrader.upgrade(UnixUpgrader.java:32)= * > *** > > at > com.linoma.gaservices.upgrader.UpgradeStarter.startUpgrade(UpgradeStarter= .java:25) > **** > > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native > Method)**** > > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java= :79) > **** > > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorI= mpl.java:43) > **** > > at java.lang.reflect.Method.invoke(Method.java:618)**** > > at my.app.upgrader.Startup.main(Startup.java:72)**** > > Caused by: java.sql.SQLException: Java exception: ': > java.lang.NullPointerException'.**** > > at > org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown > Source)**** > > at > org.apache.derby.impl.jdbc.Util.newEmbedSQLException(Unknown Source)**** > > at org.apache.derby.impl.jdbc.Util.javaException(Unknown > Source)**** > > at > org.apache.derby.impl.jdbc.TransactionResourceImpl.wrapInSQLException(Unk= nown > Source)**** > > at > org.apache.derby.impl.jdbc.TransactionResourceImpl.handleException(Unknow= n > Source)**** > > at > org.apache.derby.impl.jdbc.EmbedConnection.handleException(Unknown Source= ) > **** > > at > org.apache.derby.impl.jdbc.EmbedConnection.(Unknown Source)**** > > at > org.apache.derby.jdbc.Driver30.getNewEmbedConnection(Unknown Source)**** > > at org.apache.derby.jdbc.InternalDriver.connect(Unknown > Source)**** > > at org.apache.derby.jdbc.AutoloadedDriver.connect(Unknown > Source)**** > > at > java.sql.DriverManager.getConnection(DriverManager.java:572)**** > > at > java.sql.DriverManager.getConnection(DriverManager.java:218)**** > > at > my.app.upgrader.db.DerbyDatabaseUpgrader.closeDB(DerbyDatabaseUpgrader.ja= va:99) > **** > > at > my.app.upgrader.DefaultUpgrader.upgrade(DefaultUpgrader.java:223)**** > > ... 7 more **** > > Caused by: java.lang.NullPointerException**** > > at > org.apache.derby.impl.store.raw.data.BaseDataFileFactory.openContainer(Un= known > Source)**** > > at > org.apache.derby.impl.store.raw.data.BaseDataFileFactory.openContainer(Un= known > Source)**** > > at > org.apache.derby.impl.store.raw.xact.Xact.openContainer(Unknown Source)**= * > * > > at > org.apache.derby.impl.store.access.conglomerate.OpenConglomerate.init(Unk= nown > Source)**** > > at > org.apache.derby.impl.store.access.heap.Heap.open(Unknown Source)**** > > at > org.apache.derby.impl.store.access.RAMTransaction.openConglomerate(Unknow= n > Source)**** > > at > org.apache.derby.impl.store.access.RAMTransaction.openConglomerate(Unknow= n > Source)**** > > at > org.apache.derby.impl.sql.catalog.DataDictionaryImpl.getDescriptorViaInde= xMinion(Unknown > Source)**** > > at > org.apache.derby.impl.sql.catalog.DataDictionaryImpl.getDescriptorViaInde= x(Unknown > Source)**** > > at > org.apache.derby.impl.sql.catalog.DataDictionaryImpl.locateSchemaRow(Unkn= own > Source)**** > > at > org.apache.derby.impl.sql.catalog.DataDictionaryImpl.getSchemaDescriptor(= Unknown > Source)**** > > at > org.apache.derby.impl.sql.conn.GenericLanguageConnectionContext.initDefau= ltSchemaDescriptor(Unknown > Source)**** > > at > org.apache.derby.impl.sql.conn.GenericLanguageConnectionContext.initializ= e(Unknown > Source)**** > > at > org.apache.derby.impl.db.BasicDatabase.setupConnection(Unknown Source)***= * > > at > org.apache.derby.impl.jdbc.TransactionResourceImpl.startTransaction(Unkno= wn > Source)**** > > at > org.apache.derby.impl.jdbc.EmbedConnection.checkUserIsNotARole(Unknown > Source)**** > > at > org.apache.derby.impl.jdbc.EmbedConnection.checkUserCredentials(Unknown > Source)**** > > ... 15 more **** > > ** ** > > Regards,**** > > ** ** > > Erick**** > > ** ** > --f46d044303004d9dc904c8eb484b Content-Type: text/html; charset=windows-1252 Content-Transfer-Encoding: quoted-printable This looks like a bug to me, I recommend you open one. =A0You might also tr= y SYSCS_UTIL.SYSCS_INPLACE_COMPRESS_TABLE. =A0If you can copy the DB to you= r dev environment, reproduce the issue, and run a derby 10.8.2.2 built with line numbers, it would probably hel= p the devs pinpoint the issue.

Brett

On Wed, Sep 5, 2012 at 5:11 AM= , Erick Lichtas <elichtas@lichtasweb.com> wrote:

Hi Everyone,

=A0<= /p>

I am having an issue with the SYSCS_UTIL.SYSCS_CO= MPRESS_TABLE operation on a large table in Derby 10.8.2.2.

=A0

Statement stmt =3D null;

CallableStatement cs =3D null= ;

try {

=A0=A0=A0=A0=A0=A0 String sql =3D "sel= ect schemaname, tablename from sys.sysschemas s, "

=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 + <= /span>&= quot;sys.systables t where s.schemaid=3Dt.schemaid and t.tabletype=3D'T= '";

=A0=A0=A0=A0=A0=A0 stmt =3D con.createStatement();= =

=A0=A0=A0=A0=A0=A0 ResultSet rs =3D stmt.ex= ecuteQuery(sql);

=A0=A0=A0=A0=A0=A0 cs =3D con.prepareCall("call SYSC= S_UTIL.SYSCS_COMPRESS_TABLE(?, ?, ?)");

=A0=A0=A0=A0=A0=A0 while (rs.next()) {

=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 Str= ing schema =3D rs.getString(1).trim();

=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 Str= ing table =3D rs.getString(2).trim();

=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 log= ("Compressing " + schema + "." + table + "...");

=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 cs.= setString(1, schema);

=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 cs.= setString(2, table);

=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 cs.= setShort(3, (short) 1);

=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 cs.= execute();<= /u>

=A0=A0=A0=A0=A0=A0 }<= /span><= /span>

}

finally {

=A0=A0=A0=A0=A0=A0 if (stmt !=3D null) {

=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 stm= t.close();<= /u>

=A0=A0=A0=A0=A0=A0 }<= /span><= /span>

=A0=A0=A0=A0=A0=A0 if<= /b> (cs !=3D <= b>null<= /span>) {<= span style=3D"font-size:10.0pt;font-family:Consolas"><= /p>

=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 cs.= close();

=A0=A0=A0=A0=A0=A0 }<= /span><= /span>

}

=A0

The abov= e code, successfully runs over 3 of 20 some tables, then hits the 4th<= /sup> table with over 2 million records.=A0 After working on this table for= about 5 minutes, the process terminates with an SQLException wrapping a NP= E.=A0

This happened in a production instance and I have no= t yet set up a reproduction in a development environment.=A0 I=92m wonderin= g if anyone has any thoughts based on the stacktrace below?

=A0

com.lino= ma.gaservices.upgrader.UpgradeException: Java exception: ': java.lang.N= ullPointerException'.

=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 at com.linoma.gaservices.upgrader.Defa= ultUpgrader.upgrade(DefaultUpgrader.java:227)

=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 at com= .linoma.gaservices.upgrader.UnixUpgrader.upgrade(UnixUpgrader.java:32)

=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0 at com.linoma.gaservices.upgrader.UpgradeStarter.startUpgrade(Upgrad= eStarter.java:25)

=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 at sun= .reflect.NativeMethodAccessorImpl.invoke0(Native Method)

<= p class=3D"MsoNormal">=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 at sun.= reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:79)

=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 at sun= .reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.j= ava:43)

=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0 at java.lang.reflect.Method.invoke(Method.java:618)

=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 at my.= app.upgrader.Startup.main(Startup.java:72)

Caused by: java.sql.SQLException: Java exception: ': java.lang.= NullPointerException'.

=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 at org= .apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)=

=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0 at org.apache.derby.impl.jdbc.Util.newEmbedSQLException(Unknow= n Source)

=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 at org= .apache.derby.impl.jdbc.Util.javaException(Unknown Source)

=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 at or= g.apache.derby.impl.jdbc.TransactionResourceImpl.wrapInSQLException(Unknown= Source)

=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 at org= .apache.derby.impl.jdbc.TransactionResourceImpl.handleException(Unknown Sou= rce)

=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0 at org.apache.derby.impl.jdbc.EmbedConnection.handleExcepti= on(Unknown Source)

=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 at org= .apache.derby.impl.jdbc.EmbedConnection.<init>(Unknown Source)=

=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0 at org.apache.derby.jdbc.Driver30.getNewEmbedConnection(Unknown Sour= ce)

=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 at org= .apache.derby.jdbc.InternalDriver.connect(Unknown Source)

=

=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 at org= .apache.derby.jdbc.AutoloadedDriver.connect(Unknown Source)

=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 at jav= a.sql.DriverManager.getConnection(DriverManager.java:572)

=

=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 at jav= a.sql.DriverManager.getConnection(DriverManager.java:218)

=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 at my.= app.upgrader.db.DerbyDatabaseUpgrader.closeDB(DerbyDatabaseUpgrader.java:99= )

=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0 at my.app.upgrader.DefaultUpgrader.upgrade(DefaultUpgrader.jav= a:223)

=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 ... 7 = more=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0

Caused by: java.la= ng.NullPointerException

=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 at org.apache.derby.impl.store.raw.dat= a.BaseDataFileFactory.openContainer(Unknown Source)

=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 at org= .apache.derby.impl.store.raw.data.BaseDataFileFactory.openContainer(Unknown= Source)

=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0 at org.apache.derby.impl.store.raw.xact.Xact.openCont= ainer(Unknown Source)

=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 at org= .apache.derby.impl.store.access.conglomerate.OpenConglomerate.init(Unknown = Source)

=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0 at org.apache.derby.impl.store.access.heap.Heap.open(Unk= nown Source)

=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 at org= .apache.derby.impl.store.access.RAMTransaction.openConglomerate(Unknown Sou= rce)

=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0 at org.apache.derby.impl.store.access.RAMTransaction.openCo= nglomerate(Unknown Source)

=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 at org= .apache.derby.impl.sql.catalog.DataDictionaryImpl.getDescriptorViaIndexMini= on(Unknown Source)

=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 at org.apache.derby.impl.sql.catalog.DataDic= tionaryImpl.getDescriptorViaIndex(Unknown Source)

=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 at org= .apache.derby.impl.sql.catalog.DataDictionaryImpl.locateSchemaRow(Unknown S= ource)

=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0 at org.apache.derby.impl.sql.catalog.DataDictionaryImpl.= getSchemaDescriptor(Unknown Source)

=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 at org= .apache.derby.impl.sql.conn.GenericLanguageConnectionContext.initDefaultSch= emaDescriptor(Unknown Source)

=A0= =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 at org.apache.derby.impl.sql.con= n.GenericLanguageConnectionContext.initialize(Unknown Source)=

=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 at org= .apache.derby.impl.db.BasicDatabase.setupConnection(Unknown Source)<= u>

=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0 at org.apache.derby.impl.jdbc.TransactionResourceImpl.startTransaction(= Unknown Source)

=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 at org= .apache.derby.impl.jdbc.EmbedConnection.checkUserIsNotARole(Unknown Source)=

=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0 at org.apache.derby.impl.jdbc.EmbedConnection.checkUserCredent= ials(Unknown Source)

=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 ... 15= more=A0

=A0

Regards,

= =A0

Erick

=A0

=
--f46d044303004d9dc904c8eb484b--