db-derby-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From <Malte.Kem...@de.equens.com>
Subject AW: OOM with millions of weakly-referenced Derby objects
Date Fri, 20 Nov 2009 12:04:11 GMT
Hi to all again,

I could analyse and solve the problem.

Originally I thought that if ij findes something phony it would give a return code as its
signature suggests it.

Actually after finding out that the trouble stays somehow within ij after minutes of processing
and that I have to set another IO-Stream and put it out I found exceptions-traces like this

2009-11-20 09:43:20,232 DEBUG [main] de.equens.fed.converting.data.FieldMemory: IJ ERROR:
IOException: null


2009-11-20 09:43:20,241 DEBUG [main] de.equens.fed.converting.data.FieldMemory: org.apache.derby.impl.tools.ij.ijException:
IOException: null


2009-11-20 09:43:20,251 DEBUG [main] de.equens.fed.converting.data.FieldMemory:      at org.apache.derby.impl.tools.ij.ijException.iOException(Unknown


2009-11-20 09:43:20,257 DEBUG [main] de.equens.fed.converting.data.FieldMemory:      at org.apache.derby.impl.tools.ij.StatementFinder.peekChar(Unknown


2009-11-20 09:43:20,264 DEBUG [main] de.equens.fed.converting.data.FieldMemory:      at org.apache.derby.impl.tools.ij.StatementFinder.nextStatement(Unknown


2009-11-20 09:43:20,270 DEBUG [main] de.equens.fed.converting.data.FieldMemory:      at org.apache.derby.impl.tools.ij.utilMain.runScriptGuts(Unknown


2009-11-20 09:43:20,285 DEBUG [main] de.equens.fed.converting.data.FieldMemory:      at org.apache.derby.impl.tools.ij.utilMain.goScript(Unknown


2009-11-20 09:43:20,293 DEBUG [main] de.equens.fed.converting.data.FieldMemory:      at org.apache.derby.tools.ij.runScript(Unknown


2009-11-20 09:43:20,302 DEBUG [main] de.equens.fed.converting.data.FieldMemory:      at de.equens.fed.converting.data.FieldMemory.runScript(FieldMemory.java:506)


2009-11-20 09:43:20,311 DEBUG [main] de.equens.fed.converting.data.FieldMemory:      at de.equens.fed.converting.data.FieldMemory.reorganize(FieldMemory.java:397)


2009-11-20 09:43:20,320 DEBUG [main] de.equens.fed.converting.data.FieldMemory:      at de.equens.fed.converting.pacs2swift.Pacs08ToMT103Converter.<init>(Pacs08ToMT103Converter.java:53)


2009-11-20 09:43:20,330 DEBUG [main] de.equens.fed.converting.data.FieldMemory:      at java.lang.Class.newInstanceImpl(Native


2009-11-20 09:43:20,338 DEBUG [main] de.equens.fed.converting.data.FieldMemory:      at java.lang.Class.newInstance(Class.java:1243)


2009-11-20 09:43:20,344 DEBUG [main] de.equens.fed.converting.data.FieldMemory:      at de.equens.fed.converting.pacs2swift.PacsToMT103ConvFactory.newInstance(PacsToMT103ConvFactory.java:22)


2009-11-20 09:43:20,348 DEBUG [main] de.equens.fed.converting.data.FieldMemory:      at de.equens.fed.converting.pacs2swift.PacsToMT103Handler.startElement(PacsToMT103Handler.java:102)


2009-11-20 09:43:20,352 DEBUG [main] de.equens.fed.converting.data.FieldMemory:      at org.apache.xerces.parsers.AbstractSAXParser.startElement(Unknown


2009-11-20 09:43:20,357 DEBUG [main] de.equens.fed.converting.data.FieldMemory:      at org.apache.xerces.impl.XMLNSDocumentScannerImpl.scanStartElement(Unknown


2009-11-20 09:43:20,361 DEBUG [main] de.equens.fed.converting.data.FieldMemory:      at org.apache.xerces.impl.XMLDocumentFragmentScannerImpl$FragmentContentDispatcher.dispatch(Unknown


2009-11-20 09:43:20,366 DEBUG [main] de.equens.fed.converting.data.FieldMemory:      at org.apache.xerces.impl.XMLDocumentFragmentScannerImpl.scanDocument(Unknown


2009-11-20 09:43:20,370 DEBUG [main] de.equens.fed.converting.data.FieldMemory:      at org.apache.xerces.parsers.XML11Configuration.parse(Unknown


2009-11-20 09:43:20,374 DEBUG [main] de.equens.fed.converting.data.FieldMemory:      at org.apache.xerces.parsers.XML11Configuration.parse(Unknown


2009-11-20 09:43:20,377 DEBUG [main] de.equens.fed.converting.data.FieldMemory:      at org.apache.xerces.parsers.XMLParser.parse(Unknown


2009-11-20 09:43:20,380 DEBUG [main] de.equens.fed.converting.data.FieldMemory:      at org.apache.xerces.parsers.AbstractSAXParser.parse(Unknown


2009-11-20 09:43:20,383 DEBUG [main] de.equens.fed.converting.data.FieldMemory:      at org.apache.xerces.jaxp.SAXParserImpl.parse(Unknown


2009-11-20 09:43:20,385 DEBUG [main] de.equens.fed.converting.data.FieldMemory:      at javax.xml.parsers.SAXParser.parse(Unknown


2009-11-20 09:43:20,387 DEBUG [main] de.equens.fed.converting.data.FieldMemory:      at de.equens.fed.converting.Pacs_MT103.run(Pacs_MT103.java:76)


2009-11-20 09:43:20,388 DEBUG [main] de.equens.fed.converting.data.FieldMemory:      at de.equens.fed.converting.Pacs_MT103.main(Pacs_MT103.java:174)


That gave me the Idea that the sql-script I am trying to play on the database is probably
not UTF-8.

And .. voilĂ   ... after converting this script into UTF-8 all works fine, finally.

There are 2 things to learn about.

1.	IJ seems not work with problems as I would aspect it as normal user, intuitively there
should be either an exception or a returncode that tells you that something is going wrong
instead of producing 5 mb ouput for a 3 kb script in my case.
2.	here is obviously an os-specific java behaviour (or may be its just the behaviour of the
different JVM) on how to read text files since the very same sql-script worked on Solaris
and Windows  with a Sun-JVM (JDK 5) that seems to be a bit more tolerant.


In other words I would count that situation as not nice behaviour of ij, since the return
code should have been given as I would suppose it looking at the method signature.




Von: Malte.Kempff@de.equens.com [mailto:Malte.Kempff@de.equens.com] 
Gesendet: Donnerstag, 19. November 2009 13:26
An: derby-user@db.apache.org
Betreff: AW: OOM with millions of weakly-referenced Derby objects


Hi to all,

I have somehow a problem using derby on AIX 5.3

Derby is used as embedded DB by a little program. The database is constructed when it is not
there using the specific driver property. That seems to work. Then I am using ij to create
the tables etc if the first statement crashes because of having not table. This strategy works
perfectly on Windows and Sun-Solaris as well, but now I am facing trouble on an AIX-machine.

Following program method uses ij

private void runScript(String fileName) throws FileNotFoundException, UnsupportedEncodingException,
IOException, SQLException


      BufferedInputStream iStream = null;

      PrintStream         pStream = null;



            FileInputStream fileStream   = new FileInputStream(fileName);

            iStream                      = new BufferedInputStream(fileStream);

            ByteArrayOutputStream bOut   = new ByteArrayOutputStream(512);

            pStream                      = new PrintStream(bOut);


            logger.info("running Script '"+fileName+"'");


            int result = ij.runScript(m_conn, iStream, "UTF-8", pStream, "UTF-8");

            logger.debug("Result code is: " + result);


            if (result!=0)



                  throw new RuntimeException("problems running "+fileName+" rc:"+result+"\n
see logs for more information");





            if (pStream!=null)





            if (iStream!=null)







Obviously concerning the logs ij is called as in this line

int result = ij.runScript(m_conn, iStream, "UTF-8", pStream, "UTF-8");

after several minutes the program seems to crash without any exception I could see the derby.log
got following entries (I usually expect

but does not show any of ij as it does when I use it on Window or Solaris)

following JVM is used on our AIX 5.3:

$ /usr/java5_64/bin/java -version

java version "1.5.0"

Java(TM) 2 Runtime Environment, Standard Edition (build pap64dev-20061003a (SR3))

IBM J9 VM (build 2.3, J2RE 1.5.0 IBM J9 2.3 AIX ppc64-64 j9vmap6423-20061003 (JIT enabled)

J9VM - 20060915_08260_BHdSMr

JIT  - 20060908_1811_r8

GC   - 20060906_AA)

JCL  - 20061003



1.    have derby and ij ever run successfully on AIX (e.g. 5.3) with Java5 (64 bit)?

2.    is there a possibility to activate logs from ij so we can better see what is happening?

3.    Did anybody face a similar problem and maybe got a workaround?


Thanks in advance




View raw message