db-derby-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dag Wanvik <dag.wan...@oracle.com>
Subject Re: NPE
Date Fri, 14 Dec 2012 23:26:09 GMT
Hi Pavel,

speculating a bit, it could be that somehow the Oracle query eats up
memory so that the in-memory Derby doesn't get enough, and that somehow,
when it tries to instantiate its result set this fails. Your workaround
could work because this allows Derby to instantiate it while there is
still space on the heap. Obviously, Derby should have reported OOM if
that is indeed the problem.. Id you try to monitor the app and see if
you're ever close to the max heap size?  Mind you, just guessing here...

Dag


On 11.12.2012 17:53, Pavel Bortnovskiy wrote:
> Dag, et al. if I may add a bit of color to what's happening...
> I've done lots of various tests (unfortunately, still unable to distill it down to an
easy-to-reproduce tester).
> And what I discovered is baffling...
>
> The issue seems to occur (with 100% certainty) under the following circumstances:
> (1) two db connections - one to Derby (in-memory only mode) and Oracle
> (2) a prepared statement is executed (such as "select * from <table>") against
Derby
> (3) Oracle is queried for tables with:
> final ResultSet resultSet = databaseMetaData.getTables(null, null, null, new String[]{"TABLE"});
> (4) if the result set from (2) is iterated over AFTER the (3) then Derby fails with the
described NPE; however, if it is iterated over BEFORE (3), then OK!
>
> It may also be worth noting that my resultsets are made "rewindable", i.e. with:
>                 statement = connection.prepareStatement(
>                         sql,
>                         ResultSet.TYPE_SCROLL_INSENSITIVE,
>                         ResultSet.CONCUR_READ_ONLY);
>
>
> Finally: although I am completely perplexed as to what's causing this problem, I found
a work-around which, in my estimation, will have minimal impact: at a particular point in
my code (but effectively before calling getTables() in (3)), the result set from (2) is iterated
to the first record and then rewound:
>
> (not exact code, but that's what effectively is occurring):
>
> If (resultSet.next()) {
>    resultSet.beforeFirst();
> }
>
> As odd as it sounds, it seems to have fixed the issue...
>
> I am planning to return to this as time allows. Then I may be able to create a stand-alone
tester.
> But if you have any thoughts as to what might be going on, I would be most appreciative.
>
> Regards,
> Pavel.
>
> -----Original Message-----
> From: Dag Wanvik [mailto:dag.wanvik@oracle.com]
> Sent: Monday, December 10, 2012 3:15 PM
> To: derby-user@db.apache.org
> Subject: Re: NPE
>
>
> On 10.12.2012 19:21, Pavel Bortnovskiy wrote:
>> I upgraded to 10.9.1.0 and rerun with debug libraries (please disregard my previous
message).
>>
>> Mon Dec 10 13:19:44 EST 2012
>> Thread[DatabasePublisher[BOND_WRITER],5,main] (XID = 289), (SESSIONID = 5), (DATABASE
= memory:test), (DRDAID = null), Failed Statement is: select *  from READER java.lang.NullPointerException
>>         at org.apache.derby.impl.store.access.conglomerate.GenericScanController.reopenAfterEndTransaction(GenericScanController.java:1048)
>>         at
> It looks like "container" is null, cf. the line 1048:
>
>         case SCAN_HOLD_INIT:
> ***>         reusableRecordIdSequenceNumber =
>                 container.getReusableRecordIdSequenceNumber();
>
> i.e. has been closed, cf the assignment just above:
>
>         ContainerHandle container = open_conglom.reopen();
>
> The call to reopen just returns null if OpenConglomerate has been closed somehow.
>
> Smells like a race condition of sorts, maybe some missing synchronization in the Derby
code.
> If you are able to post a self contained repro program, although this might be tricky
if this is indeed a race, this would increase your chances of having this fixed soon..
>
> Thanks,
> Dag
>
>
>  In any case the NullPointerException shoudln't be seen by the application code, so this
is a bug.
>
>
>> org.apache.derby.impl.store.access.conglomerate.GenericScanController.fetchRows(GenericScanController.java:622)
>>         at org.apache.derby.impl.store.access.heap.HeapScan.fetchNextGroup(HeapScan.java:322)
>>         at org.apache.derby.impl.sql.execute.BulkTableScanResultSet.reloadArray(BulkTableScanResultSet.java:353)
>>         at org.apache.derby.impl.sql.execute.BulkTableScanResultSet.getNextRowCore(BulkTableScanResultSet.java:308)
>>         at org.apache.derby.impl.sql.execute.ProjectRestrictResultSet.getNextRowCore(ProjectRestrictResultSet.java:261)
>>         at org.apache.derby.impl.sql.execute.ScrollInsensitiveResultSet.getNextRowFromSource(ScrollInsensitiveResultSet.java:801)
>>         at org.apache.derby.impl.sql.execute.ScrollInsensitiveResultSet.getNextRowCore(ScrollInsensitiveResultSet.java:518)
>>         at org.apache.derby.impl.sql.execute.BasicNoPutResultSetImpl.getNextRow(BasicNoPutResultSetImpl.java:478)
>>         at org.apache.derby.impl.jdbc.EmbedResultSet.movePosition(EmbedResultSet.java:432)
>>         at org.apache.derby.impl.jdbc.EmbedResultSet.next(EmbedResultSet.java:376)
>>         at QueryResults$1.hasNext(QueryResults.java:268)
>>         at DatabasePublisher.process(DatabasePublisher.java:318)
>>         at java.lang.Thread.run(Thread.java:662)
>>
>> -----Original Message-----
>> From: Rick Hillegas [mailto:rick.hillegas@oracle.com]
>> Sent: Monday, December 10, 2012 1:05 PM
>> To: derby-user@db.apache.org
>> Subject: Re: NPE
>>
>> Hi Pavel,
>>
>> Can you reproduce this problem with the debug version of Derby? That would give us
line numbers in the stack trace.
>>
>> Thanks,
>> -Rick
>>
>> On 12/10/12 9:23 AM, Pavel Bortnovskiy wrote:
>>> Hello:
>>>
>>> While testing my application, which uses Derby in memory-only mode, I
>>> get NPE inside Derby under "mysterious" circumstances.
>>>
>>> At this point, I can't even figure out what's triggering it and where
>>> this could happen, but let me preempt my troubleshooting with some
>>> stack traces. Perhaps someone, with good knowledge of Derby's code,
>>> may offer some explanation. Here are some stack traces:
>>>
>>> Mon Dec 10 12:12:36 EST 2012 Thread[DatabasePublisher[WRITER],5,main]
>>> (XID = 235), (SESSIONID = 5), (DATABASE = memory:test), (DRDAID =
>>> null), Failed Statement is: select * from READER
>>>
>>> java.lang.NullPointerException
>>>
>>> at
>>> org.apache.derby.impl.store.access.conglomerate.GenericScanController.
>>> reopenAfterEndTransaction(Unknown
>>> Source)
>>>
>>> at
>>> org.apache.derby.impl.store.access.conglomerate.GenericScanController.
>>> fetchRows(Unknown
>>> Source)
>>>
>>> at
>>> org.apache.derby.impl.store.access.heap.HeapScan.fetchNextGroup(Unkno
>>> w
>>> n Source)
>>>
>>> at
>>> org.apache.derby.impl.sql.execute.BulkTableScanResultSet.reloadArray(
>>> U
>>> nknown
>>> Source)
>>>
>>> at
>>> org.apache.derby.impl.sql.execute.BulkTableScanResultSet.getNextRowCo
>>> r
>>> e(Unknown
>>> Source)
>>>
>>> at
>>> org.apache.derby.impl.sql.execute.ProjectRestrictResultSet.getNextRow
>>> C
>>> ore(Unknown
>>> Source)
>>>
>>> at
>>> org.apache.derby.impl.sql.execute.ScrollInsensitiveResultSet.getNextR
>>> o
>>> wFromSource(Unknown
>>> Source)
>>>
>>> at
>>> org.apache.derby.impl.sql.execute.ScrollInsensitiveResultSet.getNextR
>>> o
>>> wCore(Unknown
>>> Source)
>>>
>>> at
>>> org.apache.derby.impl.sql.execute.BasicNoPutResultSetImpl.getNextRow(
>>> U
>>> nknown
>>> Source)
>>>
>>> at org.apache.derby.impl.jdbc.EmbedResultSet.movePosition(Unknown
>>> Source)
>>>
>>> at org.apache.derby.impl.jdbc.EmbedResultSet.next(Unknown Source)
>>>
>>> at QueryResults$1.hasNext(QueryResults.java:268)
>>>
>>> at DatabasePublisher.process(DatabasePublisher.java:318)
>>>
>>> at java.lang.Thread.run(Thread.java:662)
>>>
>>> or
>>>
>>> 2012-12-10 12:12:36,537 [DatabasePublisher[WRITER]] ERROR
>>> QueryResults
>>> - Unable to execute ResultSet.next() for Statement ""select * from
>>> READER" [DERBY]"
>>>
>>> java.sql.SQLException: Java exception: ': java.lang.NullPointerException'.
>>>
>>> at
>>> org.apache.derby.impl.jdbc.SQLExceptionFactory40.getSQLException(Unkn
>>> o
>>> wn
>>> Source)
>>>
>>> at org.apache.derby.impl.jdbc.Util.newEmbedSQLException(Unknown
>>> Source)
>>>
>>> at org.apache.derby.impl.jdbc.Util.javaException(Unknown Source)
>>>
>>> at
>>> org.apache.derby.impl.jdbc.TransactionResourceImpl.wrapInSQLException
>>> (
>>> Unknown
>>> Source)
>>>
>>> at
>>> org.apache.derby.impl.jdbc.TransactionResourceImpl.handleException(Un
>>> k
>>> nown
>>> Source)
>>>
>>> at org.apache.derby.impl.jdbc.EmbedConnection.handleException(Unknown
>>> Source)
>>>
>>> at org.apache.derby.impl.jdbc.ConnectionChild.handleException(Unknown
>>> Source)
>>>
>>> at
>>> org.apache.derby.impl.jdbc.EmbedResultSet.closeOnTransactionError(Unk
>>> n
>>> own
>>> Source)
>>>
>>> at org.apache.derby.impl.jdbc.EmbedResultSet.movePosition(Unknown
>>> Source)
>>>
>>> at org.apache.derby.impl.jdbc.EmbedResultSet.next(Unknown Source)
>>>
>>> at QueryResults$1.hasNext(QueryResults.java:268)
>>>
>>> at DatabasePublisher.process(DatabasePublisher.java:318)
>>>
>>> at java.lang.Thread.run(Thread.java:662)
>>>
>>> Caused by: java.sql.SQLException: Java exception: ':
>>> java.lang.NullPointerException'.
>>>
>>> at
>>> org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknow
>>> n
>>> Source)
>>>
>>> at
>>> org.apache.derby.impl.jdbc.SQLExceptionFactory40.wrapArgsForTransport
>>> A
>>> crossDRDA(Unknown
>>> Source)
>>>
>>> ... 14 more
>>>
>>> Caused by: java.lang.NullPointerException
>>>
>>> at
>>> org.apache.derby.impl.store.access.conglomerate.GenericScanController.
>>> reopenAfterEndTransaction(Unknown
>>> Source)
>>>
>>> at
>>> org.apache.derby.impl.store.access.conglomerate.GenericScanController.
>>> fetchRows(Unknown
>>> Source)
>>>
>>> at
>>> org.apache.derby.impl.store.access.heap.HeapScan.fetchNextGroup(Unkno
>>> w
>>> n Source)
>>>
>>> at
>>> org.apache.derby.impl.sql.execute.BulkTableScanResultSet.reloadArray(
>>> U
>>> nknown
>>> Source)
>>>
>>> at
>>> org.apache.derby.impl.sql.execute.BulkTableScanResultSet.getNextRowCo
>>> r
>>> e(Unknown
>>> Source)
>>>
>>> at
>>> org.apache.derby.impl.sql.execute.ProjectRestrictResultSet.getNextRow
>>> C
>>> ore(Unknown
>>> Source)
>>>
>>> at
>>> org.apache.derby.impl.sql.execute.ScrollInsensitiveResultSet.getNextR
>>> o
>>> wFromSource(Unknown
>>> Source)
>>>
>>> at
>>> org.apache.derby.impl.sql.execute.ScrollInsensitiveResultSet.getNextR
>>> o
>>> wCore(Unknown
>>> Source)
>>>
>>> at
>>> org.apache.derby.impl.sql.execute.BasicNoPutResultSetImpl.getNextRow(
>>> U
>>> nknown
>>> Source)
>>>
>>> ... 6 more
>>>
>>> One more thing I can share is that this seems to also be a timing
>>> problem - if the code is executed slowly, line-by-line in the
>>> debugger, then this problem doesn't occur. But if run at full speed,
>>> then these exceptions are thrown. The problem is not intermittent and
>>> easily reproduced 100% of the time on my end.
>>>
>>> Any thoughts would be greatly appreciated.
>>>
>>> Pavel.
>>>
>>>
>>>
>>>             Jefferies archives and monitors outgoing and incoming
>>>             e-mail. The contents of this email, including any
>>>             attachments, are confidential to the ordinary user of the
>>>             email address to which it was addressed. If you are not
>>>             the addressee of this email you may not copy, forward,
>>>             disclose or otherwise use it or any part of it in any form
>>>             whatsoever. This email may be produced at the request of
>>>             regulators or in connection with civil litigation.
>>>             Jefferies accepts no liability for any errors or omissions
>>>             arising as a result of transmission. Use by other than
>>>             intended recipients is prohibited. In the United Kingdom,
>>>             Jefferies operates as Jefferies International Limited;
>>>             registered in England: no. 1978621; registered office:
>>>             Vintners Place, 68 Upper Thames Street, London EC4V 3BJ.
>>>             Jefferies International Limited is authorised and
>>>             regulated by the Financial Services Authority.
>>>
>> Jefferies archives and monitors outgoing and incoming e-mail. The contents of this
email, including any attachments, are confidential to the ordinary user of the email address
to which it was addressed. If you are not the addressee of this email you may not copy, forward,
disclose or otherwise use it or any part of it in any form whatsoever. This email may be produced
at the request of regulators or in connection with civil litigation. Jefferies accepts no
liability for any errors or omissions arising as a result of transmission. Use by other than
intended recipients is prohibited. In the United Kingdom, Jefferies operates as Jefferies
International Limited; registered in England: no. 1978621; registered office: Vintners Place,
68 Upper Thames Street, London EC4V 3BJ. Jefferies International Limited is authorised and
regulated by the Financial Services Authority.
>
> Jefferies archives and monitors outgoing and incoming e-mail. The contents of this email,
including any attachments, are confidential to the ordinary user of the email address to which
it was addressed. If you are not the addressee of this email you may not copy, forward, disclose
or otherwise use it or any part of it in any form whatsoever. This email may be produced at
the request of regulators or in connection with civil litigation. Jefferies accepts no liability
for any errors or omissions arising as a result of transmission. Use by other than intended
recipients is prohibited. In the United Kingdom, Jefferies operates as Jefferies International
Limited; registered in England: no. 1978621; registered office: Vintners Place, 68 Upper Thames
Street, London EC4V 3BJ. Jefferies International Limited is authorised and regulated by the
Financial Services Authority.


Mime
View raw message