Mailing-List: contact derby-dev-help@db.apache.org; run by ezmlm
Precedence: bulk
Reply-To: "Derby Development" <derby-dev@db.apache.org>
Received-SPF: pass (asf.osuosl.org: local policy)
Message-ID: <42B6D4DA.8010407@acadiau.ca>
Date: Mon, 20 Jun 2005 11:38:18 -0300
From: Philip Wilder <050503w@acadiau.ca>
User-Agent: Mozilla Thunderbird 1.0.2 (Windows/20050317)
MIME-Version: 1.0
To: Derby Development <derby-dev@db.apache.org>
Subject: Derby-213 Chat Transcripts June 15 & June 20, 2005
Content-Type: text/plain; charset=windows-1252; format=flowed
Content-Transfer-Encoding: 8bit

Hello All,

It's been a while since the last transcript posting, so to compensate 
for that I'll be submitting a double whammy today for the days of June 
15, 2005 and June 20, 2005. Kathey and I have also come to the 
conclusion since the last posting that rather then further saturate the 
poor DERBY-213 Jira issue any further we would post the transcripts 
directly to the board. There was also some talk of just posting the 
summaries and making the full transcripts on demand. I�ve since come to 
dislike this idea because it has the potential to make more work for 
everyone involved but if this idea has supporters they are welcome to 
make their opinion known.

Also please be on the lookout for a "statement completion semantics 
challenge" (to quote Kathey) in the near future. Strangely, one of the 
most difficult tasks we currently face is locking down exactly what is 
meant by Statement completion as per JDBC specifications. Once we have 
successfully done this we will know all the issues that need be 
addressed to fully resolve DERBY-213. Not surprisingly it would be 
beneficial to have multiple developers come and try to poke holes in our 
plans, hence the challenge.

###############
DERBY-213 chat summary for June 15, 2005

* Based off the ResultSet_Outline.pdf file submitted prior to the 
meeting it was established that we were using the incorrect approach to 
fix the DERBY-213 issue. Rather then making an explicit call to closeX() 
in the nextX() statement we should only be calling commit if autocommit 
is set to true. Also the commit logic should be placed in Statement 
rather then ResultSet as JDBC documentation would seem to indicate (See 
java.sql.Connection.setAutoCommit())
* Looked for places in the client code from which to would be 
appropriate to make a commit call without much success.
* Briefly discussed some peculiarities in 
org.apache.derby.diag.LockTable() but decided to ignore them for the moment.
* Assigned Philip the task of finding a suitable place to add the commit 
Framework.

* Discussed the move from posting IRC chat transcripts on JIRA back to 
the derby-dev group. Also pondered the idea of making the full 
transcripts on demand.

###############################
DERBY-213 chat summary for June 20, 2005

* Attempted a precise definition of exactly what needed to be 
accomplished and noticed holes in the Client changes proposed on June 
17th, 2005. Specifically more work needs to be done in:
1) DatabaseMetaData - The Embedded Client gives special significance to 
DatabaseMetaData ResultSets and we need to establish why.
2) CallableStatement - We need to consider the multiple ResultSets 
associated with a CallableStatement and output paramaters.
3) Statement Completion on execute - We need to give more consideration 
to Statement completion associated with calls to the Statement.execute() 
method.
* Discussed a test from jdbcapi/metadata.java and established that a 
large piece of the test in question had changed since Kathey had last 
looked at it. As consequence Philip is to create a stand alone to 
reproduce the pertinent test.
* Assigned tasks for the next meeting and agreed to meet on June 22, 2005.
#####################
DERBY-213 chat transcript for June 15, 2005

[08:47] *** #derby: Bogomips kmarsden
[08:47] *** #derby was created on Wed Jun 15 08:10:21 2005.
[08:52] kmarsden: Good morning Philip
[08:52] Bogomips: Morning Kathey.
[08:53] kmarsden: I may have to cut short this morning. OK to start a 
few minutes early
[08:53] kmarsden: ?
[08:53] Bogomips: That's fine.
[08:53] kmarsden: Thank you for your summary, It was very helpful and 
made me realize we have our commit in entirely the wrong place.
[08:54] kmarsden: We need to handle the case where we have multiple 
resultSets
[08:54] kmarsden: and we commit only after the last one is complete.
[08:54] Bogomips: Ok, then we use the same approach as in embedded.
[08:55] kmarsden: I looked at that briefly. A callback on the statement 
that takes a resultset.
[08:55] kmarsden: have you looked at it?
[08:56] Bogomips: Also briefly. It would seem that they do extensive 
checking to insure that there is only one resultset open.
[08:56] kmarsden: In client there is a completExecute() method in 
am/Statement
[08:58] Bogomips: ok
[08:58] kmarsden: I am not sure if that is right though yet.
[08:59] kmarsden: Lets look at where it is called in NetStatement
[08:59] kmarsden: in parseEXCSQLSTTreply
[09:00] kmarsden: Well No I don't think that is right.
[09:01] Bogomips: Do you know what SQLDTARD stands for off the top of 
your head?
[09:01] kmarsden: We have not even gotten the result sets yet there.
[09:01] kmarsden: I can tell you what is is but not what it stands for.
[09:02] kmarsden: It is the output parameter data.
[09:02] kmarsden: For callable statements you can return data output 
parameters and that is what is being returned here.
[09:02] Bogomips: SQL Data Reply Data
[09:03] Bogomips: Guess that fits.
[09:03] kmarsden: yep
[09:03] kmarsden: but when this completes we have not even gotten the 
resultsets yet. so it is not that.
[09:05] Bogomips: We do make a call to parseResultSetProcedure()
[09:05] Bogomips: Also, is it alright if I digress for a moment to run 
something by you?
[09:05] kmarsden: sure
[09:07] Bogomips: I told you before that I don't think the qryclsimp 
value is ever sent from the derbynetclient. At the time we didn't think 
that was a problem because there was never a time when the ResultSet was 
closed implicitly.

After toying around with Holdability in the embedded model though it 
would appear that it does close implicitly if the 
Close_Cursors_At_Commit value is set.
[09:08] Bogomips: I think this means we either have to adjust our 
workaround to accomodate this on the client side or alternatively use 
the qryclsimp value in communication with the server.
[09:09] kmarsden: Right in that case, what happens is that completing 
the statement drives a commit.
[09:09] kmarsden: The commit in turn closes the non holdable result set.
[09:10] Bogomips: Correct and we do not account for that at the moment 
correct?
[09:10] kmarsden: So hopefully if we get the commit in the right place 
on the client, that part will take care of itself.
[09:10] Bogomips: Ok, so commit would drive the close and not the other 
way around. Good call :-)
[09:11] kmarsden: It is really confusing . The way the commits and the 
result sets interact.
[09:11] Bogomips: Indeed.
[09:11] Bogomips: Alright, let's continue the search then.
[09:12] kmarsden: I want to see if there is anything in Statement 
currently that tries to handle this commit on statement completion .
[09:14] kmarsden: There is this isAutcommitableStaement
[09:15] kmarsden: says it is used to avoid autocommit in some cases
[09:16] kmarsden: in flowExecute() this thing is used to determine the 
chainAutoCommit value
[09:17] kmarsden: which sound promising
[09:17] kmarsden: and tht gets passed to writeExecute
[09:20] kmarsden: well sort of lost track of it there but there is also 
a piggybacedAutocommit in flowExecute.
[09:20] kmarsden: So there seems to be some sort of infrastructure here 
for this.
[09:22] Bogomips: We seem to be opening the ResultSet there, is that 
really the place to start thinking about autocommit?
[09:22] kmarsden: but I think I am going to send you off on this as your 
next tangent in this project.
[09:22] kmarsden: It doesn't seem right.
[09:22] Bogomips: alright.
[09:23] kmarsden: Can you look at it though and see if there is any 
infrastructure for processing on statement completion.
[09:24] kmarsden: Otherwise we need to think about implementing a 
callback like we have for embedded.
[09:25] kmarsden: I am trying to remember if I put this bug on the 
getting started list.
[09:25] Bogomips: I don't remember seeing it there, but it has been a 
while since I checked and you may have phrased it differently.
[09:26] Bogomips: As for this method I am not particularly optimistic 
but I will continue to look.
[09:26] kmarsden: I think you are right about this method.
[09:27] kmarsden: Just investigate the client for any infrastructure to 
support this.
[09:27] Bogomips: Check.
[09:27] kmarsden: Do you have any thoughts on our next course of action 
here?
[09:28] Bogomips: Let me try to phrase what we are trying to accomplish 
here. /more
[09:31] Bogomips: We want to look in the Statement and possible 
NetStatement classes for framework that would allow us to commit upon 
the completion of a Statement. Were such a framework to exist, we need 
not worry about the particulars of whether the statement is executing an 
update, insert or query, only that the Statement has completed.
/more
[09:32] Bogomips: If such a framework does not exist we have 2 choices:
1) Creation such a framework
2) Insure that the same thing is accomplished at the end of all possible 
sql actions rather then in one unified local.

The first option probably being the better choice.
[09:33] Bogomips: Can we use embedded as a template in this regard?
[09:33] kmarsden: Yes I think that is it. and yes 1 is a better choice 
if it does not exist.
[09:33] kmarsden: Yes we should use embedded as template if we can
[09:34] kmarsden: But hopefully there is such a framework in client that 
is just broken in this case.
[09:35] kmarsden: Sound ok?
[09:35] Bogomips: Certainly.
[09:37] kmarsden: You aren't getting bored or fed up with this bug are 
you? I think that it was good fortune that you picked it up and have the 
time for such careful analysis.
[09:38] Bogomips: While I would be lying if I said there wasn't some 
degree of frustration, I'm going to stick it out to the end :-)
[09:39] kmarsden: Oh no. OK we will try to speed it along then and not 
take such a scenic route.
[09:40] kmarsden: We will defer all that locking analysis for another bug.
[09:40] Bogomips: ?
Don't rush things on my account if we are going to fix this it should be 
done right.
[09:40] Bogomips: As a related aside though, my locking analysis is kind 
of strange.
[09:41] kmarsden: yes?
[09:41] kmarsden: Did you post that one yet or just the result set?
[09:42] Bogomips: Just the ResultSet one.

I wasn't sure my body of evidence was sufficient to post to Jira.

For example
[09:43] Bogomips: Depening on where and how I call new 
org.apache.derby.diag.LockTable() it appears to affect other sql calls 
to the same class.

If I I check on the lock count during calls to ResultSet.next() my tests 
tell me there is no lock held after completion.
[09:43] Bogomips: If on the other hand I omit that test, then said locks 
exist.
[09:44] kmarsden: Oh, I bet you are hitting the bit where executing 
another statement closes the result set.
[09:45] kmarsden: Or maybe I am misunderstanding what you are saying.
[09:45] kmarsden: But anyway we can talk about the locks next time but 
we will keep it constrained to testing our close.
[09:46] Bogomips: Well, that would sort of make sense except that I am 
still getting the same number of rows returned and the expected behavior 
of the orginal ResultSet.

Sure, one problem at a time.
[09:46] kmarsden: Another way to test locks might be to have another 
connection either check the lock count or try to drop the table.
[09:46] kmarsden: but for now we wil focus on our commit problem.
[09:47] kmarsden: ok. have to run. Monday?
[09:47] kmarsden: and feel free to ping me in between.
[09:48] Bogomips: Sure. While we need not have a full blown meeting 
before then I will hopefully be able to run my findings by you at an 
earlier point.
[09:48] kmarsden: yes. please do.
[09:49] Bogomips: Alright, talk to you monday.
[09:49] kmarsden: bye
[14:32] *** Deepa has joined #derby.
[15:55] Bogomips: Kathey, are callable statements the only way to 
generate multiple ResultSets per statement?
[16:12] *** Deepa has signed off IRC (Read error: 110 (Connection timed 
out)).
[16:28] kmarsden: Yes.
[16:28] kmarsden: right now they are
[16:28] Bogomips: Alright, the documentation seems to indicate as much.
[16:28] kmarsden: ultimately there might be other ways like
[16:29] kmarsden: executeQuery("select * from tab2;select * from tab2")
[16:31] Bogomips: I C.
[16:32] Bogomips: So what state is the second ResultSet in when the 
first is open?
[16:34] Bogomips: As in open, closed, etc...
[16:44] Bogomips: No worries. It was more question of curiousity rather 
then any practical knowledge at the moment.
[16:44] *** You have been disconnected. Wed Jun 15 16:44:43 2005.
Session Close (#derby): Wed Jun 15 16:44:45 2005


Session Start (Freenode:#derby): Thu Jun 16 09:14:24 2005
[09:14] *** #derby: Bogomips kmarsden
[09:14] *** #derby was created on Wed Jun 15 08:10:21 2005.
[09:17] kmarsden: Hi Philip
[09:17] Bogomips: Morning Kathey.
[09:18] Bogomips: We were to meet on Monday, correct?
[09:18] kmarsden: Yes. But if you have time I'd like to chat a bit this 
morning.
[09:19] Bogomips: Sure.
[09:20] kmarsden: First about your question from yesterday. Look at 
Statement.getMoreResults() to see what happens to the resultsets.
[09:22] kmarsden: You can pass a flag to one of the getMoreResults 
methods to not close the resultset.
[09:22] kmarsden: for autocommit that gets really confusing again.
[09:22] Bogomips: Doesn't that break the Statement jdbc spec?
[09:23] Bogomips: looking for the quote now...
[09:24] Bogomips: "By default, only one ResultSet object per Statement 
object can be open at the same time. Therefore, if the reading of one 
ResultSet object is interleaved with the reading of another, each must 
have been generated by different Statement objects. All execution 
methods in the Statement interface implicitly close a statment's current 
ResultSet object if an open one exists."
[09:24] Bogomips: I guess it says 'by default' so it is alright
[09:24] kmarsden: JDBC20 just had getMoreResults()
[09:24] kmarsden: always closed the result set
[09:24] kmarsden: now there is public boolean getMoreResults(int current)
[09:25] kmarsden: which can take
[09:25] kmarsden: KEEP_CURRENT_RESULT
[09:25] Bogomips: True enough.
[09:25] kmarsden: CLOSE_CURRENT_RESULT or even CLOSE_ALL_RESULTS
[09:27] kmarsden: So what I wanted to talk about was DERBY-213 in general
[09:27] kmarsden: It needs to be switch to Network Client
[09:28] Bogomips: done
[09:29] kmarsden: and I think that we need to add a comment that 
investigation of the issue has revealed that the client autocommit logic 
upon statement completion needs to be evaluated and perhaps 
reimplemented to match the JDBC spec and embedded as part of this fix
[09:31] kmarsden: Because it is looking like that is what we have to do. 
It is not just a simple bug fix.
[09:32] kmarsden: What do you think?
[09:33] Bogomips: Well I've been looking at the code some more, trying 
to establish the difference between the two models.

The main difference that I can tell is that in Embedded an explict check 
is made to insure that there are no open ResultSets before comitting.
[09:33] kmarsden: right
[09:33] kmarsden: where as client has no logic for autocommit at the 
statement level.
[09:34] kmarsden: Client was just closing the result set and committing 
with every close
[09:34] kmarsden: so was missing this piece
[09:34] Bogomips: It does use the Statement.generatedSection_ variable.
[09:35] kmarsden: need to pull up eclipse
[09:35] kmarsden: expain this to me while it comes up
[09:37] Bogomips: No, sorry. it looks like generatedSection_ is in 
resultSet.
Nevertheless, I believe that generated section is != null only when 
there are multiple ResultSets.
[09:38] Bogomips: If you look at the 
flowAutoCommitIfLastOpenMultipleResultSetWasJustClosed() method you will 
see numerous references to this resultSetList_[]
[09:38] kmarsden: oh yes. Normally the client generates the section 
number wich is a kind of statement identifier
[09:38] kmarsden: for stored procedures, the server generates the 
section numbers for additional resultsets
[09:40] Bogomips: So everytime there are additional ResultSets we have a 
generatedSection or just stored procedures?
[09:40] kmarsden: well right now stored procedures is the only time we 
have multiple resultsets
[09:40] kmarsden: where is the flowAutoCommitIfLastOopenMultipl... method
[09:41] Bogomips: client.am.ResultSet
[09:41] Bogomips: Basically, it will only close if all the ResultSets in 
the ResultSetList are null.
[09:42] kmarsden: Well two things about that.
[09:45] kmarsden: 1) We still need to separate closed from completed.
[09:46] kmarsden: 2) it seems kind of wrong that this mehod is in ResultSet.
[09:47] kmarsden: Whatever our check is I don't think it can be for null 
anymore because we won't actually be closing the result set
[09:48] Bogomips: Ok, understood.
[09:48] kmarsden: it will have to be rs == null || rs.completed() or 
some such.
[09:49] kmarsden: but I still like the embedded callback model better.
[09:49] kmarsden: than putting a loop in resultset that accesses it's 
statement and running through all the other resutsets.
[09:49] kmarsden: So I think we can fix all this.
[09:49] kmarsden: I think we need to
[09:50] kmarsden: 1) Understand exactly what the behaviour should be. I 
can tell you there have been many bugs in this area in embedded in the 
past until it was made right.
[09:51] kmarsden: 2) Look at embedded closely and use that as a template 
for fixing it in client.
[09:51] kmarsden: 3) Have good tests that really test all this.
[09:52] kmarsden: But the point I wanted to get across this morning is 
that if you choose to pursue this further, you shouldn't get frustrated 
with it as a drawn out bug fix but rather look at it as an important 
development project.
[09:52] Bogomips: Indeed. According to the chart I came up with we have 
at least 8 different tests that could be run.
[09:53] Bogomips: Right, it's grown into something a little bigger then 
expected :-)
[09:54] kmarsden: Which is often the case and I think very good in this 
case since you have the time to do the research and I tend to just get a 
day or two to "fix the bug"
[09:55] kmarsden: So you still game?
[09:55] Bogomips: Well, I've gone this far it would be a shame to stop 
now :-P
[09:55] kmarsden: good.
[09:56] kmarsden: So continue with your research into the spec and into 
the embedded imlplementation and perhaps come up with a test plan.
[09:57] kmarsden: There is some funny stuff in embedded with regard to 
metadata queries as I recall
[09:57] Bogomips: if (forMetaData && (lcc.getActivationCount < 1))

//do nothing

or something to that effect.
[09:58] Bogomips: actually change < to >
[09:58] Bogomips: anyway, it's something I'll look into.
[09:59] Bogomips: But, yes. For the moment either you or I should post a 
comment to Jira, alerting the others of our findings.
[09:59] kmarsden: Yes.
[10:00] kmarsden: Too bad the Jira entry is so clutttered.
[10:00] Bogomips: Indeed.
The transcripts certainly add up...
[10:00] kmarsden: I am tending to think we should go back to sending 
summaries to the list
[10:01] kmarsden: and skipping the transcripts. I think Dag must be 
bored with our hallway conversations by now.
[10:02] kmarsden: But post to JIRA what we found and flag it as 
***UPDATE** or somesuch
[10:03] Bogomips: Sure.
In the next summary I will mention our change in transcript philosophy. 
If anymone wants the full script they can email me.
[10:05] kmarsden: Good to post the transcript and say that we would like 
to stop cluttering up the JIRA entry with them. And see if there are any 
objections to making them on-demand.
[10:06] Bogomips: Alright.
[10:06] kmarsden: ok. talk to you monday.
[10:06] Bogomips: bye

#######################
DERBY-213 chat transcript for June 20, 2005

[08:36] kmarsden: Good Morning Philip
[08:36] Bogomips: Morning Kathey.
[08:37] kmarsden: Looks like you have been busy #:)
[08:37] Bogomips: Just trying to meet our timeline :-)
[08:37] kmarsden: We have a timeline?
[08:38] Bogomips: Well, the immediate objective was to have something 
posted to Jira.
[08:38] Bogomips: Not much beyond that I guess.
[08:38] kmarsden: I was shooting more for you being the first person in 
the history of JDBC to get this right on the first shot.
[08:39] Bogomips: I think that might be just a little bit optimistic :-P
[08:39] kmarsden: It has missed the 10.1 release so don't think there is 
a timeline.
[08:40] Bogomips: Alright.
[08:40] kmarsden: Were you able to come up with a behavior summary?
[08:41] Bogomips: As in the expected behavior for the different Test cases?
[08:42] kmarsden: Not that detailed. Let me make a first pass here and 
then you correct.
[08:42] Bogomips: k
[08:42] kmarsden: What I think we are doing is changing client to
[08:43] kmarsden: 1) Not close the resultset when last row is retrieved.
[08:43] kmarsden: 2) Send autocommit when the statement is complete.
[08:44] Bogomips: In the case of CLOSE_CURSORS_AT_COMMIT those two 
points are at odds but normally yes.
[08:44] Bogomips: unless of course autocommit is turned off.
[08:45] kmarsden: A statement is complete when all the last row of all 
the FORWARD_ONLY result sets has been retrieved.
[08:45] kmarsden: and all of the output parameters have been retrieved.
[08:45] kmarsden: So for 2 it should say
[08:45] kmarsden: 2) Send commit in autocommit mode when the statement 
is complete.
[08:46] kmarsden: The fact that that commit might in turn close the 
result set is incidental.
[08:47] kmarsden: I have to step away for 5 minutes. sorry. Can you 
rewrite this with what you think it is.
[08:47] Bogomips: Sure.
[08:47] kmarsden: also include scrollable result set info
[08:52] kmarsden: ok. i am back
[08:52] Bogomips: 1) Do not explicitly close the ResultSet with any 
calls to Close() or CloseX() method upon statement completion. In the 
case of ResultSets Statement completion is defined as the state when the 
current ResultSet has returned its final row through a call to next() 
and there are no other open ResultSets associated with the Statement. 
While the wording in JDBC may be ambiguous the current functionality in 
Embedded would seem to indicate that this applies to both Scrollable and 
non-scrollable ResultSets.
[08:53] kmarsden: I think we want a statement that has nothing to do 
with the code.
[08:53] kmarsden: Just JDBC.
[08:54] Bogomips: Alright.
[08:54] Bogomips: 2nd draft, just a second.
[08:54] kmarsden: k.
[08:57] Bogomips: hmmm... If I'm not making reference to the code I am 
just repeating what you said.
[08:57] kmarsden: Is what I said right?
[08:57] kmarsden: It is not complete I think
[08:58] kmarsden: It doesn't include anything about scrollable result sets.
[08:58] Bogomips: Neither does the jdbc autocommit wording.
[08:58] kmarsden: also it may need to include something about metadata
[08:59] kmarsden: but for our statement we need to decide.
[08:59] kmarsden: Your resultset summary said we MAY commit after 
fetching the lst row of the scrollable result set , but what do we plan 
to do?
[09:00] kmarsden: what does embedded do?
[09:00] Bogomips: As I mentioned, the embedded functionality would imply 
that it follows the same rules as Non-scrollable.
[09:00] Bogomips: This is the current functionality I emulate.
[09:00] kmarsden: ok.
[09:01] kmarsden: What do you think about the metadata?
[09:01] Bogomips: In terms of MetaData, as my email to the Derby-dev 
group might imply it has me a little confused.
[09:02] kmarsden: did you see my response?
[09:02] Bogomips: I did.
The inline documentation seems to indicate that you cannot commit 
DatabaseMetaData unless all other statement ResultSets are closed.
[09:03] kmarsden: in the embedded code?
[09:03] Bogomips: While I can appreciate that we cannot touch the 
Statement of the Databasemetadata object, I'm not sure why it affects 
the behavior of all the other statements.
[09:03] Bogomips: Yes, in the embedded code.
[09:03] Bogomips: I'll see if I can find the line that propels my thinking.
[09:04] kmarsden: From looking at the test case
[09:05] kmarsden: rs = s.executeQuery("SELECT * FROM SYS.SYSTABLES");
[09:05] kmarsden:
[09:05] kmarsden: System.out.println("getColumns('SYSTABLES'):");
[09:05] kmarsden: ResultSet rs2 = met.getColumns("", "SYS", "SYSTABLES", 
null);
[09:05] kmarsden: if (!rs.next()) {
[09:05] kmarsden: System.out.println("FAIL -- user result set closed by"+
[09:05] kmarsden: " intervening getColumns request");
[09:06] kmarsden: I think perhaps my assessment was wrong about the 
results getting cached up.
[09:06] Bogomips: // In case of resultset for MetaData, we will only commit
// if we are the only statement currently opened for this
// connection; otherwise we don't want to affect other
// resultSet's by committing the MetaData one.
// There is no internal xact (xact isolation) for MetaData type
// of resultSet; therefore committing (to release locks) would end
// up committing all the other resultSet for this connection.
[09:07] Bogomips: It is the only ResultSet that we seem to care about on 
a connection wide basis.
[09:08] Bogomips: That was taken from around line 416 of Embedded ResultSet.
[09:10] Bogomips: At the moment, no such provisions are provided from 
the Client code.
[09:11] kmarsden: In the test case for non holdable cursors. Is this the 
code that protects the user result set "rs"
[09:11] kmarsden: ?
[09:13] Bogomips: Actually, looking at the code again it strikes me as 
kind of strange.
rs can't be "closed" if it is returning false. If it was closed it would 
be throwing an excpetion. RS may be committed.
[09:14] Bogomips: In any case all, this code does is prevent a commit 
for ResultSets from Databasemetadata if there are more the 1 statement 
currently open.
[09:14] Bogomips: I mean this code:
if (forMetaData && (lcc.getActivationCount() > 1)) {
// we do not want to commit here as there seems to be other
// statements/resultSets currently opened for this connection.
} else if (owningStmt != null)
// allow the statement to commit if required.
owningStmt.resultSetClosing(this);
}
[09:15] kmarsden: You are right we are just talking about the commit.
[09:16] kmarsden: In the case we are talking about with non holdable 
resultsets and autocommit true and no special handling what would happen?
[09:16] kmarsden: I think the commit would be called on the statement 
execution for getColumns, is that right?
[09:18] Bogomips: The only thing I could find that distinguishes a 
normal ResultSet from a Databasemetadata resultSet are two blocks of 
code that look like the one above.

I see no reason why the commit should be called. In that case we are 
opening a new ResultSet from a seperate statement, there is no reason 
any committing should be done.
[09:19] kmarsden: I think executing a new statement plays a role in the 
result set./commit picture too.
[09:20] kmarsden: The commit occurs when the statement completes or the 
next execute occurs, whichever comes first
[09:20] kmarsden: from setAutoCommit
[09:20] kmarsden: so there is another condition for completion.
[09:20] kmarsden: or cause for completion
[09:21] Bogomips: Good eyes.
[09:23] Bogomips: Ok, that gives me a better appreciation for the 
metadata treatment.
Unlike other ResultSets, there is no statement present so a user might 
inadvertently call a commit in this instance without special handling.
[09:23] Bogomips: hmmm... this is a whole new issue.
[09:24] kmarsden: Maybe. But the funny thing is if this ananlysis of 
this case is true. It is not the code in movePosition that is protecting 
us from the commit.
[09:24] Bogomips: Let me look for instances of committing associated 
with execute in the embedded code.
[09:28] kmarsden: I think it is perfectly reasonable for any special 
handling for metadata to be added as a separate issue to DERBY-213. I 
just want to make sure we understand the whole picture.
[09:29] kmarsden: Also the commit on execution of a new statement in 
general is something we need to handle and test.
[09:29] Bogomips: hmmm....
If we have Statement A, Statement A has ResultSet AB open then we call 
A.execute() this will close ResultSet AB.

However If we had a second Statement B and we called B.execute I don't 
think it has an impact on the functioning of A...
[09:30] kmarsden: I see.
[09:30] Bogomips: Line 480 EmbeddedStatement.
[09:30] Bogomips: in the execute method.
[09:33] kmarsden: OK Then we are back to that case and why it fails 
without the special handling for CLOSE_CURSORS_AT_COMMIT
[09:33] kmarsden: I guess we'd just have to try it out.
[09:34] kmarsden: It never fetcthes the last row.
[09:34] Bogomips: We are still talking about
[09:05] kmarsden: System.out.println("getColumns('SYSTABLES'):");
[09:05] kmarsden: ResultSet rs2 = met.getColumns("", "SYS", "SYSTABLES", 
null);
[09:05] kmarsden: if (!rs.next()) {
[09:05] kmarsden: System.out.println("FAIL -- user result set closed by"+
[09:05] kmarsden: " intervening getColumns request");
correct?
[09:35] kmarsden: right.
[09:35] kmarsden: I think, that if the connection is 
CLOSE_CURSORS_AT_COMMIT this test fails.
[09:36] kmarsden: and it is the key to understanding why we have the 
special logic.
[09:37] kmarsden: It is a matter of whether a commit is sent related to 
the metadata request.
[09:39] Bogomips: Sorry, I seem to be having trouble finding your code 
snippet in the code.
[09:39] kmarsden: This is from the jdbcapi/metadata.java test
[09:40] Bogomips: I'll update and see if that was a rpoblem.
[09:40] kmarsden: But, the big caveat is that now Derby is 
HOLD_CURSORS_OVER_COMMIT by default.
[09:41] kmarsden: This test requrires CLOSE_CURSORS_AT_COMMIT
[09:41] kmarsden: and I don't think that the test was changed when the 
default was changed.
[09:42] kmarsden: So look at this case and see if it gives you any clues.
[09:42] kmarsden: Another issue for statement completion is the output 
parameters.
[09:43] kmarsden: you have to retrieve all the results and all the 
output parameters it says
[09:43] Bogomips: It's strange but I still can't find your code snippet...
Any chance you could email your file to me when you get a chance?
[09:48] kmarsden: I am absolutely dumbfounded
[09:49] Bogomips: :-(
Output parameters are not mentioned in the jdbc for ResultSet
nor in the Embedded documentation.
[09:49] kmarsden: there is a big part of this test that has gone away.
[09:49] kmarsden: Sets this connection's auto-commit mode to the given 
state. If a connection is in auto-commit mode, then all its SQL 
statements will be executed and committed as individual transactions. 
Otherwise, its SQL statements are grouped into transactions that are 
terminated by a call to either the method commit or the method rollback. 
By default, new connections are in auto-commit mode.
[09:49] kmarsden: The commit occurs when the statement completes or the 
next execute occurs, whichever comes first. In the case of statements 
returning a ResultSet object, the statement completes when the last row 
of the ResultSet object has been retrieved or the ResultSet object has 
been closed. In advanced cases, a single statement may return multiple 
results as well as output parameter values. In these cases, the commit 
occurs when all results
[09:49] kmarsden:
[09:49] kmarsden: setAutoCommit
[09:49] Bogomips: yes, that is the only place I can find reference to it.
[09:50] Bogomips: I'll check Statement to just in case.
[09:50] kmarsden: there are huge portions of this metadata test that 
have disappeared
[09:51] Bogomips: Very odd.
[09:51] Bogomips: Were you looking at an old copy?
[09:51] kmarsden: I sent that mail the other day while working on 
another codeline on a different project.
[09:51] kmarsden: I just assumed the test was the same.
[09:52] kmarsden: I have to check and see what happenned.
[09:52] Bogomips: Would you like me to try to rig up a standalone repro?
[09:53] kmarsden: Please. I can't send that file, but the snippet has 
the whole story there except that you need to set the Connection 
holdabalitiy to HOLD_CURSORS_AT_COMMIT.
[09:53] Bogomips: ?
That's the default. Do you mean close Cursors on commit?
[09:53] kmarsden: YES!
[09:54] kmarsden: sorry frazzled by the disappearing test
[09:54] Bogomips: no worries.
Also, jdbc docs for CallableStatement gives me a better idea what an 
output parameter is
[09:55] kmarsden: Output parameters are for Stored prepared statements
[09:55] kmarsden: My method can have parameters like
[09:55] Bogomips: Yes, I definately need to rig up some more advanced 
Callable tests.
[09:56] kmarsden: Look at the procedure.java test
[09:56] kmarsden: for examples.
[09:56] kmarsden: assuming it is still there!
[09:56] Bogomips: :-)
[09:56] kmarsden: Stored proceudres
[09:56] kmarsden: not stored prepared statements.
[09:56] kmarsden: frazzled
[09:57] Bogomips: No problem.
Well, I have a fair bit to do for our next meeting.
[09:58] Bogomips: 1) Replicate the test given in this chat.
[09:58] Bogomips: 2) Play with stored procedures and devlop tests for 
multiple ResultSets.
[09:59] Bogomips: 3) Continue work on the basic series of tests for 
jdbcapi/ResultSet.java testing autocommit and proper closing.
[10:00] Bogomips: Can I assume you'll attempt to track down the 
committer who removed some of the test code?
[10:01] kmarsden: 0) Refine our summary for behavior changes we are 
making for client. I really want to make sure we understand when the 
statement is complete and the autocommit is sent.
[10:01] kmarsden: Or maybe even better would be this
[10:02] kmarsden: No nevermind. not a better idea.
[10:04] kmarsden: Anyway I want to get to a point that we have in 
writing something that makes it clear to both of us when the statement 
is complete commit is sent.
[10:04] kmarsden: OK. yes I will track down that test.
[10:05] kmarsden: From what you say, It looks like embedded still has a 
bug for the output parameters.
[10:05] kmarsden: I tell you if you got this right the first time, you 
would be the first person in the history of JDBC to do so.
[10:05] Bogomips: I guess the real trick is just establishing all the 
ways in which the statement can be complete and any exceptions (i.e. 
Databasemetadata) to the completeness rules.
[10:05] kmarsden: yes
[10:13] kmarsden: OK. When shall we meet next.
[10:14] Bogomips: Wednesday would probably be good.
[10:14] kmarsden: ok. Wednesday at 5
[10:14] Bogomips: Alright, talk to you then.