db-derby-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Øystein Grøvlen (JIRA) <j...@apache.org>
Subject [jira] Commented: (DERBY-3243) (jdbc net client) exception during normal iteration through "ResultSet" of "select * from t"
Date Wed, 30 Jan 2008 15:10:35 GMT

    [ https://issues.apache.org/jira/browse/DERBY-3243?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12564040#action_12564040
] 

Øystein Grøvlen commented on DERBY-3243:
----------------------------------------

The reasoning for the check in NetCursor#locator was to be able to
both handle pre-10.3 servers, and 10.3 servers running in soft-upgrade
mode, both which ignore requests for use of locators.  I guess the
first group could be handled by recognizing that the server is
pre-10.3, and it should also be possible to come up with a separate
mechanism to handle soft upgrade.  However, It would be even better if
one where able to communicate the type of each value dynamically, but
I were not able to determine how to do this with DRDA.  Does anybody
know? 

I think that when I added the code that checked whether a locator or
LOB had been sent, I thought that the when a LOB was sent, its column
position would contain the length of the LOB with the high-order bit
set.  However, it seems that what has originally been sent is not the
length of the LOB, but the length of the length field for the LOB, and
it is not the high-order bit that is set, but the high-order bit of
byte 3.  When Layer B streaming was added, it seems the length of the
length field was fixed at 4 bytes so that now the client will always
receive 0x8004 for non-zero length LOBs.

I do not know whether the above is in accordance with the DRDA spec,
and it also seems that the client expect the high-order bit to be set when
the length is unknown.  From NetCursor#isNonTrivialDataLob:

    // if the high-order bit is set, length is unknown -> set value to x'FF..FF'
    if (((dataBuffer_[position]) & 0x80) == 0x80) {
        length = -1;
    } else {

A quick, but dirty, way to fix the reported bug would be to make the
server skip 0x8004 when allocating locators, and make the client
check for this exact value when determining whether it has received a
locator or not.


> (jdbc net client) exception during normal iteration through "ResultSet" of "select *
from t"
> --------------------------------------------------------------------------------------------
>
>                 Key: DERBY-3243
>                 URL: https://issues.apache.org/jira/browse/DERBY-3243
>             Project: Derby
>          Issue Type: Bug
>          Components: JDBC
>    Affects Versions: 10.3.2.1
>         Environment: ------------------ Java Information ------------------
> Java Version:    1.4.2_03
> Java Vendor:     Sun Microsystems Inc.
> Java home:       C:\Programme\Java\j2re1.4.2_03
> Java classpath:  derbyclient.jar
> OS name:         Windows XP
> OS architecture: x86
> OS version:      5.1
> Java user name:  cbdqok
> Java user home:  C:\Dokumente und Einstellungen\cbdqok
> Java user dir:   D:\users\cbdqok\privat\osj\lib_derby_net\lib
> java.specification.name: Java Platform API Specification
> java.specification.version: 1.4
> --------- Derby Information --------
> JRE - JDBC: J2SE 1.4.2 - JDBC 3.0
> [D:\users\cbdqok\privat\osj\lib_derby_net\lib\derbyclient.jar] 10.3.1.4 - (561794)
> ------------------------------------------------------
> ----------------- Locale Information -----------------
> ------------------------------------------------------
>            Reporter: Oliver Seidel
>            Priority: Critical
>         Attachments: derby-3243_fix_double_hashmap_entry_diff.txt, derby-3243_fix_double_hashmap_entry_diff2.txt,
derby-3243_fix_double_hashmap_entry_stat.txt
>
>
> Thanks again to the DERBY developers.
> For my convenience I use the client/server variant of DERBY.  This is where the error
occurs.  I have temporarily alleviated the problem for my case by using the embedded version.
> I'm forced to use outlook and have built a program which siphons the messages into a
derby database (phase 1 -- table has two varchar fields and a clob).  When reading them back
(phase 2 -- simple select of all records in no particular order, all 3 attributes), it has
developed an exception.  This is a result of data volume.  I have reduced the problem to a
single demonstration program.
> This behaviour, and the fact that the error is not a "DERBY Exception", but a proper
exception coming back from the JDBC call, leads me to the diagnosis that it is a derby network
client jdbc problem.
> The problem has been reproduced once by Bryan Pendleton: http://www.nabble.com/IndexOutOfBoundsException-t4926228.html
> I'll try to attach the code "error.java"; it can also be found on: http://www.os10000.net/error.java
> Please use as follows:
> javac -classpath derby.jar:derbynet.jar error.java
> java -cp .:derby.jar:derbynet.jar error
> It will create a directory "test_db" and fill it with approx 120mb data.
> Then you will observe the failure. 
> /* -*- Mode:Java; c-basic-offset:8 -*- */
> // --------------------------------------------------------------------------------
> public class error {
> 	// ------------------------------------------------------------------------
> 	public static final int port = 1527;
> 	public static final String host = "127.0.0.1";
> 	// ------------------------------------------------------------------------
> 	public static void log(String x) { System.out.println(x); };
> 	// ------------------------------------------------------------------------
> 	public static void log_stacktrace(Exception e) {
>                 java.io.StringWriter sw = new java.io.StringWriter();
>                 java.io.PrintWriter pw = new java.io.PrintWriter(sw);
>                 e.printStackTrace(pw);
>                 log(sw.toString());
> 	};
> 	// ------------------------------------------------------------------------
> 	public static void start_server() {
> 		try {
> 			org.apache.derby.drda.NetworkServerControl server =
> 				new org.apache.derby.drda.NetworkServerControl(java.net.InetAddress.getByName(host),port);
> 			server.start(null);
> 			try { java.lang.Thread.sleep(5000); } catch (Exception e) { };
> 			log("started DERBY on host "+host+" port "+Integer.toString(port)+".");
> 		} catch (Exception e) {
> 			log_stacktrace(e);
> 		};
> 	};
> 	// ------------------------------------------------------------------------
> 	public static java.sql.Connection con()
> 		throws java.sql.SQLException, java.lang.ClassNotFoundException {
> 		Class.forName("org.apache.derby.jdbc.ClientDriver");
> 		String x_dbse = "test_db";
> 		String x_user = ";user=os10000";
> 		String x_pass = ";password=ding";
> 		String x_crte = ";create=true";
> 		String x_msgs = ";retrieveMessagesFromServerOnGetMessage=true";
> 		String connect = "jdbc:derby://"+host+":"+port+"/"+x_dbse+x_user+x_crte+x_msgs;
> 		return java.sql.DriverManager.getConnection(connect);
> 	};
> 	// ----------------------------------------------------------------------
> 	public static String getclob(java.sql.Clob b) {
> 		java.io.CharArrayWriter caw = new java.io.CharArrayWriter();
> 		try {
> 			java.io.Reader rd = b.getCharacterStream();
> 			char cb[] = new char[4096];
> 			int off=0, bts = rd.read(cb);
> 			while (bts>0) { caw.write(cb,off,bts); off+=bts; bts=rd.read(cb); };
> 			rd.close();
> 		} catch(Exception e) {};
> 		String res = caw.toString();
> 		caw.close();
> 		return res;
> 	};
> 	// ----------------------------------------------------------------------
> 	public static class myclob implements java.sql.Clob {
> 		String v;
> 		public myclob(String z) { v=z; };
> 		public java.io.InputStream getAsciiStream() { return new java.io.ByteArrayInputStream(v.getBytes());
};
> 		public java.io.Reader getCharacterStream() { return new java.io.CharArrayReader(v.toCharArray());
};
> 		public String getSubString(long pos, int length) { return v.substring((int)pos,(int)(pos+length));
};
> 		public long length() { return v.length(); };
> 		public long position(java.sql.Clob pattern, long start) { return 0; };
> 	        public long position(String pattern, long start) { return 0; };
> 		public java.io.OutputStream setAsciiStream(long pos) {
> 			new java.sql.SQLException("setAsciiStream not implemented."); return null; };
> 		public java.io.Writer setCharacterStream(long pos) {
> 			new java.sql.SQLException("setCharacterStream not implemented."); return null; };
> 		public int setString(long pos, String s) {
> 			new java.sql.SQLException("setString not implemented."); return -1; };
> 		public int setString(long pos, String s, int offset, int len) {
> 			new java.sql.SQLException("setString not implemented."); return -1; };
> 		public void truncate(long len) {
> 			new java.sql.SQLException("truncate not implemented."); };
> 	};
> 	public static java.sql.Clob putclob(String x) { return new myclob(x); };
> 	
> 	// ------------------------------------------------------------------------
> 	public static String getLob(java.sql.ResultSet rs, int arg)
> 		throws java.sql.SQLException { return getclob(rs.getClob(arg)); };
> 	public static void setLob(java.sql.PreparedStatement ps, int arg, String val)
> 		throws java.sql.SQLException { ps.setClob(arg,putclob(val)); };
> 	// ------------------------------------------------------------------------
> 	public static String clean_string(String x) {
> 		return x.replaceAll("[\0\r\\\\]","").replaceAll("'","\\\"").replaceAll(",+",",");
> 	};
> 	// ------------------------------------------------------------------------
> 	public static String make_string(int digits) {
> 		double dl = (Math.random()) * (Math.pow(10.0,digits*Math.random()));
> 		int len = (int) dl;
> 		byte buf[] = new byte[len];
> 		while (len>0) {
> 			double db = 256*Math.random();
> 			buf[--len] = (byte)db;
> 		};
> 		return clean_string(new String(buf));
> 	};
> 	// ------------------------------------------------------------------------
> 	public static void update(java.sql.Connection c, String cmd)
> 		throws java.sql.SQLException {
> 		c.createStatement().execute(cmd);
> 	};
> 	// ------------------------------------------------------------------------
> 	public static final int entries=100000;
> 	public static void fill_db(java.sql.Connection c)
> 		throws java.sql.SQLException {
> 		try { update(c,"drop table mail_raw"); } catch (Exception e) {};
> 		try { update(c,"create table mail_raw ( msg varchar(999), att varchar(100), val clob
)"); } 
> 		catch (Exception e) { log_stacktrace(e); };
> 		java.sql.PreparedStatement pstmt = null;
> 		try { pstmt = c.prepareStatement("insert into mail_raw values ( ?, ?, ? )"); }
> 		catch (Exception e) { log_stacktrace(e); };
> 		for (int i=0; i<entries; i++) {
> 			pstmt.setString(1,make_string(3));
> 			pstmt.setString(2,"5 body");
> 			setLob(pstmt,3,make_string(4));
> 			pstmt.executeUpdate();
> 			if (i%100==0) log("step "+i+"/"+entries);
> 		};
> 	};
> 	// ------------------------------------------------------------------------
> 	public static void dump_db(java.sql.Connection c)
> 		throws java.sql.SQLException {
> 		log("performing dump.");
> 		java.sql.ResultSet rs = c.createStatement().executeQuery("select * from mail_raw");
> 		while (rs.next()) {
> 			String msg = rs.getString(1);
> 			String att = rs.getString(2);
> 			String val = getLob(rs,3);
> 		};
> 	};
> 	// ------------------------------------------------------------------------
> 	public static void main(String args[]) {
> 		start_server();
> 		try {
> 			java.sql.Connection c = con();
> 			fill_db(c);
> 			dump_db(c);
> 		} catch (Exception e) { log_stacktrace(e); };
> 	};
> 	// ------------------------------------------------------------------------
> };

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message