db-derby-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tomohito Nakayama (JIRA)" <derby-...@db.apache.org>
Subject [jira] Commented: (DERBY-550) BLOB : java.lang.OutOfMemoryError with network JDBC driver (org.apache.derby.jdbc.ClientDriver)
Date Thu, 13 Jul 2006 16:26:31 GMT
    [ http://issues.apache.org/jira/browse/DERBY-550?page=comments#action_12420891 ] 

Tomohito Nakayama commented on DERBY-550:
-----------------------------------------

I executed BlobOutOfMem.java with Network Server running on jdb and
tried to recognize circumstance where OutOfMemoryError happens.

The result was as next.

naka@rufelza:~/derby/test/20060714$ startDebugNetworkServer.ksh 
Initializing jdb ...
> catch java.lang.OutOfMemoryError
Deferring all java.lang.OutOfMemoryError.
It will be set after the class is loaded.
> run
run org.apache.derby.drda.NetworkServerControl start -h localhost -p 1527
Set uncaught java.lang.Throwable
Set deferred all java.lang.OutOfMemoryError
Set deferred uncaught java.lang.Throwable
> 
VM Started: Apache Derby Network Server - 10.2.0.4 alpha started and ready to accept connections
on port 1527 at 2006-07-13 15:40:31.278 GMT 

Exception occurred: java.lang.OutOfMemoryError (uncaught)
Exception occurred: java.lang.OutOfMemoryError (uncaught)"thread=DRDAConnThread_3", java.io.ByteArrayOutputStream.<init>(),
line=59 bci=37

DRDAConnThread_3[1] where
  [1] java.io.ByteArrayOutputStream.<init> (ByteArrayOutputStream.java:59)
  [2] org.apache.derby.impl.drda.DDMReader.getExtData (DDMReader.java:958)
  [3] org.apache.derby.impl.drda.DDMReader.getExtData (DDMReader.java:944)
  [4] org.apache.derby.impl.drda.DRDAConnThread.readAndSetExtParam (DRDAConnThread.java:4,355)
  [5] org.apache.derby.impl.drda.DRDAConnThread.readAndSetAllExtParams (DRDAConnThread.java:4,320)
  [6] org.apache.derby.impl.drda.DRDAConnThread.parseEXCSQLSTTobjects (DRDAConnThread.java:3,811)
  [7] org.apache.derby.impl.drda.DRDAConnThread.parseEXCSQLSTT (DRDAConnThread.java:3,640)
  [8] org.apache.derby.impl.drda.DRDAConnThread.processCommands (DRDAConnThread.java:928)
  [9] org.apache.derby.impl.drda.DRDAConnThread.run (DRDAConnThread.java:254)
DRDAConnThread_3[1] up
DRDAConnThread_3[2] list
954    
955    	
956    	if (desiredLength != -1) {
957            // allocate a stream based on a known amount of data
958 =>         baos = new ByteArrayOutputStream ((int) desiredLength);
959    	}
960    	else {
961            // allocate a stream to hold an unknown amount of data
962            baos = new ByteArrayOutputStream ();
963            //isLengthAndNullabilityUnknown = true;
DRDAConnThread_3[2] up
DRDAConnThread_3[3] list
940    	
941    
942    	byte[] getExtData (boolean checkNullability) throws DRDAProtocolException
943    	{
944 => 		return  getExtData(ddmScalarLen, checkNullability);
945    	}
946    
947    
948    	byte[] getExtData (long desiredLength, boolean checkNullability) throws DRDAProtocolException
949      {
DRDAConnThread_3[3] up
DRDAConnThread_3[4] list
4,351    				FdocaConstants.isNullable(drdaType))
4,352    				checkNullability = true;
4,353    	
4,354    			try {	
4,355 => 				byte[] paramBytes = reader.getExtData(checkNullability);
4,356    				String paramString = null;
4,357    				switch (drdaType)
4,358    				{
4,359    					case  DRDAConstants.DRDA_TYPE_LOBBYTES:
4,360    					case  DRDAConstants.DRDA_TYPE_NLOBBYTES:
DRDAConnThread_3[4] 

I confirm this part is where Andreas mentioned in previous comment.

I think this is one of very suspective places.
However I think there may be another place where memory was used in waste.
Because reports from others for memory usage told that amount of used memory was much more
larger than actual  size LOB to be streamed.

> BLOB : java.lang.OutOfMemoryError with network JDBC driver (org.apache.derby.jdbc.ClientDriver)
> -----------------------------------------------------------------------------------------------
>
>          Key: DERBY-550
>          URL: http://issues.apache.org/jira/browse/DERBY-550
>      Project: Derby
>         Type: Bug

>   Components: JDBC, Network Server
>     Versions: 10.1.1.0
>  Environment: Any environment.
>     Reporter: Grégoire Dubois
>     Assignee: Tomohito Nakayama
>  Attachments: BlobOutOfMem.java
>
> Using the org.apache.derby.jdbc.ClientDriver driver to access the
> Derby database through network, the driver is writting all the file into memory (RAM)
before sending
> it to the database.
> Writting small files (smaller than 5Mo) into the database works fine,
> but it is impossible to write big files (40Mo for example, or more), without getting
the
> exception java.lang.OutOfMemoryError.
> The org.apache.derby.jdbc.EmbeddedDriver doesn't have this problem.
> Here follows some code that creates a database, a table, and trys to write a BLOB. 2
parameters are to be changed for the code to work for you : DERBY_DBMS_PATH and FILE
> import NetNoLedge.Configuration.Configs;
> import org.apache.derby.drda.NetworkServerControl;
> import java.net.InetAddress;
> import java.io.*;
> import java.sql.*;
> /**
>  *
>  * @author  greg
>  */
> public class DerbyServer_JDBC_BLOB_test {
>     
>     // The unique instance of DerbyServer in the application.
>     private static DerbyServer_JDBC_BLOB_test derbyServer;
>     
>     private NetworkServerControl server;
>     
>     private static final String DERBY_JDBC_DRIVER = "org.apache.derby.jdbc.ClientDriver";
>     private static final String DERBY_DATABASE_NAME = "Test";
>     
>     // ###############################################################
>     // ############### SET HERE THE EXISTING PATH YOU WANT ################
>     // ###############################################################
>     private static final String DERBY_DBMS_PATH =  "/home/greg/DatabaseTest";
>     // ###############################################################
>     // ###############################################################
>     
>     
>     private static int derbyPort = 9157;
>     private static String userName = "user";
>     private static String userPassword = "password";
>     
>     // ###################################################################################
>     // ############# DEFINE HERE THE PATH TO THE FILE YOU WANT TO WRITE INTO THE DATABASE
###########
>     // ############# TRY A 100kb-3Mb FILE, AND AFTER A 40Mb OR BIGGER FILE #########################
>     // ###################################################################################
>     private static final File FILE = new File("/home/greg/01.jpg");
>     // ###################################################################################
>     // ###################################################################################
>     
>     /**
>      * <p>Used to test the server.
>      */
>     public static void main(String args[]) {
>         try {
>             DerbyServer_JDBC_BLOB_test.launchServer();
>             DerbyServer_JDBC_BLOB_test server = getUniqueInstance();
>             server.start();
>             System.out.println("Server started");
>         
>             // After the server has been started, launch a first connection to the database
to
>             // 1) Create the database if it doesn't exist already,
>             // 2) Create the tables if they don't exist already.            
>             Class.forName(DERBY_JDBC_DRIVER).newInstance();
>             Connection connection = DriverManager.getConnection ("jdbc:derby://localhost:"+derbyPort+"/"+DERBY_DATABASE_NAME+";create=true",
userName, userPassword);
>             System.out.println("Network JDBC connection to Derby succeded. Database created
if not created already.");
>             
>             Statement statement = connection.createStatement(ResultSet.TYPE_SCROLL_INSENSITIVE,
ResultSet.CONCUR_READ_ONLY);
>             Statement statement2;
>             // Create the table "file" if it doesn't already exist.
>             String [] tableNames={"file"};
>             boolean exist;
>             String currentTable;
>             ResultSet result = statement.executeQuery("SELECT TABLENAME FROM SYS.SYSTABLES");
>             for (int i=0;i<tableNames.length;i++) {
>                 exist=false;
>                 while (result.next()){
>                     if (tableNames[i].equalsIgnoreCase(result.getString(1)))
>                         exist=true;
>                 }
>             
>                 if (!exist) {
>                     statement2 = connection.createStatement();
>                     statement2.execute("CREATE TABLE file (" +
>                     "file BLOB(2G) NOT NULL)");
>                     connection.commit();
>                 }
>                 result.beforeFirst();
>             }
>             System.out.println("Table file created if not created already");
>             
>             System.out.println("File insertion into BLOB");
>             FileInputStream inputStream = new FileInputStream(FILE);
>             PreparedStatement   preparedStatement = connection.prepareStatement("INSERT
INTO file(file) VALUES (?)");
>             preparedStatement.setBinaryStream(1,inputStream,(int) FILE.length());
>             preparedStatement .execute();
>             connection.commit();
>             System.out.println("File inserted into BLOB");
>         }
>         catch (Exception e) {
>             e.printStackTrace();
>         }
>     }
>     
>     /** Creates a new instance of MckoiServer
>      * Password is used at the database creation. It will be the database password once
created.
>      */    
>     private DerbyServer_JDBC_BLOB_test() throws Exception {
>         System.setProperty("derby.system.home", DERBY_DBMS_PATH);
>         
>         // Set the server to request an authentification.
>         System.setProperty("derby.authentication.provider", "BUILTIN");
>         System.setProperty("derby.connection.requireAuthentication", "true");
>         
>         // Create a user that can connect to Derby.
>         System.setProperty("derby.user."+userName, userPassword);
>         
>         // Set Derby to grant full access to the created user (to all the databases).
>         System.setProperty("derby.database.fullAccessUsers", userName);
>         
>         //System.setProperty("derby.system.bootAll", "true");
>         
>         // See if the 9157 port is already taken.
>         // Change it if necessary.
>         boolean isPortFree = false;
>         while ( !isPortFree ) {
>             try {
>                 java.net.ServerSocket serverTest = new java.net.ServerSocket(derbyPort);
>                 serverTest.close();
>                 serverTest = null;
>                 
>                 isPortFree = true;
>             }
>             catch (Exception e) {
>                 System.out.println("Port already in use : "+derbyPort);
>                 derbyPort++;
>                 System.out.println("Try with port "+derbyPort);
>             }
>         }     
>         
>         server = new NetworkServerControl(InetAddress.getByName("localhost"),derbyPort);
>     }
>     
>     public static void launchServer() throws Exception {
>         derbyServer = new DerbyServer_JDBC_BLOB_test();
>     }
>     
>     public static DerbyServer_JDBC_BLOB_test getUniqueInstance() {
>         return derbyServer;
>     }
>     
>     /**
>      * <p>Start the server.
>      */
>     public void start() {
>         try {
>             server.start(null);
>         }
>         catch (Exception e) {
>             e.printStackTrace(System.err);
>         }
>     }
>     
>     /**
>      * <p>Stop the server.
>      */
>     public void stop() {
>         try {
>             server.shutdown();
>         }
>         catch (Exception e) {
>             e.printStackTrace(System.err);
>         }
>     }
> }

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
   http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
   http://www.atlassian.com/software/jira


Mime
View raw message