Mailing-List: contact derby-user-help@db.apache.org; run by ezmlm
Precedence: bulk
Reply-To: "Derby Discussion" <derby-user@db.apache.org>
Received-SPF: pass (asf.osuosl.org: domain of MZSanford@aol.com designates
 205.188.144.208 as permitted sender)
Message-ID: <42EA7119.7040501@aol.com>
Date: Fri, 29 Jul 2005 11:10:33 -0700
From: Matt Sanford <mzsanford@aol.com>
User-Agent: Mozilla Thunderbird 1.0.2 (Windows/20050317)
MIME-Version: 1.0
To: Derby Discussion <derby-user@db.apache.org>
Subject: Re: Cannot flush the log file to disk error
References: <42E9423B.5080009@aol.com> <42E95E68.9040105@gmail.com>
In-Reply-To: <42E95E68.9040105@gmail.com>
Content-Type: text/plain; charset=windows-1252; format=flowed
Content-Transfer-Encoding: 8bit

Thanks for the speedy assistance. I added code to display the next 
SQLException via getNextMessage() and got back :

java.nio.channels.ClosedByInterruptException
at 
java.nio.channels.spi.AbstractInterruptibleChannel.end(AbstractInterruptibleChannel.java:184)
at sun.nio.ch.FileChannelImpl.force(FileChannelImpl.java:359)
at org.apache.derby.impl.io.DirRandomAccessFile4.sync(Unknown Source)
at 
org.apache.derby.impl.store.raw.log.LogAccessFile.syncLogAccessFile(Unknown 
Source)
at org.apache.derby.impl.store.raw.log.LogToFile.flush(Unknown Source)
at org.apache.derby.impl.store.raw.log.LogToFile.flush(Unknown Source)
at org.apache.derby.impl.store.raw.log.FileLogger.flush(Unknown Source)
at org.apache.derby.impl.store.raw.xact.Xact.prepareCommit(Unknown Source)
at org.apache.derby.impl.store.raw.xact.Xact.commit(Unknown Source)
at org.apache.derby.impl.store.raw.xact.Xact.commit(Unknown Source)
at org.apache.derby.impl.store.access.RAMTransaction.commit(Unknown Source)
at 
org.apache.derby.impl.sql.conn.GenericLanguageConnectionContext.doCommit(Unknown 
Source)
at 
org.apache.derby.impl.sql.conn.GenericLanguageConnectionContext.userCommit(Unknown 
Source)
at org.apache.derby.impl.jdbc.TransactionResourceImpl.commit(Unknown Source)
at org.apache.derby.impl.jdbc.EmbedConnection.commit(Unknown Source)
...
...

Based on the info in the JDK javadocs it looks like I may have some 
concurrent access somewhere. I will try and make a smallest-possible 
test scenario and see if I can remove the problem.

Thanks again;
-- Matt Sanford


suresh.thalamati@gmail.com wrote:

>
> Hi Matt ,
>
> I came across this error only when the disk is full or when there are 
> file system errors. Surprised to find that you don't see log flush SQL 
> Exception in the derby.log. May be the error log is getting 
> reinitialized when you reboot after a shutdown inside your program. 
> You can avoid reinitialization of error log by adding 
> derby.infolog.append=true to the derby.properties.
>
> On the side note , if u are using Derby version before 10.1 , nested 
> exception were not getting logged into the derby.log file in some
> cases. I think log can not flush log exception is also one of them. 
> This issue is fixed in 10.1 ( JIRA ENTRY : DERBY-237) .
>
> If you can not change the derby version , In your program , you may 
> want to print the nested SQL Exceptions, real IO error might show up 
> if you do that.
>
> calling SYSCS_UTIL.SYSCS_CHECKPOINT_DATABASE() can only clean up log 
> files , if the log files are not needed for recovery.
> On rare cases I came across programs that starts a transaction in the 
> beginning and never commits them, when that happens log files can not 
> be reclaimed; because log files will be needed to rollback that 
> particular transaction. This can lead to disk full scenario. Your log 
> file number is small(log10) , so I doubt that is the case!.
>
> You may also want to check the permissions for the files.
>
>
> Thanks
> -suresht
>
> Matt Sanford wrote:
>
>> Hello,
>>
>> I am using and embedded Derby database to replace a large SortedSet 
>> in an RMI server program running on linux[1]. This program is 
>> allowing both submission of new data and reading of data to manage a 
>> queue of work. My program works fine at low load, but at high load I 
>> am getting a SQLException with the message "Cannot flush the log file 
>> to disk /usr/workSetDB/log/log10.dat" (filename varies). All access 
>> to the Derby code is being synchronized to prevent problems with 
>> transactional integrity. I followed the stack trace for the exception 
>> (shown below[2]) but it is unclear to me what would be causing the 
>> problem in the flush method. There are no disk or other errors in the 
>> /var/log/messages file and I have attempted to move the logs dir off 
>> to another device to see if that would help, but no luck.
>
>
>> To try and correct the problem I have also tried a spare thread 
>> calling SYSCS_UTIL.SYSCS_CHECKPOINT_DATABASE() at an interval (1 
>> second and 5 seconds) but that has also not helped the problem. Once 
>> the problem occurs my program shuts down[3] the database and 
>> reconnects using the same code it used during initial startup. On 
>> doing the shutdown and reconnect my code calls setReadOnly(false), 
>> which throws an exception with the message "A read-only user or a 
>> user in a read-only database is not permitted to disable read-only 
>> mode on a connection.". Stopping my program completely and restarting 
>> under a new JVM seems to correct the problem.
>> I have turned up the logging with some derby.properties changes[4], 
>> but the final SQL statement logged is not the insert that failed but 
>> rather the select prior to it and the only error in the log file is 
>> for the read-only problem, not the cause of the initial problem. Has 
>> anyone seen problems with the transaction log flushing under heavy 
>> write load ? I could find almost no reports of these errors when 
>> searching.
>>
>> Thanks;
>> -- Matt Sanford
>>
>> [1] RedHat AS 2.1 with 4 CPUs and 4GB RAM. Using ext3 file systems.
>> Java Version : Sun j2sdk1.4.2_03, using options : -server -Xms1600M 
>> -Xmx1600M -XX:+AggressiveHeap
>>
>> [2] -- Stack Trace --
>> ERROR XSLA0: Cannot flush the log file to disk 
>> /usr/workSetDB/log/log10.dat.
>> at org.apache.derby.iapi.error.StandardException.newException(Unknown 
>> Source)
>> at org.apache.derby.impl.store.raw.log.LogToFile.flush(Unknown Source)
>> at org.apache.derby.impl.store.raw.log.LogToFile.flush(Unknown Source)
>> at org.apache.derby.impl.store.raw.log.FileLogger.flush(Unknown Source)
>> at org.apache.derby.impl.store.raw.xact.Xact.prepareCommit(Unknown 
>> Source)
>> at org.apache.derby.impl.store.raw.xact.Xact.commit(Unknown Source)
>> at org.apache.derby.impl.store.raw.xact.Xact.commit(Unknown Source)
>> at org.apache.derby.impl.store.access.RAMTransaction.commit(Unknown 
>> Source)
>> at 
>> org.apache.derby.impl.sql.conn.GenericLanguageConnectionContext.doCommit(Unknown 
>> Source)
>> at 
>> org.apache.derby.impl.sql.conn.GenericLanguageConnectionContext.userCommit(Unknown 
>> Source)
>> at org.apache.derby.impl.jdbc.TransactionResourceImpl.commit(Unknown 
>> Source)
>> at org.apache.derby.impl.jdbc.EmbedConnection.commit(Unknown Source)
>> at 
>> com.mycode.workset.collection.JDBCPendingWorkCollection.addPendingWork(JDBCPendingWorkCollection.java:182) 
>>
>> ...
>> ...
>>
>>
>> [3] Shutting down with :
>>
>> try {
>> DriverManager.getConnection(myurl + ";shutdown=true");
>> } catch (SQLException e) {
>> // catch the "SQL Exception: Derby system shutdown." message.
>> log.info("Database shutdown complete.");
>> }
>>
>> [4] derby.properties
>>
>> derby.stream.error.file=/tmp/derby.log
>> derby.language.logStatementText=true
>> derby.stream.error.logSeverityLevel=0
>>
>>
>
>

-- 
Even a paranoid has some real enemies.
  �Henry A. Kissinger