db-derby-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mike Matrigali <mikem_...@sbcglobal.net>
Subject Re: java.sql.SQLException: Log Corrupted, has invalid data in the log stream.
Date Thu, 16 Nov 2006 23:08:16 GMT


BALDWIN, ALAN J [AG-Contractor/1000] wrote:
> First, thanks for the reply.  I'll try my best to answer all questions.
> 
> Yes, I realize that deleting transaction log files is a bad thing, but
> the alternative is that the user cannot even open the application if we
> don't.  I guess it's worth the user having to re-enter what they did
> last if need be.
> 
> Our 1000 users are 1000 different users using 1000 different instances
> of the same thick client running on their desktops.  We use Java
> Webstart to deploy the swing app to the clients.  All single-process to
> my knowledge.  
thanks, I was wondering if this was some highly multi-threaded log bug.

> 
> This consistency checker you mention... what is it?  Where do I find it?
http://db.apache.org/derby/docs/dev/adminguide/cadminconsist01.html
> 
> No, I have not been able to reproduce it.  We are trying to just gather
> as much about each instance as it happens and try to find a pattern.  I
> will log a JIRA issue to capture as much as I can in one central
> location.  Part of the problem is that our user base is scattered across
> corn fields around the US, and not the most technically savvy folks.
> That, and I usually get information second or third-hand.  This makes it
> hard to get a pattern.
> 
> 
> Your questions [my replies in brackets]:
> 
> As you are seeing this frequently, some questions:
> 1) is it possible that the same database is being accessed from within
>     the same JVM but from 2 different class loaders (ie. DERBY-700)?
> [Very unlikely... there is only one app using the derby jars or our
> database, and from a single classloader within JavaWebstart]
> 
> 2) Is the database located on a disk which can be accessed by 2
>     different machines, and if so is it possible 2 machines are looking
>     at the db using derby?  Note it does not even take 2 updating
>     accesses to cause problems, just the act of booting a db can
>     cause writes to the log file.
> [No, database must be on the user's PC local disk with no external
> access]

ok, we have had problems in the past where a users home directory was
actually some shared filesystem resource.  And could be easily accessed
from multiple machines by the user.  In this case Derby can't coordinate
the proper access to the single db from multiple machines.
> 
> 3) are you running with  derby.system.durability=test?
> [No, I'm not familiar with this property.  Can you elaborate?]
You should not be using this, just wanted to make sure.  you can
search derby docs on the apache site.  I always just use the search box
in this page:  http://db.apache.org/derby/manuals/index.html
> 
> 4) Are the disks where the log files are written set to cache writes,
>     or do they do proper disk sync when requested.  Are they scsi or
>     ide?
> [These are standard IDE or SATA disks 99.9% of the time.]
There is a software option called "cache writes" available on XP on
disks.  When enabled, writes to the disk which have been requested
by the application to not return until they are ensured on disk,
return immediately rather than waiting until they hit the drive.  I
believe this option is often the default.  Derby's ability to recover
after a system crash depends on log data hitting disk when we ask it
to.  One can check the setting of this on XP by going to:
start->settings->control panel->system->hardware->device manager
o find disk drive you are interested in, and double click - go to 
policies tab, last check box option for me is:
[] Enable write caching on the disk
    This setting enables write caching to improve disk performance,
    but a power outage or equipment failure might result in data loss
    or corruption.

In Derby's case this same warning from microsoft applies to writes to
the log file.
> 
> 5) can you post any of the data or the log files, obviously if the
>     data is private in any way you won't be able to put it in JIRA.
> [I will log a JIRA and include as much as I can, xx'ing out urls, etc...
> if I have to]
best chance to understand would be to post an entire database with the
complete set of logs for multiple of these cases.

second best would be at least the posting the set of log files in the
corrupted db.  It is sometimes hard to understand the logs without
the db, but better than nothing.

third would be to at least post complete derby.log from each error.
> 
> 6) Is the log error in derby.log always exactly the same?  If not
> it may help to post multiple derby.log's, again maybe someone will
> see a pattern.  For instance is the nested stack
> in derby.log always say - it would probably be really interesting if
> the number was always 768:
> java.lang.ClassNotFoundException: ERROR XBM0U: No class was registered 
> for identifier 768.
> 
> [I'll try to get as much of a pattern as I can into the JIRA issue]
I'd like to think about what more information derby can dump into the 
derby.log when it encounters this issue, for this it is important to
understand what errors you are seeing.  It will help a lot if you
can initially include as many different derby.log's as you have.  Note
that once you get this error it is likely to reproduce everytime you
boot the same db, so multiple derby.log instances from the same db
are not that interesting.  I always retry the boot at least once to
see if anything changes, but 99.9% you are just going to get the same
error.
> 
> 
> Thanks for everyone's help.
> 
> -Alan-
> 
> 

snipped ...

>>>
>>>I don't see any exceptions being thrown in our application before any
>>
>>of the
>>
>>
>>>corruption cases that would indicate that something in the app had
> 
> gone
> 
>>>awry.  

I am not sure how your app logs this kind of info.  It is likely that 
whatever went wrong happened the LAST time the app shutdown, not this
time.  If at all possible it would be interesting to ask those seeing
the corruptions if anything drastic happened the last time - like OS
crash, machine crash, blue screen, disk error.

Also when your app is finished with derby, to you do a "clean" shutdown
like "shutdown=true".  If so, the fact that we are running through redo
means that it is likely the last time this app shutdown derby it did not 
do a shutdown=true.  Derby should still recover if you don't do a clean
shutdown, but it will save startup time by flushing all dirty cache 
pages to disk and then marking the log so redo is not necessary.  Note
that a clean shutdown with uncommitted, active transactions will result
in redo log records being processed the next time you boot.
>>>
>>>
>>>
>>>Also, attached is my derby.log file.
>>>
>>>
>>>
>>>-Alan-
>>>
>>>
>>>
>>>
>>>
>>>-----Original Message-----
>>>From: BALDWIN, ALAN J [AG-Contractor/1000] 
>>>Sent: Thursday, November 16, 2006 9:42 AM
>>>To: derby-user@db.apache.org
>>>Subject: java.sql.SQLException: Log Corrupted, has invalid data in the
>>
>>log
>>
>>
>>>stream.
>>>
>>>
>>>
>>>Hi all,
>>>
>>>We are having log corruption issues with Derby inside of a java swing
>>>application.
>>>
>>>Here is the error:
>>>
>>>java.sql.SQLException: Log Corrupted, has invalid data in the log
>>
>>stream.
>>
>>
>>>We have about 1000 users using this software, and I would guess that
> 
> at
> 
>>any
>>
>>
>>>given time we see 4-5 with log corruption.  Deleting "logXXX.dat"
> 
> files
> 
>>from
>>
>>
>>>within the log folder fixes the problem for the time being, but I
> 
> would
> 
>>like
>>
>>
>>>to get at the root of the problem.  Is this safe to do as an interim
>>>solution?  I assume these are transaction logs, so it doesn't seem
> 
> like
> 
>>a
>>
>>
>>>good idea to delete them, but that is the only thing that seems to
>>
>>work.
>>
>>
>>>Also, anecdotally, this seems to be more prevalent in version 10.2
>>
>>since we
>>
>>
>>>upgraded from 10.1.3 a few weeks ago.
>>>
>>>I'm also going to dig around on JIRA to seem if I come up with
>>
>>anything.
>>
>>
>>>Any ideas?
>>>
>>>Thanks,
>>>
>>>
>>>
>>
>>
>>
>>
> ------------------------------------------------------------------------
> ---------------------------------
> 
>>This e-mail message may contain privileged and/or confidential
> 
> information, and is intended to be received only by persons entitled to
> receive such information. If you have received this e-mail in error,
> please notify the sender immediately. Please delete it and all
> attachments from any servers, hard drives or any other media. Other use
> of this e-mail by you is strictly prohibited.
> 
>>
>>All e-mails and attachments sent and received are subject to
> 
> monitoring, reading and archival by Monsanto. The recipient of this
> e-mail is solely responsible for checking for the presence of "Viruses"
> or other "Malware". Monsanto accepts no liability for any damage caused
> by any such code transmitted by or accompanying this e-mail or any
> attachment.
> 
> ------------------------------------------------------------------------
> ---------------------------------
> 
>>
>>
> 
> 
> ---------------------------------------------------------------------------------------------------------
> This e-mail message may contain privileged and/or confidential information, and is intended
to be received only by persons entitled to receive such information. If you have received
this e-mail in error, please notify the sender immediately. Please delete it and all attachments
from any servers, hard drives or any other media. Other use of this e-mail by you is strictly
prohibited.
> 
> 
> All e-mails and attachments sent and received are subject to monitoring, reading and
archival by Monsanto. The recipient of this e-mail is solely responsible for checking for
the presence of "Viruses" or other "Malware". Monsanto accepts no liability for any damage
caused by any such code transmitted by or accompanying this e-mail or any attachment.
> ---------------------------------------------------------------------------------------------------------
> 
> 
> 


Mime
View raw message