Mailing-List: contact users-help@activemq.apache.org; run by ezmlm
Precedence: bulk
Reply-To: users@activemq.apache.org
Received-SPF: neutral (athena.apache.org: 216.139.236.26 is neither permitted
 nor denied by domain of joe.carter@gmail.com)
Date: Tue, 26 Jul 2011 04:25:28 -0700 (PDT)
From: JoeC <joe.carter@gmail.com>
To: users@activemq.apache.org
Message-ID: <1311679528547-3695392.post@n4.nabble.com>
In-Reply-To: <AANLkTin=bEzOEeCezsXBHpvxgHRArqxdLdsCcTrKBP8N@mail.gmail.com>
References: <AANLkTim_NPyHV-gCzcB8pH2m0eyso2vm25p-_byNWHFP@mail.gmail.com>
 <AANLkTi=AjMsTOKT-RidXhtiHoKfGzz7NAgVVo0NveQF+@mail.gmail.com>
 <AANLkTin=bEzOEeCezsXBHpvxgHRArqxdLdsCcTrKBP8N@mail.gmail.com>
Subject: Re: KahaDB corruption
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit

I'm currently on 5.5.0 and ran into a different and unrecoverable kahadb
case.
I ran the system out of diskspace and not unreasonably activemq didn't like
it.
After freeing up some space I ran into database corruption as follows.
2011-07-26 10:00:23,316 | INFO  | Corrupt journal records found in
'/opt/ivb/apache-activemq-5.5.0/data/kahadb/db-326.log' between offsets:
19460423-21031378 | org.apache.kahadb.journal.Journal | main
...
2011-07-26 10:00:23,826 | INFO  | Recovering from the journal ... |
org.apache.activemq.store.kahadb.MessageDatabase | main
2011-07-26 10:00:23,953 | ERROR | Failed to start ActiveMQ JMS Message
Broker. Reason: org.apache.activemq.protobuf.InvalidProtocolBufferException:
Protocol message contained an invalid tag (zero). |
org.apache.activemq.broker.BrokerService | main

Removing the db.data made no difference.
I then removed the db-326.log file and restarted twice.
The first it complains about not finding db-326.log.
The second time is uses a newly created db-1.log.

Fortunately this was not a production environment, so the data doesn't
matter however I would like a way of recovering the data. This could even be
an offline process.
i.e. I quickly reset the database to restore service and then push in the
older messages later.
My application domain is somewhat tolerant of that approach but it is not
tolerant of extended outages.
For me, I'd rather (temporarily) lose some data than have a long outage so a
fully automated recovery is what I'd ideally like irrespective of
corruption.

Cheers
Joe


JoeC wrote:
> 
> I've upgraded to 5.4.2 and will let you know how it goes.
> I didn't rebuild the index as I've already restarted the process.
> In normal operation the queues should be empty for our application so
> that was not an issue for me.
> 
> Thanks
> Joe
> 
> On 23 February 2011 18:06, Gary Tully &lt;gary.tully@gmail.com&gt; wrote:
>> 5.4.2 is better w.r.t abortive shutdown, but for this case, rebuilding
>> the index should work.
>> remove kahadb/db.data and restart, it will parse the journal to
>> rebuild the index.
>>
> 


--
View this message in context: http://activemq.2283324.n4.nabble.com/KahaDB-corruption-tp3321382p3695392.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.