Mailing-List: contact users-help@activemq.apache.org; run by ezmlm
Precedence: bulk
Reply-To: users@activemq.apache.org
Subject: Re: kahadb recovery
To: users@activemq.apache.org
References: <91cdff26-2237-a5cd-b77e-4a038b139405@futurelogix.co.uk>
From: Steve Hill <shill@futurelogix.co.uk>
Message-ID: <4ab9fa05-06a9-5485-43be-b4a4e91a0ff6@futurelogix.co.uk>
Date: Sat, 22 Apr 2017 17:56:53 -0400
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.12; rv:45.0)
 Gecko/20100101 Thunderbird/45.8.0
MIME-Version: 1.0
In-Reply-To: <91cdff26-2237-a5cd-b77e-4a038b139405@futurelogix.co.uk>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 7bit
archived-at: Sat, 22 Apr 2017 21:57:04 -0000

We now realize that the number after the @ is the file number being 
processed and after the colon is the offset in the file.  Eg, 
97226:18610005 means db-97226.log is being processed.  Based on this at 
least we now know how far we are through the startup.  It looks like it 
will take a further 2 hours to start, with a total startup time of 8 
hours - this feels somewhat excessive!

On 4/22/17 4:53 PM, Steve Hill wrote:
> Hi!
>
> Can anyone provide guidance on why restarting ActiveMQ can take 
> hours?  We were seeing lots of message not found in sequence and it is 
> currently at 5.9M entries recovered.  Is there a way to know how many 
> entries there are so we have an idea when it might finish?
>
> There are currently about 1600 files in the kahadb data directory and 
> the total size is 30GB of what about 700mb is the db.data file.
>
>
> 2017-04-22 16:28:21,079 | DEBUG | message not found in sequence id 
> index: ID:atas01p.x.com-57366-1491745186855-3:1:20207097:1:1 | 
> org.apache.activemq.store.kahadb.MessageDatabase | main
> 2017-04-22 16:28:21,082 | DEBUG | message not found in sequence id 
> index: ID:atas01p.x.com-57366-1491745186855-3:1:20207096:1:1 | 
> org.apache.activemq.store.kahadb.MessageDatabase | main
> 2017-04-22 16:28:21,187 | DEBUG | message not found in sequence id 
> index: ID:atas01p.x.com-57366-1491745186855-3:1:20462448:1:1 | 
> org.apache.activemq.store.kahadb.MessageDatabase | main
> 2017-04-22 16:30:56,681 | INFO  | @97201:5488429, 5600000 entries 
> recovered .. | org.apache.activemq.store.kahadb.MessageDatabase | main
> 2017-04-22 16:36:17,337 | INFO  | @97210:5155402, 5700000 entries 
> recovered .. | org.apache.activemq.store.kahadb.MessageDatabase | main
> 2017-04-22 16:41:45,794 | INFO  | @97217:11700458, 5800000 entries 
> recovered .. | org.apache.activemq.store.kahadb.MessageDatabase | main
> 2017-04-22 16:47:19,324 | INFO  | @97226:18610005, 5900000 entries 
> recovered .. | org.apache.activemq.store.kahadb.MessageDatabase | main
>
> Any guidance would be much appreciated.
>
> Thanks!
>
> Steve.
>