activemq-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Gary Tully <gary.tu...@gmail.com>
Subject Re: KahaDB disk utilization
Date Wed, 26 Sep 2012 20:16:37 GMT
maybe work with a 5.7-SNAPSHOT for the moment, work on 5.7 is nearly
complete, so you can expect a release candidate early next week.

On 26 September 2012 18:23, DV <vindimy@gmail.com> wrote:
> Though I really like the idea of mKahaDB, it's unusable to us right
> now because of the bug described in AMQ-3841 - we frequently encounter
> it with AMQ 5.6 and the fix isn't due until 5.7 is out :/
>
> - DV
>
> On Wed, Sep 26, 2012 at 2:54 AM, Claus Ibsen <claus.ibsen@gmail.com> wrote:
>> Hi
>>
>>
>>
>> On Wed, Sep 26, 2012 at 5:14 AM, DV <vindimy@gmail.com> wrote:
>>> Are you absolutely certain that there are exactly 10M messages on the
>>> broker? I've seen a huge KahaDB store with only a few messages in the
>>> queue because the earlier published messages weren't de-queued
>>> ("stuck" messages).
>>>
>>
>> Yes and you can use multi KahaDB to distribute destinations in different files
>> which can help reduce disk space. You can read a bit about it here:
>> http://activemq.apache.org/kahadb.html
>>
>>
>>> On the other hand, maybe KahaDB sacrifices space for performance, kind
>>> of like MongoDB does.
>>>
>>> - DV
>>>
>>> On Wed, Sep 19, 2012 at 8:29 AM, Oleg Dulin <oleg.dulin@gmail.com> wrote:
>>>> Dear Distinguished Colleagues:
>>>>
>>>> I've been trying to understand something.
>>>>
>>>> My queues take Java object messages. Serialized into bytes, each Java object
>>>> is about 1.5K on average.
>>>>
>>>> When I have a backlog of 10 million objects on one of the queues, the disk
>>>> utilization of kahadb is around 50gigs. But this math here:
>>>>
>>>> groovy:000> 1500*10000000/1024/1024/1024;
>>>> ===> 1.9698386192
>>>>
>>>> means that 10M objects should only take up less than 2gigs.
>>>>
>>>> Even if I had 3 queues going with the same objects, the total would be 6gigs
>>>> max. So why the disk utilization overhead ? How much does a single JMS
>>>> message need ?
>>>>
>>>>
>>>> --
>>>> Regards,
>>>> Oleg Dulin
>>>> NYC Java Big Data Engineer
>>>> http://www.olegdulin.com/
>>>>
>>>>
>>>
>>>
>>>
>>> --
>>> Best regards, Dmitriy V.
>>
>>
>>
>> --
>> Claus Ibsen
>> -----------------
>> Red Hat, Inc.
>> FuseSource is now part of Red Hat
>> Email: cibsen@redhat.com
>> Web: http://fusesource.com
>> Twitter: davsclaus
>> Blog: http://davsclaus.com
>> Author of Camel in Action: http://www.manning.com/ibsen
>
>
>
> --
> Best regards, Dmitriy V.



-- 
http://fusesource.com
http://blog.garytully.com

Mime
View raw message