Mailing-List: contact oak-dev-help@jackrabbit.apache.org; run by ezmlm
Precedence: bulk
Reply-To: oak-dev@jackrabbit.apache.org
Received-SPF: pass (nike.apache.org: domain of chetan.mehrotra@gmail.com
 designates 209.85.214.174 as permitted sender)
MIME-Version: 1.0
In-Reply-To: <CE099B8F.45451%mueller@adobe.com>
References: 
 <CAHCW-mkehKm9P7bzQSje8--gemf0qjKY9+VLkqK8vja9-pZKeQ@mail.gmail.com>
	<CE099B8F.45451%mueller@adobe.com>
Date: Mon, 15 Jul 2013 16:49:54 +0530
Message-ID: 
 <CAHCW-mkbqeaF4DhFgy4sHROF_XEZXxR1KD0eDhma2YPteSoxTQ@mail.gmail.com>
Subject: Re: Very large blobid with MongoMK
From: Chetan Mehrotra <chetan.mehrotra@gmail.com>
To: oak-dev@jackrabbit.apache.org
Content-Type: text/plain; charset=windows-1252
Content-Transfer-Encoding: quoted-printable

> This is not just one blob id, but a list of blob ids. It's 796 blob ids,
> each blob id is 8 KB. Total size is about 6 MB.

Okie its a multi valued property and in OakDirectory  for Lucene the
block size is set to 4092 while the minBlockSize in AbstractBlobStore
is 4096. So effectively all the lucene index blocks are getting
inlined.

Should we tweak either of limits so that these blobs do not get
inlined and instead saved separately otherwise I think this array
would soon outgrow the 16 MB limit?

Chetan Mehrotra


On Mon, Jul 15, 2013 at 4:02 PM, Thomas Mueller <mueller@adobe.com> wrote:
> Hi,
>
> This is not just one blob id, but a list of blob ids. It's 796 blob ids,
> each blob id is 8 KB. Total size is about 6 MB.
>
> Regards,
> Thomas
>
> On 7/15/13 11:46 AM, "Chetan Mehrotra" <chetan.mehrotra@gmail.com> wrote:
>
>>> The blob id shouldn't be larger than a few KB at most (it can be
>>> configured).
>>
>>Yup thats what I was thinking after looking into the
>>org.apache.jackrabbit.mk.blobs.AbstractBlobStore#convertBlobToId.
>>Would have a closer look.
>>
>>Chetan Mehrotra
>>
>>
>>On Mon, Jul 15, 2013 at 2:53 PM, Thomas Mueller <mueller@adobe.com> wrote=
:
>>> Hi,
>>>
>>> The blob id shouldn't be larger than a few KB at most (it can be
>>> configured).
>>>
>>> Regards,
>>> Thomas
>>>
>>>
>>>
>>> On 7/15/13 11:13 AM, "Chetan Mehrotra" <chetan.mehrotra@gmail.com>
>>>wrote:
>>>
>>>>Hi,
>>>>
>>>>I was trying to get an estimate of the size [1] of various nodes
>>>>document in MongoDB for a fresh CQ installation. The largest node was
>>>>for path =8C4:/oak:index/lucene/:data/_5_Lucene41_0.tim =8C weighing
>>>>upward of 6 MB. It has one binary property jcr:data
>>>>
>>>>"jcr:data" : {
>>>>               "r13fd2c82e10-0-1" :
>>>>"[\":blobId:00fc1f3fd76c1715424c4f4.....00031aea1\"]
>>>>}
>>>>
>>>>The value stored above is very large. Before digging in further wanted
>>>>to check if this is expected or the blobid should be something
>>>>smaller?
>>>>
>>>>Regards
>>>>Chetan
>>>>
>>>>[1] var max =3D 0;
>>>>var maxObj
>>>>db.nodes.find().forEach(function(obj) {
>>>>    var curr =3D Object.bsonsize(obj);
>>>>    if(max < curr) {
>>>>        max =3D curr;
>>>>        maxObj =3D obj;
>>>>    }
>>>>})
>>>>print(max);
>>>>printjson(maxObj);
>>>>
>>>>Chetan Mehrotra
>>>
>