Mailing-List: contact dev-help@jackrabbit.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@jackrabbit.apache.org
Received-SPF: pass (athena.apache.org: domain of jukka.zitting@gmail.com
 designates 66.249.82.232 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=message-id:date:from:to:subject:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references;
        b=GmrsnOrtUWA9Qu/I6drqu5DcOwErT6hb39Rwwti/oNDh8svO1TzWCuDrwCz72wV1OGfPJeDwjdI1tntOLfKRus3+rZKWRrv54zIxv+1t2yGYtGrizaK2pH7JzJuLG4oRh7wvYj06IvM+b6byxY/vnbCKD6V5VPXit3bePiTI++Q=
Message-ID: <510143ac0805160855m1f7d2522r9eff27379b911a91@mail.gmail.com>
Date: Fri, 16 May 2008 18:55:49 +0300
From: "Jukka Zitting" <jukka.zitting@gmail.com>
To: dev@jackrabbit.apache.org
Subject: Re: Memory use of item states
In-Reply-To: <90a8d1c00805160807wc2f49eer78cd793802e59392@mail.gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit
Content-Disposition: inline
References: <510143ac0805160644q6d90404r2f46cb79169c589@mail.gmail.com>
	 <90a8d1c00805160807wc2f49eer78cd793802e59392@mail.gmail.com>

Hi,

On Fri, May 16, 2008 at 6:07 PM, Stefan Guggisberg
<stefan.guggisberg@day.com> wrote:
> On Fri, May 16, 2008 at 3:44 PM, Jukka Zitting <jukka.zitting@gmail.com> wrote:
>> * Remove memorized hashCodes from item ID's: 8MB ~ 3%
>> * Make NodeState.propertyNames a sorted list instead of a HashSet: 18MB ~ 7%
>
> there's the classic trade off of memory usage vs. performance.
> while the above modifications reduce memory usage they
> will negatively impact performance.

Yep.

> the amount of memory saved is significant, we'll have to find out
> how much it impacts (read) performance.

The HashSet could be fairly easily optimized for memory without a
speed impact, since the java.util implementation just wraps a HashMap
that's quite a verbose way to store a set. A single array acting as
the hash table that points directly to the objects contained in the
set should be as fast (or even slightly faster) than HashMap while
using noticeably less memory. Of course it's extra code, so perhaps
something to propose for commons-collections.

>> * Use a single InternalValue object instead of an array for
>> single-valued properties: 17MB ~ 7%
>
> impressive. i had chosen arrays in order to keep the code
> simple, i.e. avoid stmts like
>
> if (singleValued) { ... } else { ... }.
>
> but i guess that's not worth the extra memory overhead...

An array with a single reference is something like 16-20 bytes, so it
adds up. There might be some way to cleanly avoid the if statements.

>> * Use the underlying value objects directly instead of the
>> InternalValue wrappers in PropertyState: 18MB ~ 7%
>
> hmm, i'd be interested to see what exactly you've done
> to achieve this. don't you end up with long nasty
> switch/case stmts?

I had to hack InternalValue a bit, basically add a public
InternalValue(Object data, int type) constructor so I could transform
back and forth between the data objects and InternalValues containing
them. Note that the type is already contained in PropertyState.

Again, it seems like the InternalValue overhead is 16-20 bytes per
value, which is not too bad, but starts to show in aggregate.

BR,

Jukka Zitting