Return-Path: Delivered-To: apmail-ibatis-user-java-archive@www.apache.org Received: (qmail 14870 invoked from network); 22 Jul 2005 14:02:21 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (209.237.227.199) by minotaur.apache.org with SMTP; 22 Jul 2005 14:02:21 -0000 Received: (qmail 55274 invoked by uid 500); 22 Jul 2005 14:02:16 -0000 Delivered-To: apmail-ibatis-user-java-archive@ibatis.apache.org Received: (qmail 55205 invoked by uid 500); 22 Jul 2005 14:02:15 -0000 Mailing-List: contact user-java-help@ibatis.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user-java@ibatis.apache.org Delivered-To: mailing list user-java@ibatis.apache.org Received: (qmail 55138 invoked by uid 99); 22 Jul 2005 14:02:15 -0000 Received: from asf.osuosl.org (HELO asf.osuosl.org) (140.211.166.49) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 22 Jul 2005 07:02:15 -0700 X-ASF-Spam-Status: No, hits=0.1 required=10.0 tests=HTML_60_70,HTML_MESSAGE,RCVD_BY_IP,SPF_HELO_PASS,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (asf.osuosl.org: domain of clinton.begin@gmail.com designates 64.233.184.198 as permitted sender) Received: from [64.233.184.198] (HELO wproxy.gmail.com) (64.233.184.198) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 22 Jul 2005 07:02:07 -0700 Received: by wproxy.gmail.com with SMTP id 71so336217wra for ; Fri, 22 Jul 2005 07:02:11 -0700 (PDT) DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:message-id:date:from:reply-to:to:subject:in-reply-to:mime-version:content-type:references; b=QaDGCRQ8uE2Ra0d5aZBqDFTdTiW6b9Ep64Dn9N4DbbPPtwFBoS/LohN4KphUaX9BDLVwHGYw9iow8PEKbEQaT0rkyEx9Y7evoLiQ74mgA2KUxL/jyBDB7ePp5D09oOyYUiTli9QDkXOhgIabFOwBAx2QXyo/DfPpQa3dC8XKSe4= Received: by 10.54.37.72 with SMTP id k72mr1215811wrk; Fri, 22 Jul 2005 07:02:11 -0700 (PDT) Received: by 10.54.93.11 with HTTP; Fri, 22 Jul 2005 07:02:11 -0700 (PDT) Message-ID: <16178eb105072207026f6d4fcc@mail.gmail.com> Date: Fri, 22 Jul 2005 08:02:11 -0600 From: Clinton Begin Reply-To: cbegin@ibatis.com To: user-java@ibatis.apache.org Subject: Re: Estimating amount of memory need for cache In-Reply-To: <00d801c58ec4$581d8130$72aafe96@polygonatum> Mime-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_Part_1720_32763513.1122040931177" References: <8A9C57F4-4F4D-48B4-B555-7B65D5EA6766@sun.com> <16178eb10507212019addf2d1@mail.gmail.com> <007501c58ea6$14bd41f0$72aafe96@polygonatum> <16178eb10507220615b1584a6@mail.gmail.com> <00c201c58ec0$d44369e0$72aafe96@polygonatum> <16178eb1050722062624745600@mail.gmail.com> <00d801c58ec4$581d8130$72aafe96@polygonatum> X-Virus-Checked: Checked by ClamAV on apache.org X-Spam-Rating: minotaur.apache.org 1.6.2 0/1000/N ------=_Part_1720_32763513.1122040931177 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Content-Disposition: inline You can keep a running set of instance references to ensure that you don't= =20 count the same object twice. Cheers, Clinton On 7/22/05, Michal Malecki wrote: >=20 > I can think about it during weekend (at last) :) first - two general=20 > questions: > 1) this sizeof method would return size of object and all objects that ar= e=20 > referenced by it; so invoking it in the loop for array and adding the res= ult=20 > is not good because some objects can be referenced many times > 2)there is no problem with primitives and strings :) but what we do with= =20 > all other classes; it it enough to check all attributes by reflection=20 > (probably invoking setAccessible) until we have only primitives and array= s=20 > of primitives? > As I read this article in the JavaWorld (I have never time for that :()= =20 > this sizeof is very tricky :( So some simplyfying assumptions must be mad= e > More ideas after weekend > Michal Malecki >=20 > ----- Original Message -----=20 > *From:* Clinton Begin =20 > *To:* user-java@ibatis.apache.org=20 > *Sent:* Friday, July 22, 2005 3:26 PM > *Subject:* Re: Estimating amount of memory need for cache >=20 >=20 > Good idea Michal, >=20 > Sounds like something useful that could be added to Commons Lang. Would= =20 > you have an interest in contributing it, or are you going to make me writ= e=20 > it? ;-) >=20 > Cheers, > Clinton >=20 > On 7/22/05, Michal Malecki wrote:=20 > >=20 > > I think that in many cases estimating memory taken maybe quite easy=20 > > (when there are no circual references). Then it's ease to write utility= that=20 > > would analyze object using reflection or to implement interface that ha= s=20 > > size() method; If you know structure of the bean and there are well kno= wn=20 > > types (strings, primitives maybe dates/timestamps) estimating size is a= s=20 > > easy as writing hashCode :) > > Michal Malecki > > =20 > > ----- Original Message -----=20 > > *From:* Clinton Begin =20 > > *To:* user-java@ibatis.apache.org=20 > > *Sent:* Friday, July 22, 2005 3:15 PM=20 > > *Subject:* Re: Estimating amount of memory need for cache > >=20 > > Thanks for the info Michal, > >=20 > > I suppose then you'd have to double expected size of each string in you= r=20 > > model then, to make the estimate safer. > >=20 > > Too bad there isn't something along the lines of System.sizeOf (Object= =20 > > obj).=20 > >=20 > > Cheers, > > Clinton > >=20 > > On 7/22/05, Michal Malecki wrote:=20 > > >=20 > > > Hello, > > > I think it's not quite so e.g. all characters in the memory are 2=20 > > > bytes and serialized there are in UTF-8 > > > here is article about it: > > > http://www.javaworld.com/javaworld/javaqa/2003-12/02-qa-1226-sizeof.h= tml=20 > > >=20 > > > Regards > > > Michal Malecki > > > =20 > > >=20 > > > You could serialize the results it to disk. That would give you a=20 > > > pessimistic estimation of the size it will be in memory. That is, the= cost=20 > > > of serialization will be higher than the amount of RAM used, so you c= an feel=20 > > > relatively confident that it will be smaller in memory. > > >=20 > > > Cheers, > > > Clinton > > >=20 > > > On 7/21/05, Nathan Maves wrote:=20 > > > >=20 > > > > What is the best and most accurate way to calculate the space neede= d > > > > to cache a query. > > > >=20 > > > > Say I have a query that returns 5 strings on average of 20 chars=20 > > > > each. > > > >=20 > > > > The query returns 5000 rows. > > > >=20 > > > > I know this seems high but just humor me and remember I have access= =20 > > > > to big servers here at Sun :) > > > >=20 > > > > Nathan > > > >=20 > > >=20 > > >=20 > >=20 > ------=_Part_1720_32763513.1122040931177 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Content-Disposition: inline
You can keep a running set of instance references to ensure that you don't = count the same object twice.

Cheers,
Clinton

On 7/22/05, Michal Malecki <misiek_x@poczta.onet.pl> wrote:
I can think about it during weekend (a= t last) :)=20 first  - two general questions:
1) this sizeof method would return siz= e of object=20 and all objects that are referenced by it; so invoking it in the loop for a= rray=20 and adding the result is not good because some objects can be referenced ma= ny=20 times
2)there is no problem with primitives = and strings=20 :) but what we do with all other classes; it it enough to check all attribu= tes=20 by reflection (probably invoking setAccessible) until we have only primitiv= es=20 and arrays of primitives?
 
As I read this article in the JavaWorl= d (I have=20 never time for that :() this sizeof is very tricky :( So some simplyfying= =20 assumptions must be made
 
More ideas after weekend
Michal Malecki
----- Original Message -----
From:=20 Clinton=20 Begin
Sent: Friday, July 22, 2005 3:26 P= M
Subject: Re: Estimating amount of = memory=20 need for cache


Good idea Michal,

Sounds like something useful = that=20 could be added to Commons Lang.  Would you have an interest in=20 contributing it, or are you going to make me write it? =20 ;-)

Cheers,
Clinton

On 7/22/05, Michal=20 Malecki <misiek_x@poczt= a.onet.pl>=20 wrote:
I think that in many cases estimat= ing memory=20 taken maybe quite easy (when there are no circual references). The= n=20 it's ease to write utility that would analyze object using reflection o= r to=20 implement interface that has size() method; If you know structure of th= e=20 bean and there are well known types (strings, primitives maybe=20 dates/timestamps)  estimating size is as easy as writing hashCode= =20 :)
Michal=20 Malecki
-----=20 Original Message -----
From:=20 Clinton Begin=20
Sent:=20 Friday, July 22, 2005 3:15 PM
Subject:=20 Re: Estimating amount of memory need for cache

Thanks for the info Michal,

I suppose then= =20 you'd have to double expected size of each string in your model then,= to=20 make the estimate safer.

Too bad there isn't something along t= he=20 lines of System.sizeOf (Object obj). =20

Cheers,
Clinton

On 7/22/05, Michal=20 Malecki <misiek_x@p= oczta.onet.pl> wrote:=20
Hello,
I think it's not quite so e.g.= all=20 characters in the memory are 2 bytes and serialized there are in=20 UTF-8
here is article about it:
Regards
Michal Malecki

You= =20 could serialize the results it to disk.  That would give you= a=20 pessimistic estimation of the size it will be in memory.  Th= at=20 is, the cost of serialization will be higher than the amount of R= AM=20 used, so you can feel relatively confident that it will be smalle= r in=20 memory.

Cheers,
Clinton

On 7/21/05, Nathan Maves <Nath= an.Maves@sun.com > wrote:=20
What=20 is the best and most accurate way to calculate the space=20 needed
to cache a query.

Say I have a query that retu= rns 5=20 strings on average of 20 chars each.

The query returns 5= 000=20 rows.

I know this seems high but just humor me and remem= ber I=20 have access
to big servers here at Sun=20 :)

Nathan



------=_Part_1720_32763513.1122040931177--