Mailing-List: contact dev-help@harmony.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@harmony.apache.org
Received-SPF: pass (herse.apache.org: domain of zhanghuangzhu@gmail.com
 designates 209.85.132.241 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=beta;
        h=received:message-id:date:from:to:subject:in-reply-to:mime-version:content-type:references;
        b=lx0guOqSzDVSKdvzQroXWd8iODzIjldyZ7PsRq+M6YYE4q1347uIUbmDQENNrzNkjjfCxz8D/fKoY0vZR+EOhsrdBqSZunUgSqz8EfsxkI2F8UqXK/sx04rIGpfF4fsqUb+q/wOIQFq4zbAf6Fqtsz9diVc7kuRG9H/yYnfWCYs=
Message-ID: <4d0b24970705220556j7d86665cx104020e27682f88b@mail.gmail.com>
Date: Tue, 22 May 2007 20:56:47 +0800
From: "Andrew Zhang" <zhanghuangzhu@gmail.com>
To: dev@harmony.apache.org
Subject: Re: [performance][benchmark] A tool for writing performance test
In-Reply-To: <0vq4pm5m3ge.fsf@gmail.com>
MIME-Version: 1.0
Content-Type: multipart/alternative;
	boundary="----=_Part_107700_20467708.1179838607602"
References: <4d0b24970705201105n3787f1dcoef7b4502e97e21ae@mail.gmail.com>
	 <253b20230705220515n64a89b00rabc039ad2599c759@mail.gmail.com>
	 <4d0b24970705220525t4697c04eueefd50846f81e297@mail.gmail.com>
	 <0vq4pm5m3ge.fsf@gmail.com>

------=_Part_107700_20467708.1179838607602
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

On 22 May 2007 05:45:53 -0700, Egor Pasko <egor.pasko@gmail.com> wrote:
>
> On the 0x2DE day of Apache Harmony Andrew Zhang wrote:
> > On 5/22/07, Sergey Kuksenko <sergey.kuksenko@gmail.com> wrote:
> > >
> > > Hi Andrew,
> > >
> > > I have a couple questions about punit.
> > >
> > > The first - How do punit calculates amount of used memory?
> > > Is it simple difference (totalMemory-freeMemory)?
> > > Does it take into acoount the amount of used memory before running a
> > > bench?
> > > Also, for example, on you examples for ArrayList amount of used memory
> is
> > > around of 3M.
> > > But for such amount of used memory, obtained values are more related
> to
> > > how
> > > GC works, not the real behavior of ArrayList. What do you think?
> >
> >
> > Hi Sergey,
> >
> > p-unit calls gc before running the test method, and it collects the
> memory
> > consumption every 10ms by default. Yes, it depends on gc, but p-unit
> follows
> > this way because we can not ask our users to call gc in their codes. So
> the
> > memory consumption is the maximum memory consumed during the execution
> of
> > the method.
>
> Andrew, that looks very nice, but induces extra constraints on
> usage. For example, I am now in the middle of writing a set of
> performance tests to test JITted code performance with regards to loop
> versioning optimization. And it appears that disabling GC during JIT
> perf tests is a good idea to get smoother and more predictable
> results, thus, simplifying performance analysis of the single
> component (JIT).
>
> makes sense?


 Hi Sergey,

ah... sorry, I miss something in my previous mail. It returns the relative
value (maximun - the start point).

Is there still any problem with your scenario?

> p-unit is designed to be pluggable, so if you take a look at the code,
> it's
> > all in MemoryWatcher. If you have a good idea how to monitor the memory,
> you
> > can easily write your own and install the watcher to the core. I'm
> really
> > glad to hear some smart ideas about how to monitor memory. :)
>
> Ideally, we could write VM specific methods to estimate eaten
> memory. For example, JIT eats a lot for generated machine-level
> code. We have to take it into account.
>
> > And the second question  -does p-unit have tools/frameworks/api/etc for
> > > creating a multythreaded tests?
> > > Not for purposes to test threading, but for purposes to get higher
> > > processors utilization on multycore multyprocessor machines.
> >
> >
> > ah... good question. I didn't think about it yet. p-unit uses a runner
> to
> > run the test suites. An easy way to do that is use different thread to
> run
> > different test suite. But it may affect the performance data sine you're
> > running other tests at meanwhile?
> >
> > On 5/20/07, Andrew Zhang <zhanghuangzhu@gmail.com> wrote:
> > > >
> > > > Hi all,
> > > >
> > > > I wrote a little tool for performance test called punit[1]. It's a
> > > > framework
> > > > for performance test.
> > > >
> > > > I used punit to run a performance test ArrayList/Vector#add against
> > > > Harmony
> > > > and RI. The result report is very straightforward.
> > > > From the report, you can easliy see that Harmony performs faster in
> > > > ArrayList#add, but consumes much more memory.
> > > > More details, please refer to
> > > >
> > > >
> > >
> http://zhanghuangzhu.blogspot.com/2007/05/use-punit-to-write-performance-test.html
> > > > .
> > > >
> > > > I think if we're interested in the performance test, punit may help
> us
> > > in
> > > > some degree.
> > > > btw, punit supports to run test suite concurrently too. Hopefully,
> you
> > > > like
> > > > it!
> > > >
> > > > [1] punit:  http://p-unit.sourceforge.net/
> > > >
> > > > --
> > > > Best regards,
> > > > Andrew Zhang
> > > >
> > > > http://zhanghuangzhu.blogspot.com/
> > > >
> > >
> > >
> > >
> > > --
> > > Best regards,
> > > ---
> > > Sergey Kuksenko.
> > > Intel Enterprise Solutions Software Division.
> > >
> >
> >
> >
> > --
> > Best regards,
> > Andrew Zhang
> >
> > http://zhanghuangzhu.blogspot.com/
>
> --
> Egor Pasko
>
>


-- 
Best regards,
Andrew Zhang

http://zhanghuangzhu.blogspot.com/

------=_Part_107700_20467708.1179838607602--