Return-Path: X-Original-To: apmail-db-derby-dev-archive@www.apache.org Delivered-To: apmail-db-derby-dev-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 14DEA9ED0 for ; Tue, 12 Jun 2012 15:44:26 +0000 (UTC) Received: (qmail 23685 invoked by uid 500); 12 Jun 2012 15:44:25 -0000 Delivered-To: apmail-db-derby-dev-archive@db.apache.org Received: (qmail 23611 invoked by uid 500); 12 Jun 2012 15:44:25 -0000 Mailing-List: contact derby-dev-help@db.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: Delivered-To: mailing list derby-dev@db.apache.org Received: (qmail 23604 invoked by uid 99); 12 Jun 2012 15:44:25 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 12 Jun 2012 15:44:25 +0000 X-ASF-Spam-Status: No, hits=1.7 required=5.0 tests=FREEMAIL_ENVFROM_END_DIGIT,HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of akssps011@gmail.com designates 209.85.210.44 as permitted sender) Received: from [209.85.210.44] (HELO mail-pz0-f44.google.com) (209.85.210.44) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 12 Jun 2012 15:44:21 +0000 Received: by dacx6 with SMTP id x6so7041275dac.31 for ; Tue, 12 Jun 2012 08:44:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :content-type; bh=0YVceLJJbT9FlMP1BeQ1/oWXtm293ma3oIspPM2kT2A=; b=g2mVM7stNNzPwZeJVc7qd0LnQBvzAz0Gc3KD0H0WdFssaAhFiwgm2f3Hgn5wz1iLMt LPr5AqmZ2GIkvR219hbvBZ9hubesYYuZy5/BNbKg+rfNo+M3oTsIq3QXaEF+BUq1Nj10 ZsAgO8BbO5v42SLW+nfYSJovp3hWTqyNJhAFw2n8xJG4vKGiXzfptxHI0/rGYi6xKwMV plbFfvWSbdhWG4UbvlEeYaWgKyK04J4lNG+YxuA+hjDBzHCRg7kDkA5cs2HJRj/2es1e +k2S5kyX/m2ry8RL/+FgZ8jQDTTRJJEQIQuoOjcm6BPUCX7oJyllw4Uc1rKQeZcsTX5C BGzg== Received: by 10.68.217.40 with SMTP id ov8mr39797838pbc.131.1339515840477; Tue, 12 Jun 2012 08:44:00 -0700 (PDT) MIME-Version: 1.0 Received: by 10.143.83.20 with HTTP; Tue, 12 Jun 2012 08:43:40 -0700 (PDT) In-Reply-To: <4FD70619.6090300@oracle.com> References: <4FD66004.5070803@oracle.com> <4FD68F10.6070106@gmail.com> <4FD70619.6090300@oracle.com> From: siddharth srivastava Date: Tue, 12 Jun 2012 21:13:40 +0530 Message-ID: Subject: Re: Alternative code coverage report using JaCoCo To: derby-dev@db.apache.org Content-Type: multipart/alternative; boundary=e89a8ff24709d7b9a104c2485556 X-Virus-Checked: Checked by ClamAV on apache.org --e89a8ff24709d7b9a104c2485556 Content-Type: text/plain; charset=ISO-8859-1 Hi The coverage report does seem pretty simple and easy to understand. Comparing the jacoco [1] and emma [2] reports for DRDAServerStarter I was wondering if the code coverage results in jacoco are not affected by the Java Security Manager since security manager has been shut off by default for emma's code coverage results. [1]: http://people.apache.org/~kristwaa/jacoco/org.apache.derby.iapi.jdbc/DRDAServerStarter.html [2]: http://dbtg.foundry.sun.com/derby/test/coverage/_files/10c.html On 12 June 2012 14:34, Kristian Waagan wrote: > On 12.06.2012 02:36, Bryan Pendleton wrote: >>> >>> I've been looking into adding support for JaCoCo [1] for Derby >> >> >> Thanks for investigating this, Kristian. >> >> I looked at the report you published, and I found it clear and >> easy to read. >> >> Why would we choose one tool over another? Is there a reason to >> trust the reports produced by one tool more than those produced >> by another? Is it easier to configure or lower-overhead to operate >> one tool versus the other? > > > Hi Bryan, > > Note that we don't have to choose one of the tools - we can use both if we > want to. > When I started this effort we were having problems getting EMMA to produce > results reliably. I think that issue has been resolved now (Knut made > changes to part of our test framework). > > I'd say both tools are just about as easy to configure, but JaCoCo may have > a stronger focus on integration (currently ant, maven, Java agent). > I don't know exactly about overhead, but my gut feeling is they're in the > same league. I've tried other tools that affected performance a lot (i.e. it > took ages to run the tests) and that required a lot of resources (either > during data recording or during report generation). Note that JaCoCo doesn't > have a separate instrumentation/compilation step. > Based on a quick chat with Knut Anders offline the raw result files from > JaCoCo may be somewhat smaller, but the files are so small anyway that it > doesn't matter. > Report generation with JaCoCo is very fast. > I believe both tools are pretty simplistic in what data they record or > derive, but it's enough for our current usage. > > >> >> Or are you just anticipating that Emma will fall into disrepair >> over time? > > > Yes, this is the main reason to look at another tool in my opinion. > With time, EMMA may get into trouble as new versions of Java are released. > > There are also alternative ways to implement code coverage tools with Java > 7, so in the future there may be a better tool available. However, the > current level of overhead seems to be acceptable to me (when used with test > runs). > > > -- > Kristian > >> >> thanks, >> >> bryan >> > -- Regards Siddharth Srivastava --e89a8ff24709d7b9a104c2485556 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Hi

The coverage report does seem pretty simple and easy to understan= d.=A0

Comparing the jacoco [1] and= emma [2] reports for DRDAServerStarter
I was wondering if the code coverage results in jacoco are not affected by = the
Java Security Manager since security manager has been shut off by d= efault for emma's=A0
code coverage results.

[1]: http://people.apache.org/~kristwaa/jacoco/org.apache.derb= y.iapi.jdbc/DRDAServerStarter.html
[2]: http://dbtg.foundry.sun.com/derby/test/coverage/_files/10c.html
On 12 June 2012 14:34, Kristian Waagan <kristian.waagan@oracle.com> wrote:
> On 12.06.2012 02:36, Bryan Pendleton wrote:
>>>
>>= ;> I've been looking into adding support for JaCoCo [1] for Derby>>
>>
>> Thanks for investigating this, Kristian.<= br> >>
>> I looked at the report you published, and I found it c= lear and
>> easy to read.
>>
>> Why would we cho= ose one tool over another? Is there a reason to
>> trust the repor= ts produced by one tool more than those produced
>> by another? Is it easier to configure or lower-overhead to operate=
>> one tool versus the other?
>
>
> Hi Bryan,>
> Note that we don't have to choose one of the tools - we = can use both if we
> want to.
> When I started this effort we were having problems ge= tting EMMA to produce
> results reliably. I think that issue has been= resolved now (Knut made
> changes to part of our test framework). >
> I'd say both tools are just about as easy to configure, bu= t JaCoCo may have
> a stronger focus on integration (currently ant, m= aven, Java agent).
> I don't know exactly about overhead, but my = gut feeling is they're in the
> same league. I've tried other tools that affected performance a lo= t (i.e. it
> took ages to run the tests) and that required a lot of r= esources (either
> during data recording or during report generation)= . Note that JaCoCo doesn't
> have a separate instrumentation/compilation step.
> Based on a q= uick chat with Knut Anders offline the raw result files from
> JaCoCo= may be somewhat smaller, but the files are so small anyway that it
> doesn't matter.
> Report generation with JaCoCo is very fast= .
> I believe both tools are pretty simplistic in what data they reco= rd or
> derive, but it's enough for our current usage.
> >
>>
>> Or are you just anticipating that Emma will fa= ll into disrepair
>> over time?
>
>
> Yes, this = is the main reason to look at another tool in my opinion.
> With time= , EMMA may get into trouble as new versions of Java are released.
>
> There are also alternative ways to implement code coverage too= ls with Java
> 7, so in the future there may be a better tool availab= le. However, the
> current level of overhead seems to be acceptable t= o me (when used with test
> runs).
>
>
> --
> Kristian
>
>>=
>> thanks,
>>
>> bryan
>>
>
<= br>

--
Regards
Siddharth Srivastava
--e89a8ff24709d7b9a104c2485556--