Mailing-List: contact dev-help@systemml.incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@systemml.incubator.apache.org
MIME-Version: 1.0
In-Reply-To: <CACs8=aTetT3S5SxKkjLU_GKhFP5QjU+NdhjFqsYwL_Tr3KACfg@mail.gmail.com>
References: <CACs8=aTetT3S5SxKkjLU_GKhFP5QjU+NdhjFqsYwL_Tr3KACfg@mail.gmail.com>
From: Nakul Jindal <nakul02@gmail.com>
Date: Sat, 13 May 2017 22:53:22 -0700
Message-ID: <CAOcw5OzBoPb92+OW6cbgkRMO_zekEzM6MasU36SbWr32PpuUhw@mail.gmail.com>
Subject: Re: SYSTEMML-1451: Performance Regression Tests
To: dev@systemml.incubator.apache.org
Content-Type: multipart/alternative; boundary="001a11439e3e9c5d78054f758fb9"
archived-at: Sun, 14 May 2017 05:53:49 -0000

--001a11439e3e9c5d78054f758fb9
Content-Type: text/plain; charset="UTF-8"

Hi Krishna,

I think the intention is to compare the performance to previous versions of
systemml.

We can add a csv or excel file that contains performance info from previous
versions to the repo and whena performance test is run, new numbers from
the performance run can be added into that csv/excel file and the graph
generated automatically. There are python apis to manipulate excel files,
but I have not looked at what they can and cannot do.

We could also keep the spreadsheet online (on Google docs for instance) and
have the tests update that when the user that runs the perf provides the
appropriate login credentials.

Thoughts?

-Nakul


On Fri, May 12, 2017 at 9:52 PM, Krishna Kalyan <krishnakalyan3@gmail.com>
wrote:

> Hello All,
> I had a questions on SYSTEMML-1451
> (https://issues.apache.org/jira/browse/SYSTEMML-1451)
>
> I am not quite sure how we could achieve the 4th point  (Automatically
> compare this performance to previous version to check for performance
> regressions)
> I was wondering if the expectation is to download an older versions of
> SystemML jar (previous release) from maven repository and run performance
> tests?. Please your initial thoughts on how to go about this?.
>
> Regards,
> Krishna
>

--001a11439e3e9c5d78054f758fb9--