systemml-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From dusenberr...@gmail.com
Subject Re: [TESTS] What are the best practices for dml script testing. Thanks.
Date Thu, 11 Jan 2018 18:19:12 GMT
Hi Janardhan,

Prithvi’s answer is great.  To add a bit more about the main script, please see the `run_tests.dml`
script in the `nn/test` folder: https://github.com/apache/systemml/tree/master/scripts/nn/test.
 In general, `grad_check.dml` and `test.dml` contain test functions for the gradients and
other types of correctness, respectively.  By convention, each test prints out the name or
phrase that identifies that test, and any failure results in an additional printing of a phrase
beginning with `ERROR: ....`.  Thus, when the tests are run, one can clearly see whether or
not any tests have failed.  In terms of actually running the tests with a main script, the
`run_tests.dml` script simply imports the other files and calls every test function.  Therefore,
one only needs to manually run that file in order to run all of the `nn` tests.

At some point in the past, we ended up also creating a junit test that simply runs `run_tests.dml`
and checks for the presence of any `ERROR:` messages in the output.  That way, the `nn` tests
are also run daily alongside all of the classic java tests.

For the FM algorithm, the layers could be added directly into the `nn/layers` directory, and
thus you could add the associated gradient tests and any other correctness tests directly
into the existing `grad_check.dml` and `test.dml` files, and call those functions in `run_tests.dml`.

-Mike

--

Mike Dusenberry
GitHub: github.com/dusenberrymw
LinkedIn: linkedin.com/in/mikedusenberry

Sent from my iPhone.


> On Jan 11, 2018, at 9:35 AM, Prithviraj Sen <senp@us.ibm.com> wrote:
> 
> Hi Janardhan
> 
> SystemML allows scripts to import other scripts. So its possible for your test script
to import your script and test portions of it (assuming things are appropriately function-ified).
> 
> If your question is specifically to check gradients of a certain objective, then you
may want to follow the recipe of 'grad_check.dml'. In general, its a good idea to test out
smaller portions (gradients or otherwise) of a large dml script in a piecemeal manner.
> 
> regards
> prithvi
> 
> 
> Janardhan Pulivarthi ---01/11/2018 09:23:39 AM---Hi prithvirajsen, If a script contains
the gradients, then we could compare them with
> 
> From: Janardhan Pulivarthi <janardhan.pulivarthi@gmail.com>
> To: senp@us.ibm.com, dev@systemml.apache.org
> Date: 01/11/2018 09:23 AM
> Subject: [TESTS] What are the best practices for dml script testing. Thanks.
> 
> 
> 
> 
> Hi prithvirajsen,
> 
> If a script contains the gradients, then we could compare them with numerical gradients,
as we have done in the `grad_check.dml`, file. 
> 
> How can we place a test script with a main script (dml), so that we can be confident
that the script will work. (just like junit test for java files)
> 
> Thanks,
> Janardhan.
> 
> 

Mime
  • Unnamed multipart/alternative (inline, 7-Bit, 0 bytes)
View raw message