madlib-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From GitBox <...@apache.org>
Subject [GitHub] [madlib] njayaram2 commented on a change in pull request #363: Add tests for madlib_keras_predict
Date Wed, 03 Apr 2019 23:31:16 GMT
njayaram2 commented on a change in pull request #363: Add tests for madlib_keras_predict
URL: https://github.com/apache/madlib/pull/363#discussion_r271970651
 
 

 ##########
 File path: src/ports/postgres/modules/deep_learning/test/madlib_keras.sql_in
 ##########
 @@ -173,3 +172,36 @@ SELECT madlib_keras_predict(
     'x',
     $$ optimizer=SGD(lr=0.01, decay=1e-6, nesterov=True), loss='categorical_crossentropy',
metrics=['accuracy']$$::text,
     'cifar10_predict');
+
+-- Validate that prediction output table exists and has correct schema
+SELECT assert(UPPER(atttypid::regtype::TEXT) = 'INTEGER', 'id column should be INTEGER type')
+    FROM pg_attribute WHERE attrelid = 'cifar10_predict'::regclass
+        AND attname = 'id';
+
+SELECT assert(UPPER(atttypid::regtype::TEXT) =
+    'DOUBLE PRECISION', 'prediction column should be DOUBLE PRECISION type')
+    FROM pg_attribute WHERE attrelid = 'cifar10_predict'::regclass
+        AND attname = 'prediction';
+
+-- Validate correct number of rows returned.
+SELECT assert(COUNT(*) = 2, "Output table of madlib_keras_predict should have two rows")
FROM cifar10_predict;
+
+-- First test that all values are in set of class values; if this breaks, it's definitely
a problem.
+SELECT assert(prediction in (0,1),'Predicted value not in set of defined class values for
model') FROM cifar10_predict;
+
+-- Then test that each of the two images is correctly predicted.  If this breaks, it's likely
a different problem, or
+   -- possibly due to the test model we chose not being trained well enough.  Based on current
testing, it looks
+   -- reliable.
+SELECT assert(prediction=0,'Predicted value not in set of defined class values for model')
FROM cifar10_predict WHERE id=1;
+SELECT assert(prediction=1,'Predicted value not in set of defined class values for model')
FROM cifar10_predict WHERE id=2;
 
 Review comment:
   I actually tried running it multiple times, and it did have different predictions. The
assert failed some times.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

Mime
View raw message