madlib-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From GitBox <...@apache.org>
Subject [GitHub] [madlib] fmcquillan99 commented on pull request #518: DL: [AutoML] Add support for Hyperopt on top of MOP for MPP + AutoML best-so-far
Date Mon, 12 Oct 2020 23:31:33 GMT

fmcquillan99 commented on pull request #518:
URL: https://github.com/apache/madlib/pull/518#issuecomment-707396018


   (11)
   ```
   SELECT madlib.madlib_keras_automl('cifar10_train_packed', 
                                     'automl_cifar10_output', 
                                     'model_arch_library', 
                                     'automl_cifar10_mst_table',
                                     ARRAY[2], 
                                     $${'loss': ['categorical_crossentropy'], 
                                        'optimizer_params_list': [ 
                                            {'optimizer': ['Adam'],'lr': [0.0001, 0.01, 'log']},
                                            {'optimizer': ['RMSprop'],'lr': [0.0001, 0.01,
'log'], 'decay': [1.0e-7, 1.0e-5, 'log']},
                                            {'optimizer': ['SGD'],'lr': [0.0001, 0.01, 'log'],
'momentum': [0.9, 0.99,'log']}],
                                        'metrics':['accuracy']}$$, 
                                     $${'batch_size': [128], 'epochs': [1]}$$,
                                     'hyperopt', 
                                     'num_configs=100, num_iterations=3, algorithm=tpe',
                                     NULL,                    -- random state
                                     NULL,                  -- object table
                                     FALSE,                 -- use GPUs
                                     'cifar10_val_packed', -- validation table
                                     1,                     -- metrics compute freq
                                     NULL,                  -- name
                                     NULL);                 -- descr
   ```
   produces
   ```
   select mst_key, metrics_elapsed_time from hopt_cifar10_output_info_c100n3tperun1 order
by mst_key;
    mst_key |                 metrics_elapsed_time                 
   ---------+------------------------------------------------------
          1 | {627.882262945175,1264.80915999413,1896.74943590164}
          2 | {614.573877811432,1251.1022670269,1883.14707994461}
          3 | {638.407788038254,1286.73397421837,1939.58357810974}
          4 | {624.925609111786,1273.55782914162,1925.82674407959}
          5 | {650.984225034714,1307.78145503998,1962.8148560524}
          6 | {637.914520978928,1294.23532605171,1949.12843108177}
          7 | {651.188719987869,1301.32990193367,1948.88206887245}
          8 | {637.525157928467,1288.19901585579,1935.65923500061}
          9 | {651.277820110321,1301.09531402588,1947.13559818268}
         10 | {637.898158073425,1287.54387116432,1933.5972070694}
         11 | {650.482151985168,1309.5760037899,1961.18471598625}
         12 | {637.149097919464,1295.67461585999,1947.66235399246}
         13 | {651.543440818787,1299.3868060112,1948.78301692009}
         14 | {638.292811870575,1285.66767001152,1935.02927088737}
         15 | {650.077238082886,1305.37611913681,1962.8950419426}
         16 | {636.498329162598,1291.3173930645,1948.41972708702}
         17 | {649.946799993515,1303.33863401413,1949.58992981911}
         18 | {636.373380899429,1289.27607584,1936.36742186546}
         19 | {647.414880990982,1297.31931090355,1945.65248298645}
         20 | {634.09289598465,1284.170140028,1932.07456684113}
         21 | {684.188683986664,1330.76613283157,1980.06952404976}
         22 | {670.57110285759,1317.26948404312,1966.71574902534}
         23 | {651.919174909592,1304.4483089447,1949.68165898323}
         24 | {637.834177970886,1290.82985186577,1936.79075098038}
         25 | {655.077679872513,1298.5494530201,1945.88967299461}
         26 | {642.029677867889,1285.51564288139,1932.42474102974}
         27 | {647.519994020462,1291.86638307571,1940.63740706444}
         28 | {634.281761884689,1277.6536450386,1927.27670001984}
         29 | {649.227880954742,1295.78293895721,1939.8649418354}
         30 | {635.665500879288,1282.13676691055,1926.41391181946}
         31 | {646.891736984253,1292.39306688309,1939.45323801041}
         32 | {633.289309024811,1278.57114887238,1925.29498195648}
         33 | {645.044659852982,1294.90844988823,1934.83831381798}
         34 | {631.370570898056,1281.09457278252,1921.84612298012}
         35 | {641.974558115005,1287.15608096123,1933.71887016296}
         36 | {628.698891162872,1273.55222702026,1920.55405902863}
         37 | {645.359504938126,1286.81019496918,1936.24443602562}
         38 | {632.138350963593,1273.69606280327,1921.84107279778}
         39 | {645.965959072113,1291.83859109879,1934.28793001175}
         40 | {631.842108011246,1278.0611550808,1921.37190818787}
         41 | {644.744366168976,1284.78299307823,1926.56299805641}
         42 | {631.000986099243,1271.36041498184,1912.94822907448}
         43 | {646.629382133484,1289.81746912003,1938.30674505234}
         44 | {633.08934211731,1276.87801408768,1924.48914599419}
         45 | {645.794209003448,1289.52689504623,1928.10301995277}
         46 | {632.544068098068,1276.46976709366,1915.18053913116}
         47 | {648.967436075211,1296.18764710426,1942.72390508652}
         48 | {635.180493116379,1282.65127015114,1928.7596950531}
         49 | {642.685960054398,1281.86446094513,1924.27530503273}
         50 | {628.867849111557,1268.81643414497,1910.81892299652}
         51 | {642.25784611702,1285.31040906906,1931.93548893929}
         52 | {628.323985099792,1271.58122897148,1918.26403403282}
         53 | {641.810137033463,1285.25560903549,1929.13687109947}
         54 | {628.287924051285,1271.25157594681,1915.76375198364}
         55 | {645.721904993057,1285.12515211105,1928.89319014549}
         56 | {632.100169181824,1271.49339413643,1915.78533101082}
         57 | {639.617475986481,1275.61342597008,1915.39825177193}
         58 | {626.503052949905,1262.02722096443,1901.32048177719}
         59 | {635.73348903656,1280.25348901749,1919.08957004547}
         60 | {622.708857059479,1266.27361917496,1905.91475009918}
         61 | {640.503421068192,1279.74774312973,1920.75052404404}
         62 | {627.23721909523,1266.51659917831,1906.79211616516}
         63 | {642.081022977829,1277.49766278267,1916.42965888977}
         64 | {628.109483957291,1264.7400097847,1903.20749592781}
         65 | {640.490365028381,1283.00291013718,1924.50952506065}
         66 | {627.058392047882,1269.47951006889,1910.89861011505}
         67 | {636.89220905304,1277.09766602516,1910.96581101418}
         68 | {624.116094112396,1263.90056610107,1898.12311601639}
         69 | {635.469167947769,1269.55181598663,1904.01561403275}
         70 | {622.43970489502,1256.59510493279,1890.07668590546}
         71 | {636.119792938232,1276.15144109726,1913.03098011017}
         72 | {622.38174700737,1262.04434704781,1899.77307701111}
         73 | {635.743231058121,1276.63025021553,1917.14123606682}
         74 | {622.766848087311,1263.39644908905,1903.07419419289}
         75 | {630.598103046417,1267.86923599243,1899.72781896591}
         76 | {617.88782787323,1254.63754701614,1886.25667786598}
         77 | {632.587329149246,1270.71484613419,1902.50029301643}
         78 | {619.189999103546,1257.35159301758,1889.55543804169}
         79 | {631.962325811386,1268.57928681374,1901.31368494034}
         80 | {618.959751844406,1255.32190680504,1888.31697797775}
         81 | {634.275179862976,1261.66588997841,1895.5607509613}
         82 | {619.90931391716,1248.77173399925,1882.48632383347}
         83 | {633.14914894104,1265.85549592972,1897.17652893066}
         84 | {619.785959005356,1252.29544901848,1884.09737896919}
         85 | {627.898339033127,1261.44491815567,1893.74317097664}
         86 | {615.068039178848,1248.09124517441,1879.57727813721}
         87 | {633.1245470047,1268.63815689087,1904.82987594604}
         88 | {619.971349954605,1255.54224300385,1891.63547801971}
         89 | {634.308418035507,1267.44264101982,1903.04254603386}
         90 | {621.244497060776,1254.47942519188,1889.07282710075}
         91 | {638.813256025314,1286.2553050518,1923.13115119934}
         92 | {624.587430000305,1271.47850799561,1909.81140208244}
         93 | {635.323696136475,1271.31673598289,1910.10420203209}
         94 | {622.143538951874,1258.46868395805,1896.75948596001}
         95 | {636.06889295578,1269.57318091393,1905.99937295914}
         96 | {621.923580884933,1255.54979085922,1892.35493183136}
         97 | {635.867208957672,1275.91143798828,1907.85138106346}
         98 | {622.153614997864,1262.45370602608,1895.00853896141}
         99 | {632.058423995972,1264.59007382393,1902.03439188004}
        100 | {618.850169897079,1251.59025883675,1889.0108628273}
   (100 rows)
   ```
   These times should be cumulative, not relative to when the trial started.  We want to be
able to do a plot that looks like:
   
   ![image](https://user-images.githubusercontent.com/10538173/95799055-5846b500-0ca8-11eb-9f3a-b2dc35b416d0.png)
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



Mime
View raw message