madlib-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From GitBox <...@apache.org>
Subject [GitHub] [madlib] fmcquillan99 commented on pull request #518: DL: [AutoML] Add support for Hyperopt on top of MOP for MPP + AutoML best-so-far
Date Mon, 26 Oct 2020 19:26:55 GMT

fmcquillan99 commented on pull request #518:
URL: https://github.com/apache/madlib/pull/518#issuecomment-716772313


   (1) install of Hyperopt component
   - needs to be installed on master
   - is only needed if hyperopt is called (not if only hyperband is called)
   - I added a note to user docs (next commit)
   
   (2) detailed NOTICE and CONTEXT messages can be supressed
   - OK, supressed
   
   (3) Should not report _3 or _4 or whatever in messages since user does not know what these
are.
   - OK, message fixed
   
   (4) (5) hyperopt is not learning
   - OK, learning now
   
   (6) correctly set number of workers
   - OK, computed correctly now
   
   (7)
   
   (8) confirm loss values reported to hyperopt for all configs (not best from group)
   - OK, confirmed
   
   (9) setting default values
   - OK, set properly now
   
   (10) param naming suggestions
   - OK, fixed naming 
   
   (11) cumulative times
   ```
    mst_key |                                                                            metrics_elapsed_time
                                                                            
   ---------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------
          1 | {3.44599199295044,5.5086510181427,7.59378290176392,9.76283192634583,11.7568578720093,14.0877909660339,16.2628719806671,18.3627188205719,20.6051309108734,22.6256680488586}
          2 | {3.64783596992493,5.74869298934937,7.73203301429749,9.90109395980835,11.9493429660797,14.2302348613739,16.4116108417511,18.5028810501099,20.7483429908752,22.9229199886322}
          3 | {25.3283171653748,27.2733652591705,29.2196431159973,31.2333950996399,33.013738155365,35.0193402767181,36.8298072814941,38.8312292098999,40.8666622638702,42.6906530857086}
          4 | {25.4672672748566,27.4201622009277,29.3609321117401,31.3780810832977,33.1518592834473,35.1660552024841,37.1421611309052,38.971488237381,41.0169200897217,42.8317811489105}
          5 | {45.613331079483,47.5778069496155,49.7638850212097,51.9869689941406,53.9853930473328,56.2250180244446,58.5185329914093,60.7617180347443,63.0875821113586,65.1909580230713}
          6 | {45.7591459751129,47.7180619239807,49.9061949253082,52.1345801353455,54.1592149734497,56.3740320205688,58.6804759502411,60.9120979309082,63.2361199855804,65.3321549892426}
          7 | {68.48650598526,71.0383489131927,73.7156028747559,76.0485279560089,78.6191580295563,81.2393538951874,83.7727980613708,86.2238938808441,88.8117818832397,91.40256690979}
          8 | {68.6526188850403,71.2017469406128,73.8855209350586,76.213299036026,78.8485488891602,81.4080820083618,83.9574830532074,86.3879079818726,88.9709119796753,91.6372621059418}
          9 | {94.9169039726257,97.4595720767975,100.022624015808,102.602681159973,104.947999954224,107.506361961365,110.27393913269,112.921897172928,115.296147108078,117.905650138855}
         10 | {95.0961761474609,97.620619058609,100.199231147766,102.778658151627,105.320446014404,107.670147180557,110.440493106842,113.101521015167,115.461306095123,118.067128181458}
         11 | {121.154954195023,123.238182067871,125.558053016663,127.94621014595,130.058441162109,132.456755161285,134.789571046829,136.911457061768,139.302124023438,141.447726964951}
         12 | {121.302850008011,123.383195161819,125.702442169189,128.096446990967,130.208800077438,132.607058048248,134.943905115128,137.053812026978,139.451186180115,141.589867115021}
         13 | {144.590404987335,146.865453958511,148.973741054535,151.335968017578,153.698374986649,155.985013008118,158.293015003204,160.394205093384,162.702248096466,164.992741107941}
         14 | {144.756483078003,147.041687965393,149.141814947128,151.50594997406,153.962126970291,156.153010129929,158.461601018906,160.565253019333,162.872231960297,165.163074016571}
         15 | {168.281131982803,170.96831202507,173.619775056839,176.261294126511,178.772081136703,181.430242061615,184.221279144287,186.935801029205,189.439113140106,192.158553123474}
         16 | {168.455712080002,171.14265704155,173.794762134552,176.440241098404,178.940042972565,181.715918064117,184.400903940201,187.127205133438,189.61900305748,192.334141969681}
         17 | {195.59405207634,197.74609208107,200.189806938171,202.618988990784,204.730000972748,207.099671125412,209.533874988556,211.708842992783,214.096724033356,216.496123075485}
         18 | {195.76095199585,197.90920996666,200.347979068756,202.780756950378,204.888078927994,207.249115943909,209.746890068054,211.860005140305,214.24716591835,216.653197050095}
         19 | {219.868572473526,222.539808273315,225.132962465286,227.739342451096,230.226921319962,233.084603309631,235.675887346268,238.05983042717,240.733500480652,243.349875450134}
         20 | {220.287011384964,222.718561410904,225.308660268784,227.940581321716,230.405136346817,233.260821342468,235.855934381485,238.236332416534,240.908479452133,243.594557285309}
   (20 rows)
   
   ```
   - OK
   
   (12) metrics_iters missing from info table
   ```
   \d automl_output_info
                Table "public.automl_output_info"
             Column          |        Type        | Modifiers 
   --------------------------+--------------------+-----------
    mst_key                  | integer            | not null
    model_id                 | integer            | 
    compile_params           | text               | 
    fit_params               | text               | 
    model_type               | text               | 
    model_size               | double precision   | 
    metrics_elapsed_time     | double precision[] | 
    metrics_type             | text[]             | 
    loss_type                | text               | 
    training_metrics_final   | double precision   | 
    training_loss_final      | double precision   | 
    training_metrics         | double precision[] | 
    training_loss            | double precision[] | 
    validation_metrics_final | double precision   | 
    validation_loss_final    | double precision   | 
    validation_metrics       | double precision[] | 
    validation_loss          | double precision[] | 
    metrics_iters            | integer[]          | 
   ```
   - OK, added


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



Mime
View raw message