predictionio-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Kenneth Chan <>
Subject Re: Remove engine registration
Date Sat, 17 Sep 2016 02:51:44 GMT
Pat, would you explain more about the 'instanceId' as in
`pio register --variant path/to/some-engine.json --instanceId
some-REST-compatible-resource-id`  ?

Currently PIO also has a concept of engineInstanceId, which is output of
train. I think you are referring to different thing, right?


On Fri, Sep 16, 2016 at 12:58 PM, Pat Ferrel <> wrote:

> This is a great discussion topic and a great idea.
> However the cons must also be addressed, we will need to do this before
> multi-tenant deploys can happen and the benefits are just as large as
> removing `pio build`
> It would be great to get rid of manifest.json and put all metadata in the
> store with an externally visible id so all parts of the workflow on all
> machines will get the right metadata and any template specific commands
> will run from anywhere on any cluster machine and in any order. All we need
> is a global engine-instance id. This will make engine-instances behave more
> like datasets, which are given permanent ids with `pio app new …` This
> might be a new form of `pio register` and it implies a new optional param
> to pio template specific commands (the instance id) but removes a lot of
> misunderstandings people have and easy mistakes in workflow.
> So workflow would be:
> 1) build with SBT/mvn
> 2) register any time engine.json changes so make the json file an optional
> param to `pio register --variant path/to/some-engine.json --instanceId
> some-REST-compatible-resource-id` the instance could also be
> auto-generated and output or optionally in the engine.json. `pio engine
> list` lists registered instances with instanceId. The path to the binary
> would be put in the instanceId and would be expected to be the same on all
> cluster machines that need it.
> 3) `pio train --instanceId` optional if it’s in engine.json
> 4) `pio deploy --instanceId` optional if it’s in engine.json
> 5) with easily recognized exceptions all the above can happen in any order
> on any cluster machine and from any directory.
> This takes one big step to multi-tenancy since the instance data has an
> externally visible id—call it a REST resource id…
> I bring this up not to confuse the issue but because if we change the
> workflow commands we should avoid doing it often because of the disruption
> it brings.
> On Sep 16, 2016, at 10:42 AM, Donald Szeto <> wrote:
> Hi all,
> I want to start the discussion of removing engine registration. How many
> people actually take advantage of being able to run pio commands everywhere
> outside of an engine template directory? This will be a nontrivial change
> on the operational side so I want to gauge the potential impact to existing
> users.
> Pros:
> - Stateless build. This would work well with many PaaS.
> - Eliminate the "pio build" command once and for all.
> - Ability to use your own build system, i.e. Maven, Ant, Gradle, etc.
> - Potentially better experience with IDE since engine templates no longer
> depends on an SBT plugin.
> Cons:
> - Inability to run pio engine training and deployment commands outside of
> engine template directory.
> - No automatic version matching of PIO binary distribution and artifacts
> version used in the engine template.
> - A less unified user experience: from pio-build-train-deploy to build,
> then pio-train-deploy.
> Regards,
> Donald

View raw message