predictionio-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Adam Drew <>
Subject Info / resources for scaling PIO?
Date Tue, 24 Apr 2018 14:37:35 GMT
Hi all!

Is there any info on how to scale PIO to multiple nodes? I’ve gone through a lot of the
docs on the site and haven’t found anything. I’ve tested PIO running with HBASE and ES
for metadata and events, and with using just ES for both (my preference thusfar) and have
my models on local storage. Would scaling simply be a matter of deploying clustered ES, and
then finding some way to share my model storage, such as NFS or HDFS? The question then is
what (if anything) has to be done for the nodes to “know” about changes on other nodes.
For example, if the model gets trained on node A does node B automatically know about that?

I hope that makes sense. I’m coming to PIO with no prior experience for the underlying apache
bits (spark, hbase / hdfs, etc) so there’s likely things I’m not considering. Any help
/ docs / guidance is appreciated.


View raw message