predictionio-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Donald Szeto <don...@apache.org>
Subject Re: PIO deploy: old processes still live after redeployment
Date Wed, 11 Apr 2018 22:39:24 GMT
Are you able to find any occurrence of port 8004? The excerpt you showed
was about port 8000.

On Wed, Apr 11, 2018 at 3:18 PM, George Yarish <gyarish@griddynamics.com>
wrote:

> Hi Donald
>
> Thank you for the answer.
> it looks fine to me, but processes still alive:
>
> 2018-04-11 13:00:25,331 INFO  org.apache.predictionio.workflow.MasterActor
> [pio-server-akka.actor.default-dispatcher-5] - Undeploying any existing
> engine instance at http://0.0.0.0:8000
> 2018-04-11 13:00:26,467 INFO  org.apache.predictionio.workflow.MasterActor
> [pio-server-akka.actor.default-dispatcher-9] - Stop server command
> received.
> 2018-04-11 13:00:26,468 INFO  org.apache.predictionio.workflow.MasterActor
> [pio-server-akka.actor.default-dispatcher-9] - Server is shutting down.
> 2018-04-11 13:00:26,617 INFO  spray.can.server.HttpListener
> [pio-server-akka.actor.default-dispatcher-2] - Bound to /0.0.0.0:8000
> 2018-04-11 13:00:26,620 INFO  org.apache.predictionio.workflow.MasterActor
> [pio-server-akka.actor.default-dispatcher-4] - Engine is deployed and
> running. Engine API is live at http://0.0.0.0:8000
>
> Thank you,
> George
>
>
> On Wed, Apr 11, 2018 at 11:48 AM, Donald Szeto <donald@apache.org> wrote:
>
>> Hi George,
>>
>> Mind checking the content of pio.log at where you run `pio deploy`? It
>> should contain some messages regarding undeploying existing engine servers
>> on the same port.
>>
>> Regards,
>> Donald
>>
>> On Thu, Apr 5, 2018 at 6:43 PM, George Yarish <gyarish@griddynamics.com>
>> wrote:
>>
>>> Hi everyone,
>>>
>>> I faced a strange situation recently.
>>> One of our engines has frequently old processes after daily redeployment.
>>>
>>> We used to redeploy by crontab rule that looks like:
>>> nohup pio deploy --port 8004 >> ${DEPLOY_LOG_FILE} 2>&1 &
>>>
>>>
>>> This is output of ps aux command:
>>>
>>> [da_mcom_milan@mdc2pr113 ~]$ ps aux | grep 8004
>>>
>>> da_mcom+  7854  0.0  0.0 112660   996 pts/1    S+   21:34   0:00 grep
>>> --color=auto *8004*
>>>
>>> 1. da_mcom+ 30932  0.6  0.9 26473632 595932 ?     Sl   01:57   7:51 java
>>> -cp /data/PredictionIO-0.11.0-incubating/conf:/data/PredictionIO
>>> -0.11.0-incubating/plugins/*:/data/PredictionIO-0.11.0-incub
>>> ating/lib/spark/*:/data/PredictionIO-0.11.0-incubating/lib/p
>>> io-assembly-0.11.0-incubating.jar:/usr/hdp/2.5.3.0-37/hadoop
>>> /conf:/data/PredictionIO/vendors/hbase/conf
>>> -Dpio.log.dir=/home/da_mcom_milan org.apache.predictionio.tools.console.Console
>>> deploy --port *8004* --pio-home /data/PredictionIO-0.11.0-incubating
>>>
>>> 2. da_mcom+ 31097  1.2  1.2 17065124 822596 ?     Sl   01:57  14:48
>>> /usr/java/default/bin/java -Dhdp.version=2.5.3.0-37 -cp
>>> /data/PredictionIO-0.11.0-incubating/conf/:/usr/hdp/2.5.3.0-
>>> 37/hadoop/conf/:/data/PredictionIO/vendors/hbase/conf/:/usr/
>>> hdp/current/spark-client/conf/:/data/PredictionIO/vendors/sp
>>> ark/lib/spark-assembly-1.6.2.2.5.3.0-37-hadoop2.7.3.2.5.3.0-
>>> 37.jar:/data/PredictionIO/vendors/spark/lib/datanucleus-api-
>>> jdo-3.2.6.jar:/data/PredictionIO/vendors/spark/lib/datanucle
>>> us-core-3.2.10.jar:/data/PredictionIO/vendors/spark/lib/data
>>> nucleus-rdbms-3.2.9.jar:/usr/hdp/2.5.3.0-37/hadoop/conf/:/
>>> usr/hdp/current/hadoop-client/lib/aws-java-sdk-s3-1.10.6.
>>> jar:/usr/hdp/current/hadoop-client/lib/aws-java-sdk-core-
>>> 1.10.6.jar:/usr/hdp/current/hadoop-client/lib/aws-java-sdk-kms-1.10.6.jar
>>> -Xms1g -Xmx1g -Dpio.log.dir=/home/da_mcom_milan
>>> org.apache.spark.deploy.SparkSubmit --conf
>>> spark.driver.extraClassPath=/data/PredictionIO-0.11.0-incuba
>>> ting/conf:/usr/hdp/2.5.3.0-37/hadoop/conf:/data/PredictionIO/vendors/hbase/conf
>>> --conf spark.driver.extraJavaOptions=-Dpio.log.dir=/home/da_mcom_milan
>>> --class org.apache.predictionio.workflow.CreateServer --jars
>>> file:/data/PredictionIO/some-engine/target/scala-2.10/univer
>>> sal-recommender_2.10-0.6.0.jar,file:/data/PredictionIO/some-engine
>>> /target/scala-2.10/universal-recommender-assembly-0.6.
>>> 0-deps.jar,file:/data/PredictionIO-0.11.0-incubating/lib/spa
>>> rk/pio-data-elasticsearch1-assembly-0.11.0-incubating.jar,
>>> file:/data/PredictionIO-0.11.0-incubating/lib/spark/pio-dat
>>> a-hbase-assembly-0.11.0-incubating.jar,file:/data/Prediction
>>> IO-0.11.0-incubating/lib/spark/pio-data-hdfs-assembly-
>>> 0.11.0-incubating.jar,file:/data/PredictionIO-0.11.0-
>>> incubating/lib/spark/pio-data-jdbc-assembly-0.11.0-incubatin
>>> g.jar,file:/data/PredictionIO-0.11.0-incubating/lib/spark/
>>> pio-data-localfs-assembly-0.11.0-incubating.jar --files
>>> file:/data/PredictionIO-0.11.0-incubating/conf/log4j.propert
>>> ies,file:/usr/hdp/2.5.3.0-37/hadoop/conf/core-site.xml,file:
>>> /data/PredictionIO/vendors/hbase/conf/hbase-site.xml
>>> file:/data/PredictionIO-0.11.0-incubating/lib/pio-assembly-0.11.0-incubating.jar
>>> --engineInstanceId AWKSaLQrSe_zVbacBHWI --engine-variant
>>> file:/data/PredictionIO/some-engine/engine.json --ip 0.0.0.0 --port
>>> *8004* --event-server-ip 0.0.0.0 --event-server-port 7070
>>> --json-extractor Both --env PIO_STORAGE_SOURCES_HBASE_TYPE
>>> =hbase,PIO_ENV_LOADED=1,PIO_STORAGE_REPOSITORIES_METADATA_NA
>>> ME=pio_meta,PIO_FS_BASEDIR=/home/da_mcom_milan/.pio_store,PI
>>> O_STORAGE_SOURCES_ELASTICSEARCH_HOSTS=mdc2vra176,PIO_STORAGE
>>> _SOURCES_HBASE_HOME=/data/PredictionIO/vendors/hbase,PIO_HOM
>>> E=/data/PredictionIO-0.11.0-incubating,PIO_FS_ENGINESDIR=/
>>> home/da_mcom_milan/.pio_store/engines,PIO_STORAGE_SOURCES_
>>> LOCALFS_PATH=/home/da_mcom_milan/.pio_store/models,PIO_
>>> STORAGE_SOURCES_ELASTICSEARCH_TYPE=elasticsearch,PIO_STORAGE
>>> _REPOSITORIES_METADATA_SOURCE=ELASTICSEARCH,PIO_STORAGE_
>>> REPOSITORIES_MODELDATA_SOURCE=LOCALFS,PIO_STORAGE_
>>> REPOSITORIES_EVENTDATA_NAME=pio_event,PIO_STORAGE_SOURCES_
>>> ELASTICSEARCH_CLUSTERNAME=pros-dev,PIO_FS_TMPDIR=/home/
>>> da_mcom_milan/.pio_store/tmp,PIO_STORAGE_REPOSITORIES_
>>> MODELDATA_NAME=pio_model,PIO_STORAGE_REPOSITORIES_
>>> EVENTDATA_SOURCE=HBASE,PIO_CONF_DIR=/data/PredictionIO-0.
>>> 11.0-incubating/conf,PIO_STORAGE_SOURCES_ELASTICSEARCH_
>>> PORTS=9300,PIO_STORAGE_SOURCES_LOCALFS_TYPE=localfs
>>>
>>> 3. da_mcom+ 32505  0.6  0.8 26473632 581164 ?     Sl   02:00   7:54 java
>>> -cp /data/PredictionIO-0.11.0-incubating/conf:/data/PredictionIO
>>> -0.11.0-incubating/plugins/*:/data/PredictionIO-0.11.0-incub
>>> ating/lib/spark/*:/data/PredictionIO-0.11.0-incubating/lib/p
>>> io-assembly-0.11.0-incubating.jar:/usr/hdp/2.5.3.0-37/hadoop
>>> /conf:/data/PredictionIO/vendors/hbase/conf
>>> -Dpio.log.dir=/home/da_mcom_milan org.apache.predictionio.tools.console.Console
>>> deploy --port *8004* --pio-home /data/PredictionIO-0.11.0-incubating
>>>
>>> 4. da_mcom+ 32671  1.4  1.3 19757488 881448 ?     Sl   02:00  16:27
>>> /usr/java/default/bin/java -Dhdp.version=2.5.3.0-37 -cp
>>> /data/PredictionIO-0.11.0-incubating/conf/:/usr/hdp/2.5.3.0-
>>> 37/hadoop/conf/:/data/PredictionIO/vendors/hbase/conf/:/usr/
>>> hdp/current/spark-client/conf/:/data/PredictionIO/vendors/sp
>>> ark/lib/spark-assembly-1.6.2.2.5.3.0-37-hadoop2.7.3.2.5.3.0-
>>> 37.jar:/data/PredictionIO/vendors/spark/lib/datanucleus-api-
>>> jdo-3.2.6.jar:/data/PredictionIO/vendors/spark/lib/datanucle
>>> us-core-3.2.10.jar:/data/PredictionIO/vendors/spark/lib/data
>>> nucleus-rdbms-3.2.9.jar:/usr/hdp/2.5.3.0-37/hadoop/conf/:/
>>> usr/hdp/current/hadoop-client/lib/aws-java-sdk-s3-1.10.6.
>>> jar:/usr/hdp/current/hadoop-client/lib/aws-java-sdk-core-
>>> 1.10.6.jar:/usr/hdp/current/hadoop-client/lib/aws-java-sdk-kms-1.10.6.jar
>>> -Xms1g -Xmx1g -Dpio.log.dir=/home/da_mcom_milan
>>> org.apache.spark.deploy.SparkSubmit --conf
>>> spark.driver.extraClassPath=/data/PredictionIO-0.11.0-incuba
>>> ting/conf:/usr/hdp/2.5.3.0-37/hadoop/conf:/data/PredictionIO/vendors/hbase/conf
>>> --conf spark.driver.extraJavaOptions=-Dpio.log.dir=/home/da_mcom_milan
>>> --class org.apache.predictionio.workflow.CreateServer --jars
>>> file:/data/PredictionIO/some-engine/target/scala-2.10/univer
>>> sal-recommender_2.10-0.6.0.jar,file:/data/PredictionIO/some-engine
>>> /target/scala-2.10/universal-recommender-assembly-0.6.
>>> 0-deps.jar,file:/data/PredictionIO-0.11.0-incubating/lib/spa
>>> rk/pio-data-elasticsearch1-assembly-0.11.0-incubating.jar,
>>> file:/data/PredictionIO-0.11.0-incubating/lib/spark/pio-dat
>>> a-hbase-assembly-0.11.0-incubating.jar,file:/data/Prediction
>>> IO-0.11.0-incubating/lib/spark/pio-data-hdfs-assembly-
>>> 0.11.0-incubating.jar,file:/data/PredictionIO-0.11.0-
>>> incubating/lib/spark/pio-data-jdbc-assembly-0.11.0-incubatin
>>> g.jar,file:/data/PredictionIO-0.11.0-incubating/lib/spark/
>>> pio-data-localfs-assembly-0.11.0-incubating.jar --files
>>> file:/data/PredictionIO-0.11.0-incubating/conf/log4j.propert
>>> ies,file:/usr/hdp/2.5.3.0-37/hadoop/conf/core-site.xml,file:
>>> /data/PredictionIO/vendors/hbase/conf/hbase-site.xml
>>> file:/data/PredictionIO-0.11.0-incubating/lib/pio-assembly-0.11.0-incubating.jar
>>> --engineInstanceId AWKSaLQrSe_zVbacBHWI --engine-variant
>>> file:/data/PredictionIO/some-engine/engine.json --ip 0.0.0.0 --port
>>> *8004* --event-server-ip 0.0.0.0 --event-server-port 7070
>>> --json-extractor Both --env PIO_STORAGE_SOURCES_HBASE_TYPE
>>> =hbase,PIO_ENV_LOADED=1,PIO_STORAGE_REPOSITORIES_METADATA_NA
>>> ME=pio_meta,PIO_FS_BASEDIR=/home/da_mcom_milan/.pio_store,PI
>>> O_STORAGE_SOURCES_ELASTICSEARCH_HOSTS=mdc2vra176,PIO_STORAGE
>>> _SOURCES_HBASE_HOME=/data/PredictionIO/vendors/hbase,PIO_HOM
>>> E=/data/PredictionIO-0.11.0-incubating,PIO_FS_ENGINESDIR=/
>>> home/da_mcom_milan/.pio_store/engines,PIO_STORAGE_SOURCES_
>>> LOCALFS_PATH=/home/da_mcom_milan/.pio_store/models,PIO_
>>> STORAGE_SOURCES_ELASTICSEARCH_TYPE=elasticsearch,PIO_STORAGE
>>> _REPOSITORIES_METADATA_SOURCE=ELASTICSEARCH,PIO_STORAGE_
>>> REPOSITORIES_MODELDATA_SOURCE=LOCALFS,PIO_STORAGE_
>>> REPOSITORIES_EVENTDATA_NAME=pio_event,PIO_STORAGE_SOURCES_
>>> ELASTICSEARCH_CLUSTERNAME=pros-dev,PIO_FS_TMPDIR=/home/
>>> da_mcom_milan/.pio_store/tmp,PIO_STORAGE_REPOSITORIES_
>>> MODELDATA_NAME=pio_model,PIO_STORAGE_REPOSITORIES_
>>> EVENTDATA_SOURCE=HBASE,PIO_CONF_DIR=/data/PredictionIO-0.
>>> 11.0-incubating/conf,PIO_STORAGE_SOURCES_ELASTICSEARCH_
>>> PORTS=9300,PIO_STORAGE_SOURCES_LOCALFS_TYPE=localfs
>>>
>>> For some reasons, processes 30932 and 31097 still alive and process
>>> listened by 32671
>>> Any ideas?
>>>
>>> Thank you,
>>> George
>>>
>>
>>
>
>
> --
>
> George Yarish, Java Developer
>
> Grid Dynamics
>
> 197101, Rentgena Str., 5A, St.Petersburg, Russia
>
> Cell: +7 950 030-1941
>
> Read Grid Dynamics' Tech Blog
> <http://blog.griddynamics.com/?utm_campaign=Big%20Data%20Blog%20social%20media%20promotion&utm_medium=CTA&utm_source=Email>
>
>
>

Mime
View raw message