predictionio-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Pat Ferrel <...@occamsmachete.com>
Subject Re: Multiple apps problem
Date Sat, 21 Jan 2017 17:24:02 GMT
Yeah, there is hidden metadata associated with engines/templates and it is very hard to detect
when it changes or what changes when. I’ve gotten into these situations and starting from
scratch is the only reliable way to fix it.

For what it’w worth we in PredictionIO have is as one of our main goals to make this at
least more clear for users, and at best make shared metadata should work as expected in all
cases. the first start at this will be stateless commands.


On Jan 21, 2017, at 1:17 AM, Hossein Bahrami <h.bahrami@live.com> wrote:

Hi,
 
Problem fixed after changing the indexName again .
 
Thanks for your support .
 
Sent from Mail <https://go.microsoft.com/fwlink/?LinkId=550986> for Windows 10
 
From: Pat Ferrel <mailto:pat@occamsmachete.com>
Sent: Friday, January 20, 2017 4:49 PM
To: Hossein Bahrami <mailto:h.bahrami@live.com>
Cc: user@predictionio.incubator.apache.org <mailto:user@predictionio.incubator.apache.org>
Subject: Re: Multiple apps problem
 
execute the entire workflow:
erase manifest.json in your template directories
change the indexName in both engine.json into a completely new name
pio build
pio train
pio deploy

For both engines.

On Jan 20, 2017, at 1:09 AM, Hossein Bahrami <h.bahrami@live.com <mailto:h.bahrami@live.com>>
wrote:

Hi,
I did as you mentioned, it didn’t helped, still have problem.
 
 
From: Pat Ferrel <mailto:pat@occamsmachete.com>
Sent: Wednesday, January 18, 2017 5:52 PM
To: user@predictionio.incubator.apache.org <mailto:user@predictionio.incubator.apache.org>
Subject: Re: Multiple apps problem
 
Yes this will happen since you had the same indexName for both.

execute the entire workflow:
erase manifest.json in your template directories
pio build
pio train
pio deploy

For both engines.


On Jan 18, 2017, at 2:53 AM, Hossein Bahrami <h.bahrami@live.com <mailto:h.bahrami@live.com>>
wrote:

As I checked the elasticsearch logs apparently the training of the app A deleted the index
of the other app .
 
(Each of the apps are in different directories and have their own copies of UR temples and
different name and access key and everything else)
 
I don’t understand why training of one app should delete some index in elasticsearch which
doesn’t belong to it.
 
Sent from Mail <https://go.microsoft.com/fwlink/?LinkId=550986> for Windows 10
 
From: Hossein Bahrami <mailto:h.bahrami@live.com>
Sent: Wednesday, January 18, 2017 11:30 AM
To: Pat Ferrel <mailto:pat@occamsmachete.com>; user@predictionio.incubator.apache.org
<mailto:user@predictionio.incubator.apache.org>
Subject: RE: Multiple apps problem
 
Hi,
 
I have 2 directories and 2 copies of UR template.
After running build, train, deploy chain I get the error after training the other app.
 
 
Sent from Mail <https://go.microsoft.com/fwlink/?LinkId=550986> for Windows 10
 
From: Pat Ferrel <mailto:pat@occamsmachete.com>
Sent: Tuesday, January 17, 2017 7:52 PM
To: user@predictionio.incubator.apache.org <mailto:user@predictionio.incubator.apache.org>
Subject: Re: Multiple apps problem
 
Do you have 2 directories with 2 copies of the Universal Recommender? PIO requires this.

Also please build, train, and deploy both engines. The metadata for the engines doesn’t
get changed or used until you perform all workflow steps.



On Jan 17, 2017, at 7:56 AM, Hossein Bahrami <h.bahrami@live.com <mailto:h.bahrami@live.com>>
wrote:

Hi Pat,
 
After changing the indexName the problem for mistaking the models resolved
but I find out that after retraining one model the other one get below error when sending
queries to it.
 
And as you mentioned I checked engine.json for both apps and they have not any similar setting
 
Stack Trace:
org.elasticsearch.indices.IndexMissingException: [abcIndexName] missing
                at org.elasticsearch.cluster.metadata.MetaData.convertFromWildcards(MetaData.java:884)
                at org.elasticsearch.cluster.metadata.MetaData.concreteIndices(MetaData.java:692)
                at org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.<init>(TransportSearchTypeAction.java:118)
                at org.elasticsearch.action.search.type.TransportSearchQueryThenFetchAction$AsyncAction.<init>(TransportSearchQueryThenFetchAction.java:71)
                at org.elasticsearch.action.search.type.TransportSearchQueryThenFetchAction$AsyncAction.<init>(TransportSearchQueryThenFetchAction.java:65)
                at org.elasticsearch.action.search.type.TransportSearchQueryThenFetchAction.doExecute(TransportSearchQueryThenFetchAction.java:62)
                at org.elasticsearch.action.search.type.TransportSearchQueryThenFetchAction.doExecute(TransportSearchQueryThenFetchAction.java:52)
                at org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:75)
                at org.elasticsearch.action.search.TransportSearchAction.doExecute(TransportSearchAction.java:100)
                at org.elasticsearch.action.search.TransportSearchAction.doExecute(TransportSearchAction.java:43)
                at org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:75)
                at org.elasticsearch.action.support.HandledTransportAction$TransportHandler.messageReceived(HandledTransportAction.java:63)
                at org.elasticsearch.action.support.HandledTransportAction$TransportHandler.messageReceived(HandledTransportAction.java:51)
                at org.elasticsearch.transport.netty.MessageChannelHandler.handleRequest(MessageChannelHandler.java:222)
                at org.elasticsearch.transport.netty.MessageChannelHandler.messageReceived(MessageChannelHandler.java:114)
                at org.elasticsearch.common.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)
                at org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
                at org.elasticsearch.common.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791)
                at org.elasticsearch.common.netty.channel.Channels.fireMessageReceived(Channels.java:296)
                at org.elasticsearch.common.netty.handler.codec.frame.FrameDecoder.unfoldAndFireMessageReceived(FrameDecoder.java:462)
                at org.elasticsearch.common.netty.handler.codec.frame.FrameDecoder.callDecode(FrameDecoder.java:443)
                at org.elasticsearch.common.netty.handler.codec.frame.FrameDecoder.messageReceived(FrameDecoder.java:303)
                at org.elasticsearch.common.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)
                at org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
                at org.elasticsearch.common.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791)
                at org.elasticsearch.common.netty.OpenChannelsHandler.handleUpstream(OpenChannelsHandler.java:74)
                at org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
                at org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:559)
                at org.elasticsearch.common.netty.channel.Channels.fireMessageReceived(Channels.java:268)
                at org.elasticsearch.common.netty.channel.Channels.fireMessageReceived(Channels.java:255)
                at org.elasticsearch.common.netty.channel.socket.nio.NioWorker.read(NioWorker.java:88)
                at org.elasticsearch.common.netty.channel.socket.nio.AbstractNioWorker.process(AbstractNioWorker.java:108)
                at org.elasticsearch.common.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:337)
                at org.elasticsearch.common.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:89)
                at org.elasticsearch.common.netty.channel.socket.nio.NioWorker.run(NioWorker.java:178)
                at org.elasticsearch.common.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
                at org.elasticsearch.common.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)
                at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
                at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
                at java.lang.Thread.run(Thread.java:745)
 
Sent from Mail <https://go.microsoft.com/fwlink/?LinkId=550986> for Windows 10
 
From: Hossein Bahrami <mailto:h.bahrami@live.com>
Sent: Monday, January 16, 2017 9:18 AM
To: Pat Ferrel <mailto:pat@occamsmachete.com>; user@predictionio.incubator.apache.org
<mailto:user@predictionio.incubator.apache.org>
Subject: RE: Multiple apps problem
 
Hi Pat,
 
Thanks for your help, problem fixed.
The issue was about using same indexNames, everything else was correct except indexName.
 
Sent from Mail <https://go.microsoft.com/fwlink/?LinkId=550986> for Windows 10
 
From: Pat Ferrel <mailto:pat@occamsmachete.com>
Sent: Friday, January 13, 2017 7:40 PM
To: user@predictionio.incubator.apache.org <mailto:user@predictionio.incubator.apache.org>
Subject: Re: Multiple apps problem
 
You have 2 copies of the UR directory and 2 engine.json files and 2 input event sets? Make
sure there are different “indexName” set for each model in the 2 engine.json files. Otherwise
training of one engine will wipe out the other since they point to the same indexName.

To have 2 completely separate applications with different data you need:
1) 2 different EventServer appNames with different appids, create with `pio app new <your-app-name>`
2) 2 different indexNames for 2 models, set in the 2 engine.json files
3) each engine deployed on a separate port (you have this already)


On Jan 13, 2017, at 12:35 AM, Hossein Bahrami <h.bahrami@live.com <mailto:h.bahrami@live.com>>
wrote:

Hi Pat,
 
I have a Ubuntu machine which PredictionIo (0.9.7-aml) installed on it,
I have two application both based on Universal recommender template.
I trained both of the apps and deployed them on different ports .
Results of the apps were quite normal but the problem which I figured out later is that one
of the apps was predicting items with score zero so I retrained it then scores become normal
numbers but after this action (retraining) the other app start returning empty results, so
I retrained that one and then the first one returned zero score items again .
 
This is like closed circle and retraining each of apps causes the other app return empty results
or abnormal zero scored results.
 
Regards 
Hossein
 
From: Pat Ferrel <mailto:pat@occamsmachete.com>
Sent: Thursday, January 12, 2017 5:57 PM
To: user@predictionio.incubator.apache.org <mailto:user@predictionio.incubator.apache.org>
Subject: Re: Multiple apps problem
 
This forum is to support Apache PIO, and the latest UR runs on it so I advise you upgrade.
Instructions are here on actionml.com <http://actionml.com/> or here for Apache PIO:
http://predictionio.incubator.apache.org/install/ <http://predictionio.incubator.apache.org/install/>

I don’t understand your explanation, there should be one train per PredictionServer, you
can’t have one PS and two “engines” or configurations. The 2 engines must be on 2 different
ports if deployed separately.

Can you explain your setup and what you are trying to do?


On Jan 12, 2017, at 7:39 AM, Hossein Bahrami <h.bahrami@live.com <mailto:h.bahrami@live.com>>
wrote:

Hi everyone,
 
I have a very strange problem. I have two application running on one predictionio engine,
each time I train one of them the other one’s predictions returning zero score items for
one of them and no items at all for the other.
 
I am using UR algorithm for both of apps
 
pio version : 0.9.7-aml
 
regards


Mime
View raw message