predictionio-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Marius Rabenarivo <mariusrabenar...@gmail.com>
Subject Re: [PIO 0.10.0][Universtal Recommender 0.5] Exception when using remote Elasticsearch while training
Date Mon, 10 Apr 2017 15:27:37 GMT
Yes, I set the clustername in Elasticsearch config too.

Thank you for your advices about fields.

2017-04-10 19:07 GMT+04:00 Pat Ferrel <pat@occamsmachete.com>:

> Do you have the clustername set in Elasticsearch config also? It is
> changed in /usr/local/elasticsearch/config/elasticsearch.yml Setting this
> in both places will decouple server name from cluster name. Unfortunately
> changing a cluster name will call for a complete reset of PredictionIO data
> since it will loose track of engine metadata. I have found it easiest to
> export/backup then delete all apps and re-install or erase any table in
> hbase and elasticsearch. You have to get the clustername right before you
> start because it’s a pain to change.
>
> Also the fields must correspond to arrays of strings, not an array of true
> or false, an array of one element, a string of “T” or “F” would work for
> this. For fields the property must be attached to all items by using a $set
> event, then your visible and sponsored will work more like what you expect.
> They are not working at all now.
>
>
> On Apr 9, 2017, at 10:43 PM, Marius Rabenarivo <mariusrabenarivo@gmail.com>
> wrote:
>
> Hello,
>
> Find attached the pio-env.sh and engine.json
>
> that we are using.
>
> 2017-04-08 18:46 GMT+04:00 Marius Rabenarivo <mariusrabenarivo@gmail.com>:
>
>> I'll share them monday.
>>
>> 2017-04-07 22:23 GMT+04:00 Pat Ferrel <pat@occamsmachete.com>:
>>
>>> We use a clustername different from hostname all the time. In fact if
>>> you have more than one node you will have to make them different since
>>> there are multiple hostnames and one clustername.
>>>
>>> The clustername should be something unique to your application because
>>> ES has the sometimes helpful, but often maddeningly bad and hidden, habit
>>> of merging nodes into a cluster by clustername. For instance the default
>>> for ES is to merge all nodes on a LAN into one cluster.
>>>
>>> Anyway can you share your engine.json and the result of curl with
>>> exactly the same hostname? You need to add the clustername to pio-env.sh
>>> too so can you share that as well?
>>>
>>>
>>> On Apr 7, 2017, at 12:25 AM, Marius Rabenarivo <
>>> mariusrabenarivo@gmail.com> wrote:
>>>
>>> It works when using the default port 9200 and by changing the
>>> Elasticsearch cluster name to the same as the hostname as suggested here
>>>
>>> http://stackoverflow.com/questions/25885411/elasticsearch-er
>>> ror-failed-to-connect-to-master-no-route-to-host
>>>
>>> 2017-04-07 9:57 GMT+04:00 Marius Rabenarivo <mariusrabenarivo@gmail.com>
>>> :
>>>
>>>> I tried to run the command
>>>>
>>>> curl -X GET http://engine-server:9200
>>>>
>>>> and I'm getting a JSON response but when I'm running the command
>>>>
>>>> pio train
>>>>
>>>> I got
>>>>
>>>> org.elasticsearch.hadoop.rest.EsHadoopTransportException: java.net.NoRouteToHostException:
>>>> No route to host (Host unreachable)
>>>>
>>>> exception
>>>>
>>>> 2017-04-07 9:18 GMT+04:00 Marius Rabenarivo <mariusrabenarivo@gmail.com
>>>> >:
>>>>
>>>>> I tried using these configurations in the sparkConf section of
>>>>> engine.json
>>>>> It has the same effect as the code I mentioned below but I still
>>>>> getting the same exception.
>>>>>
>>>>> 2017-04-07 5:03 GMT+04:00 Pat Ferrel <pat@occamsmachete.com>:
>>>>>
>>>>>> BTW the docs for sparkConf are here: http://actionml.com/docs
>>>>>> /ur_config#ur-spark-conf
>>>>>>
>>>>>>
>>>>>> On Apr 6, 2017, at 4:52 PM, Pat Ferrel <pat@occamsmachete.com>
wrote:
>>>>>>
>>>>>> PIO in the past used the “transport client” of Elasticsearch,
which
>>>>>> is on port 9300 by default, The UR uses the REST client for some
things
>>>>>> like rapid writes to ES using Spark using a lib supplied by Elasticsearch.
>>>>>>
>>>>>> So, Spark needs to have config that the Elasticsearch library can
>>>>>> read. This can be done most easily by putting it into a sparkConf
section
>>>>>> of engine.json
>>>>>>
>>>>>> Like this:
>>>>>>
>>>>>>   "sparkConf": {
>>>>>>     "spark.serializer": "org.apache.spark.serializer.KryoSerializer",
>>>>>>     "spark.kryo.registrator": "org.apache.mahout.sparkbindings.io
>>>>>> .MahoutKryoRegistrator",
>>>>>>     "spark.kryo.referenceTracking": "false",
>>>>>>     "spark.kryoserializer.buffer": "300m",
>>>>>>     "es.index.auto.create": “true”,
>>>>>>     “es.nodes”: “node1,node2”  <========== add this
>>>>>>   },
>>>>>>
>>>>>> The list is comma separated host ip addresses of DNS names so no
>>>>>> spaces. If you are using the default port 9200 you don’t need to
specify
>>>>>> it. These values are defined by Elasticsearch so see their docs if
you have
>>>>>> any question about what can be configured in this way. For instance
you can
>>>>>> also specify password and username in this manner.
>>>>>>
>>>>>>
>>>>>> On Apr 6, 2017, at 4:27 PM, Marius Rabenarivo <
>>>>>> mariusrabenarivo@gmail.com> wrote:
>>>>>>
>>>>>> Hello,
>>>>>>
>>>>>> I try to run training from a remote machine.
>>>>>>
>>>>>> It fails in later stage due to this exception :
>>>>>>
>>>>>> org.elasticsearch.hadoop.rest.EsHadoopNoNodesLeftException:
>>>>>> Connection error (check network and/or proxy settings)- all nodes
failed;
>>>>>> tried [[engine-server:9300]]
>>>>>>
>>>>>> According to the stack trace the error is coming from this line :
>>>>>>
>>>>>> https://github.com/actionml/universal-recommender/blob/maste
>>>>>> r/src/main/scala/EsClient.scala#L179
>>>>>>
>>>>>> It was using a default configuration to access ES.
>>>>>>
>>>>>> I tried to replace this line with :
>>>>>>
>>>>>>
>>>>>>     val elasticSearchConf = Storage.getConfig("ELASTICSEARCH").get
>>>>>>
>>>>>>     val conf = Map(
>>>>>>       "es.mapping.id" -> "id",
>>>>>>       "es.nodes" -> elasticSearchConf.properties.get("HOSTS").head,
>>>>>>       "es.port" -> elasticSearchConf.properties.get("PORTS").head
>>>>>>     )
>>>>>>
>>>>>>     indexRDD.saveToEs(newIndexURI, conf)
>>>>>>
>>>>>> but I still getting the same error.
>>>>>>
>>>>>> Regards,
>>>>>>
>>>>>> Marius
>>>>>>
>>>>>> --
>>>>>> You received this message because you are subscribed to the Google
>>>>>> Groups "actionml-user" group.
>>>>>> To unsubscribe from this group and stop receiving emails from it,
>>>>>> send an email to actionml-user+unsubscribe@googlegroups.com.
>>>>>> To post to this group, send email to actionml-user@googlegroups.com.
>>>>>> To view this discussion on the web visit https://groups.google.co
>>>>>> m/d/msgid/actionml-user/CAC-ATVES0FM4iFtXGoU9-URikEq%2B2d6Na
>>>>>> V10Eh1S4_t1Ub4PcA%40mail.gmail.com
>>>>>> <https://groups.google.com/d/msgid/actionml-user/CAC-ATVES0FM4iFtXGoU9-URikEq%2B2d6NaV10Eh1S4_t1Ub4PcA%40mail.gmail.com?utm_medium=email&utm_source=footer>
>>>>>> .
>>>>>> For more options, visit https://groups.google.com/d/optout.
>>>>>>
>>>>>>
>>>>>> --
>>>>>> You received this message because you are subscribed to the Google
>>>>>> Groups "actionml-user" group.
>>>>>> To unsubscribe from this group and stop receiving emails from it,
>>>>>> send an email to actionml-user+unsubscribe@googlegroups.com.
>>>>>> To post to this group, send email to actionml-user@googlegroups.com.
>>>>>> To view this discussion on the web visit https://groups.google.co
>>>>>> m/d/msgid/actionml-user/EA18C433-834A-4917-99C3-A2460C281667
>>>>>> %40occamsmachete.com
>>>>>> <https://groups.google.com/d/msgid/actionml-user/EA18C433-834A-4917-99C3-A2460C281667%40occamsmachete.com?utm_medium=email&utm_source=footer>
>>>>>> .
>>>>>> For more options, visit https://groups.google.com/d/optout.
>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "actionml-user" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to actionml-user+unsubscribe@googlegroups.com.
>>> To post to this group, send email to actionml-user@googlegroups.com.
>>> To view this discussion on the web visit https://groups.google.co
>>> m/d/msgid/actionml-user/CAC-ATVGHNYHJ-V78B1knbpqWGn_cpkvYCC_
>>> %2BW3Tmdw6kywcSTw%40mail.gmail.com
>>> <https://groups.google.com/d/msgid/actionml-user/CAC-ATVGHNYHJ-V78B1knbpqWGn_cpkvYCC_%2BW3Tmdw6kywcSTw%40mail.gmail.com?utm_medium=email&utm_source=footer>
>>> .
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>>
>>
>
> --
> You received this message because you are subscribed to the Google Groups
> "actionml-user" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to actionml-user+unsubscribe@googlegroups.com.
> To post to this group, send email to actionml-user@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/
> msgid/actionml-user/CAC-ATVG8madmx19TqijGra4n6TxDSnxvz
> iB4RNggnvAosB7Etg%40mail.gmail.com
> <https://groups.google.com/d/msgid/actionml-user/CAC-ATVG8madmx19TqijGra4n6TxDSnxvziB4RNggnvAosB7Etg%40mail.gmail.com?utm_medium=email&utm_source=footer>
> .
> For more options, visit https://groups.google.com/d/optout.
> <engine.json><pio-env.sh>
>
>

Mime
View raw message