Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id 60852200D4A for ; Tue, 28 Nov 2017 17:49:17 +0100 (CET) Received: by cust-asf.ponee.io (Postfix) id 5F0D0160C07; Tue, 28 Nov 2017 16:49:17 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 546C0160BE7 for ; Tue, 28 Nov 2017 17:49:16 +0100 (CET) Received: (qmail 39857 invoked by uid 500); 28 Nov 2017 16:49:15 -0000 Mailing-List: contact user-help@predictionio.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@predictionio.apache.org Delivered-To: mailing list user@predictionio.apache.org Received: (qmail 39847 invoked by uid 99); 28 Nov 2017 16:49:15 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd2-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 28 Nov 2017 16:49:15 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd2-us-west.apache.org (ASF Mail Server at spamd2-us-west.apache.org) with ESMTP id 9E9FB1A13CB for ; Tue, 28 Nov 2017 16:49:14 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd2-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 0.072 X-Spam-Level: X-Spam-Status: No, score=0.072 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=2, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-2.8, SPF_SOFTFAIL=0.972] autolearn=disabled Authentication-Results: spamd2-us-west.apache.org (amavisd-new); dkim=pass (1024-bit key) header.d=appchoose.io Received: from mx1-lw-us.apache.org ([10.40.0.8]) by localhost (spamd2-us-west.apache.org [10.40.0.9]) (amavisd-new, port 10024) with ESMTP id YeVazwjxiv_R for ; Tue, 28 Nov 2017 16:49:11 +0000 (UTC) Received: from mail-wr0-f178.google.com (mail-wr0-f178.google.com [209.85.128.178]) by mx1-lw-us.apache.org (ASF Mail Server at mx1-lw-us.apache.org) with ESMTPS id 110D65F177 for ; Tue, 28 Nov 2017 16:49:11 +0000 (UTC) Received: by mail-wr0-f178.google.com with SMTP id w62so693571wrc.6 for ; Tue, 28 Nov 2017 08:49:10 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=appchoose.io; s=google; h=from:mime-version:subject:message-id:date:to; bh=g1P6H0/j1SR/ZO1nRvJThk8annhOpZtfw7awGpOB+BE=; b=Xt6X2iFIrKMQj5OR1JiICqqySUPcemRqf8CoY+ghjy6aK/N9L4ztI+oZkqoGsjsbBk km+k4LA7wDg0q/1ITAeuVyalugAviXYDFImDbR34u1lbrA4ueUBia9qOsySifnMTf6NQ oWsj1/MeMK5G3SduFQE0xwkMgByTwEhrWTXJM= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:mime-version:subject:message-id:date:to; bh=g1P6H0/j1SR/ZO1nRvJThk8annhOpZtfw7awGpOB+BE=; b=jMhPG8UjEUSlnS7Zv4CI3x/S4yOr68Ikb6tpZr37cnQbQ8lmYosQSDUsUpNo20KTLc iFZoLNmDdqcBdnrTZ52HjbJj0bochjSj9eYDsHkQ+Amw4/TtUmUoUsJTcYh/BWRJ5NOY HnprkwxPaCWhUB4x3iX63lHYkeqztINsyMQdwXog4+kwqpjodpYe2xilGRL2kpsuQWM2 4bEVaZCFW3BMJsdjiEhyklPKLIljnozbstKkhKjSKYlZ4JqPGgPN8m7U3HlWFIC8KZNP Aid2/0bQPEKvR4qhbJ9cuwyN0KBXmQ9gCUjHuXNdTU6/ueV0RbfD7HtJiI6FU1Zgy97v DGog== X-Gm-Message-State: AJaThX6VGtpkRpv6eXP9P3ayaaq+TE0yE3Je4UXae4uv9CQmzzPH8gbY RwMSIlMjp4taTNjS7pdAK63REC3JH9c= X-Google-Smtp-Source: AGs4zMa7hmc/f2TViQ7ctvZBYoxFfNrrO6jMrpLZhS1gG4hs0vwe81eu54UoTTYP92UmG0j1/eCmtA== X-Received: by 10.223.175.100 with SMTP id z91mr34829658wrc.138.1511887749854; Tue, 28 Nov 2017 08:49:09 -0800 (PST) Received: from [10.10.0.81] (LPuteaux-656-1-277-203.w80-15.abo.wanadoo.fr. [80.15.146.203]) by smtp.gmail.com with ESMTPSA id g99sm50214540wrd.72.2017.11.28.08.49.09 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 28 Nov 2017 08:49:09 -0800 (PST) From: Thibaut Gensollen - Choose Content-Type: multipart/alternative; boundary="Apple-Mail=_5C365229-DC7B-4FD2-B4D7-F780352D2318" Mime-Version: 1.0 (Mac OS X Mail 11.1 \(3445.4.7\)) Subject: Error with a Cluster : ConnectionRefused Message-Id: <3B2E3FA3-215E-49AB-AB3A-9A2EAF0DCD8C@appchoose.io> Date: Tue, 28 Nov 2017 17:49:08 +0100 To: user@predictionio.apache.org X-Mailer: Apple Mail (2.3445.4.7) archived-at: Tue, 28 Nov 2017 16:49:17 -0000 --Apple-Mail=_5C365229-DC7B-4FD2-B4D7-F780352D2318 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 Hi Guys, I tried to set up a Predictionio cluster, working with an elasticsearch = and hadoop clusters.=20 The fact is, everything is working well (Hbase, ES or Hadoop/Spark), and = as soon as I tried =C2=AB pio-start-all =C2=BB I am getting these = errors: And then, my hadoop is not working anymore : hdfs dfs -mkdir test mkdir: Call =46rom master.c.choose-ninja-01.internal/10.128.0.4 to = master.c.choose-ninja-01.internal:9000 failed on connection exception: = java.net.ConnectException: Connection refused; For more details see: = http://wiki.apache.org/hadoop/ConnectionRefused = I tried working with a gcloud dataproc, or following your tutorial = http://actionml.com/docs/small_ha_cluster = with some vm engines (still = on gcloud), but the results are the same.. Any Idea ? We tried so many = times that I am beginning thinking that we cannot use predictionio with = an hadoop and elasticsearch clusters :/ Thanks for your help, Regards Thibaut =E2=80=94 Thibaut Gensollen CTO +33 6 76 73 08 50 --Apple-Mail=_5C365229-DC7B-4FD2-B4D7-F780352D2318 Content-Type: multipart/mixed; boundary="Apple-Mail=_2668053F-F60A-466A-94E4-26BEAD880F2E" --Apple-Mail=_2668053F-F60A-466A-94E4-26BEAD880F2E Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=utf-8
Hi Guys,

I tried to set up a Predictionio cluster, working with an = elasticsearch and hadoop clusters. 

The fact is, everything is working well = (Hbase, ES or Hadoop/Spark), and as soon as I tried = =C2=AB pio-start-all =C2=BB I am getting these = errors:

= --Apple-Mail=_2668053F-F60A-466A-94E4-26BEAD880F2E Content-Disposition: attachment; filename=-.txt Content-Type: text/plain; x-unix-mode=0644; name="-.txt" Content-Transfer-Encoding: quoted-printable aml@master:~$ pio status=0D [INFO] [Management$] Inspecting PredictionIO...=0D [INFO] [Management$] PredictionIO 0.12.0-incubating is installed at = /home/aml=0D [INFO] [Management$] Inspecting Apache Spark...=0D [INFO] [Management$] Apache Spark is installed at = /opt/spark/spark-1.6.3-bin-hadoop2.6=0D [INFO] [Management$] Apache Spark 1.6.3 detected (meets minimum = requirement of 1.3.0)=0D [INFO] [Management$] Inspecting storage backend connections...=0D [INFO] [Storage$] Verifying Meta Data Backend (Source: ELASTICSEARCH)...=0D= [INFO] [Storage$] Verifying Model Data Backend (Source: HDFS)...=0D [ERROR] [Storage$] Error initializing storage client for source HDFS.=0D java.lang.IllegalArgumentException: Wrong FS: = hdfs://master.c.choose-ninja-01.internal:9000/models, expected: file:=0D ///=0D at = org.apache.hadoop.fs.FileSystem.checkPath(FileSystem.java:649)=0D at = org.apache.hadoop.fs.RawLocalFileSystem.setWorkingDirectory(RawLocalFileSy= stem.java:547)=0D at = org.apache.hadoop.fs.FilterFileSystem.setWorkingDirectory(FilterFileSystem= .java:280)=0D at = org.apache.predictionio.data.storage.hdfs.StorageClient.(StorageClie= nt.scala:33)=0D at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native = Method)=0D at = sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAcc= essorImpl.java:62)=0D at = sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstr= uctorAccessorImpl.java:45)=0D at = java.lang.reflect.Constructor.newInstance(Constructor.java:423)=0D at = org.apache.predictionio.data.storage.Storage$.getClient(Storage.scala:252)= =0D at = org.apache.predictionio.data.storage.Storage$.org$apache$predictionio$data= $storage$Storage$$updateS2CM(S=0D torage.scala:283)=0D at = org.apache.predictionio.data.storage.Storage$$anonfun$sourcesToClientMeta$= 1.apply(Storage.scala:244)=0D at = org.apache.predictionio.data.storage.Storage$$anonfun$sourcesToClientMeta$= 1.apply(Storage.scala:244)=0D at = scala.collection.mutable.MapLike$class.getOrElseUpdate(MapLike.scala:194)=0D= at = scala.collection.mutable.AbstractMap.getOrElseUpdate(Map.scala:80)=0D at = org.apache.predictionio.data.storage.Storage$.sourcesToClientMeta(Storage.= scala:244)=0D at = org.apache.predictionio.data.storage.Storage$.getDataObject(Storage.scala:= 315)=0D at = org.apache.predictionio.data.storage.Storage$.getDataObjectFromRepo(Storag= e.scala:300)=0D at = org.apache.predictionio.data.storage.Storage$.getModelDataModels(Storage.s= cala:442)=0D at = org.apache.predictionio.data.storage.Storage$.verifyAllDataObjects(Storage= .scala:381)=0D at = org.apache.predictionio.tools.commands.Management$.status(Management.scala= :156)=0D at = org.apache.predictionio.tools.console.Pio$.status(Pio.scala:155)=0D at = org.apache.predictionio.tools.console.Console$$anonfun$main$1.apply(Consol= e.scala:721)=0D at = org.apache.predictionio.tools.console.Console$$anonfun$main$1.apply(Consol= e.scala:656)=0D at scala.Option.map(Option.scala:146)=0D at = org.apache.predictionio.tools.console.Console$.main(Console.scala:656)=0D= at = org.apache.predictionio.tools.console.Console.main(Console.scala) [ERROR] [Management$] Unable to connect to all storage backends = successfully.=0D The following shows the error message from the storage backend.=0D Data source HDFS was not properly initialized. = (org.apache.predictionio.data.storage.StorageClientException)=0D Dumping configuration of initialized storage backend sources.=0D Please make sure they are correct.=0D Source Name: ELASTICSEARCH; Type: elasticsearch; Configuration: TYPE -> = elasticsearch, HOME -> /opt/elasticsearch/e=0D lasticsearch-5.5.2=0D Source Name: HDFS; Type: (error); Configuration: (error= --Apple-Mail=_2668053F-F60A-466A-94E4-26BEAD880F2E Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=utf-8

And = then, my hadoop is not working anymore :

hdfs = dfs -mkdir test
mkdir: Call = =46rom master.c.choose-ninja-01.internal/10.128.0.4 to = master.c.choose-ninja-01.internal:9000 failed on connection exception: = java.net.ConnectException: Connection refused; For more details see: =  http://wiki.apache.org/hadoop/ConnectionRefused
=

I = tried working with a gcloud dataproc, or following your tutorial http://actionml.com/docs/small_ha_cluster with some = vm engines (still on gcloud), but the results are the same.. Any Idea ? = We tried so many times that I am beginning thinking that we cannot use = predictionio with an hadoop and elasticsearch clusters :/

Thanks for your = help,
Regards

Thibaut
=E2=80=94


= --Apple-Mail=_2668053F-F60A-466A-94E4-26BEAD880F2E Content-Disposition: inline; filename=15px.jpg Content-Type: image/jpeg; x-unix-mode=0666; name="15px.jpg" Content-Transfer-Encoding: base64 /9j/4QAYRXhpZgAASUkqAAgAAAAAAAAAAAAAAP/sABFEdWNreQABAAQAAAAtAAD/7gAOQWRvYmUA ZMAAAAAB/9sAhAAKBwcHCAcKCAgKDgkICQ4RDQoKDREUEBAREBAUEw8REBARDxMTFxcZFxcTHh4g IB4eLCsrKywxMTExMTExMTExAQsJCQsMCw0LCw0RDg4OERQODg4OFBgQEBEQEBgeFhMTExMWHhsd GRkZHRshIR4eISEpKSgpKTExMTExMTExMTH/wAARCAAXAFkDASIAAhEBAxEB/8QAbAAAAwADAQEA AAAAAAAAAAAAAAUHAQYIAwIBAQAAAAAAAAAAAAAAAAAAAAAQAAEDBAECAwcEAwAAAAAAAAIBAwUA EQQGEiExIhMHQVFhkdFCFIGhMlIjVRcRAQAAAAAAAAAAAAAAAAAAAAD/2gAMAwEAAhEDEQA/ALNW Kke15s/ue9OaZF5xR0ZgN8s50FVFOyCrhFxVFKymgIN7X6rTHXfTjZ9X2HEejJv8iDJVXPx3+Qqq W7C34xVV9hXRUoPH0dkpDNk9qTLyHMgRymzFDJSRCMn0JUv2ugp8qqVc5adpkhtk1NYzMicbg47y llEHIuZEbnlDwQwRbWLqq9K27UZuc0/aJHVJvLOSwcbFcy8Z4lVSEWm/P8HNVVEIEVFG/Re1BXqz UMhNc2T1RXKnZWWPAwBeJrEx20VwRUbEog3zBEQUJE5d1WmunTs7qOz5unz+SUhhs47mTiPqqkSC 22r/AIFNVXiQCvhVehJ0oK7WahkLAbH6puZk3JSpx8a28rWJjgimIqiIXEG+YIiChJcu6rTXT5XZ dV23J0mTyClGjYN2NcNVVeQtk8CCpKqoJoKio36L2oK9RXNeHIPTrmVITO3vQ035yixjOC+jadvv aWzQovS1ulq271G3OZiY2G13HlELKycRt6RmGb3cAv8AGJNkHWxcVJVHqtBZqK5lWbb1x5iR1rZ8 mQzPMRcrHeYdabNO6qXMzE0W1lv1rfP+7t/6o/mv0oMbVhSkRvzmx6e7jymeYcZSGbcA8gURAA+b AF5nArCt0S6FTfXth9TpzYsVcmISEg2b/mA+2QqaW+0nbGpX7cUt76KKBd6KtEEptfJRW2S0PhIS 6oeR/VV6de/avmUYx3vWh1vMIAw3IxwcgiMRRGixjE1VVXpZPrRRQJ9azd10z8vCgcNva4F14ix8 nDVchtDsiX54ynwLjbkBUx1CKyZLbc6f3XKxsGSyMdwGok3gbfFo2lAjVlT5gAM3ty6+1aKKBdrm XuWluZuJr2I3tcA88pMZGGX5AIdkS/LGU+BcePIS/SmmmR8rIbu9sm0P42FOE0SR0M46IPqqgoiv kclcEBC/dLr3oooNM2XZFdckYyc12NSfdMgKQYNGybO/8i4EQkSe9ST40xlteXB1/WZTHkY7KnI9 nk5HuZDDnmsE8460jYmdnRC5ASJ39naiig9HN2kpYmcOM1mIishSFHcrIaZ8v49XxARH5r7qrth/ tFftRRQf/9k= --Apple-Mail=_2668053F-F60A-466A-94E4-26BEAD880F2E Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=us-ascii


Thibaut Gensollen

= --Apple-Mail=_2668053F-F60A-466A-94E4-26BEAD880F2E-- --Apple-Mail=_5C365229-DC7B-4FD2-B4D7-F780352D2318--