Return-Path: X-Original-To: apmail-giraph-user-archive@www.apache.org Delivered-To: apmail-giraph-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 332EA17CF7 for ; Fri, 27 Feb 2015 04:41:43 +0000 (UTC) Received: (qmail 13598 invoked by uid 500); 27 Feb 2015 04:41:43 -0000 Delivered-To: apmail-giraph-user-archive@giraph.apache.org Received: (qmail 13542 invoked by uid 500); 27 Feb 2015 04:41:42 -0000 Mailing-List: contact user-help@giraph.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@giraph.apache.org Delivered-To: mailing list user@giraph.apache.org Received: (qmail 13531 invoked by uid 99); 27 Feb 2015 04:41:42 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 27 Feb 2015 04:41:42 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of kalyan.kumar@gmail.com designates 209.85.160.178 as permitted sender) Received: from [209.85.160.178] (HELO mail-yk0-f178.google.com) (209.85.160.178) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 27 Feb 2015 04:41:37 +0000 Received: by ykq19 with SMTP id 19so6197947ykq.9 for ; Thu, 26 Feb 2015 20:41:16 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=ZlKvrOh+d26RjxLjctjVoDJAEwO6GL8JCzjlnMzATAk=; b=oSSirXN0kMTvQxuqQjvw1o7fSzmb9zq6puJerd673VrJpJomVJulJ4w4fwSBU5TKAU PnHudGPqA+Qt2CDll6RDkMk32OOoTOsuxSvxowl/BPsNCv/L8+yuJHSgXArNLGq10RZP c3UWyTBnRi+nsv/AVbObJ/0BED93G9hCOiwLRhcDRFt/Uj0lEHjmj/x3MKwd/1vDwZdM MSt+Yr4MfNqfRsaOcV1IhcNTS0t/NqHTQW/d5hmzQV/v+rmBe6szhsAsBz7GRYATUnjc 4g8PAqw4/Njl+cOcZDMHVoiaS+rpSY9xCrpoGqu3oEuwLjAaRajjnfN6UPd9Bgubnymr jaow== MIME-Version: 1.0 X-Received: by 10.236.229.225 with SMTP id h91mr11962230yhq.20.1425012076879; Thu, 26 Feb 2015 20:41:16 -0800 (PST) Received: by 10.170.197.142 with HTTP; Thu, 26 Feb 2015 20:41:16 -0800 (PST) Date: Thu, 26 Feb 2015 20:41:16 -0800 Message-ID: Subject: giraph hive example and versions From: Kalyan Kanuri To: user@giraph.apache.org Content-Type: multipart/alternative; boundary=047d7bf18428a4a6ca05100a7abb X-Virus-Checked: Checked by ClamAV on apache.org --047d7bf18428a4a6ca05100a7abb Content-Type: text/plain; charset=ISO-8859-1 Hi, I'm trying out giraph with hive with instructions from http://giraph.apache.org/hive.html and I get the following error while reading data from hive hadoop jar giraphstarter-jar-with-dependencies.jar org.apache.giraph.hive.HiveGiraphRunner -hiveconf hive.metastore.uris=thrift://..:50513 -hiveconf hive.metastore.sasl.enabled=true -hiveconf hive.metastore.kerberos.principal=hcat/... -db=dancehall -ei="com.yahoo.IdMappingsHiveToEdge, id_mappings, source=interstitials, dt=2015022323" -o "com.yahoo.IdClustersEdgeToHive, id_giraph, source=interstitials, dt=2015022323" --computationClass=com.yahoo.HiveConnectedComponents -w 1 15/02/27 04:18:58 INFO common.HiveMetastores: Connecting to metastore at ...50513 15/02/27 04:19:58 INFO common.BackoffRetryTask: Failed, but will retry in 25250 msec : org.apache.thrift.transport.TTransportException: java.net.SocketTimeoutException: Read timed out 15/02/27 04:20:24 INFO common.HiveMetastores: Connecting to metastore at ..:50513 15/02/27 04:21:24 INFO common.BackoffRetryTask: Failed, but will retry in 51984 msec : org.apache.thrift.transport.TTransportException: java.net.SocketTimeoutException: Read timed out 15/02/27 04:22:16 INFO common.HiveMetastores: Connecting to metastore at ...:50513 Exception in thread "main" java.lang.IllegalStateException: initialize: IOException occurred at com.facebook.hiveio.input.HiveApiInputFormat.initialize(HiveApiInputFormat.java:94) at org.apache.giraph.hive.input.edge.HiveEdgeInputFormat.setConf(HiveEdgeInputFormat.java:73) at org.apache.giraph.conf.ImmutableClassesGiraphConfiguration.configureIfPossible(ImmutableClassesGiraphConfiguration.java:156) at org.apache.giraph.utils.ConfigurationUtils.configureIfPossible(ConfigurationUtils.java:151) at org.apache.giraph.utils.ReflectionUtils.newInstance(ReflectionUtils.java:111) at org.apache.giraph.conf.ImmutableClassesGiraphConfiguration.createEdgeInputFormat(ImmutableClassesGiraphConfiguration.java:477) at org.apache.giraph.conf.ImmutableClassesGiraphConfiguration.createWrappedEdgeInputFormat(ImmutableClassesGiraphConfiguration.java:488) at org.apache.giraph.hive.HiveGiraphRunner.prepareHiveEdgeInputs(HiveGiraphRunner.java:425) at org.apache.giraph.hive.HiveGiraphRunner.handleCommandLine(HiveGiraphRunner.java:615) at org.apache.giraph.hive.HiveGiraphRunner.run(HiveGiraphRunner.java:293) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84) at org.apache.giraph.hive.HiveGiraphRunner.main(HiveGiraphRunner.java:286) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:601) at org.apache.hadoop.util.RunJar.run(RunJar.java:221) at org.apache.hadoop.util.RunJar.main(RunJar.java:136) Caused by: java.io.IOException: No more retries left org.apache.thrift.transport.TTransportException: java.net.SocketTimeoutException: Read timed out at com.facebook.hiveio.common.BackoffRetryTask.run(BackoffRetryTask.java:111) at com.facebook.hiveio.schema.HiveTableSchemas.lookup(HiveTableSchemas.java:156) at com.facebook.hiveio.schema.HiveTableSchemas.initTableSchema(HiveTableSchemas.java:97) at com.facebook.hiveio.input.HiveApiInputFormat.initialize(HiveApiInputFormat.java:92) ... 18 more Any hints around this ? I work at Yahoo and here are versions we are using. Is Giraph and hive-io compatiable with these versions ? hadoop: 2.6.0.5.1501231242 Hive: 0.13.0.6.1502201219 -- Thanks, Kalyan --047d7bf18428a4a6ca05100a7abb Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable
Hi,

I'm trying out giraph with hive= with instructions from http= ://giraph.apache.org/hive.html and I get the following error while read= ing data from hive


hadoop jar girap= hstarter-jar-with-dependencies.jar org.apache.giraph.hive.HiveGiraphRunner = -hiveconf hive.metastore.uris=3Dthrift://..:50513 -hiveconf hive.metastore.= sasl.enabled=3Dtrue -hiveconf hive.metastore.kerberos.principal=3Dhcat/... = =A0-db=3Ddancehall -ei=3D"com.yahoo.IdMappingsHiveToEdge, id_mappings,= source=3Dinterstitials, dt=3D2015022323" -o "com.yahoo.IdCluster= sEdgeToHive, id_giraph, source=3Dinterstitials, dt=3D2015022323" --com= putationClass=3Dcom.yahoo.HiveConnectedComponents =A0-w 1
15/02/27 04:18:58 INFO common.HiveMetastores: Connecting t= o metastore at ...50513
15/02/27 04:19:58 INFO common.BackoffRetr= yTask: Failed, but will retry in 25250 msec : org.apache.thrift.transport.T= TransportException: java.net.SocketTimeoutException: Read timed out
15/02/27 04:20:24 INFO common.HiveMetastores: Connecting to metastore at= ..:50513
15/02/27 04:21:24 INFO common.BackoffRetryTask: Failed,= but will retry in 51984 msec : org.apache.thrift.transport.TTransportExcep= tion: java.net.SocketTimeoutException: Read timed out
15/02/27 04= :22:16 INFO common.HiveMetastores: Connecting to metastore at ...:50513
Exception in thread "main" java.lang.IllegalStateException= : initialize: IOException occurred
at com.facebook.hiveio.input.HiveApiInputFormat.initia= lize(HiveApiInputFormat.java:94)
at org.apache.giraph.hive.input.edge.HiveEdgeInputFormat= .setConf(HiveEdgeInputFormat.java:73)
at org.apache.giraph.conf.ImmutableClassesGiraphCon= figuration.configureIfPossible(ImmutableClassesGiraphConfiguration.java:156= )
at org.apach= e.giraph.utils.ConfigurationUtils.configureIfPossible(ConfigurationUtils.ja= va:151)
at org= .apache.giraph.utils.ReflectionUtils.newInstance(ReflectionUtils.java:111)<= /div>
at org.apache.= giraph.conf.ImmutableClassesGiraphConfiguration.createEdgeInputFormat(Immut= ableClassesGiraphConfiguration.java:477)
at org.apache.giraph.conf.ImmutableClassesGira= phConfiguration.createWrappedEdgeInputFormat(ImmutableClassesGiraphConfigur= ation.java:488)
at org.apache.giraph.hive.HiveGiraphRunner.prepareHiveEdgeInputs(HiveGira= phRunner.java:425)
at org.apache.giraph.hive.HiveGiraphRunner.handleCommandLine(HiveGirap= hRunner.java:615)
at org.apache.giraph.hive.HiveGiraphRunner.run(HiveGiraphRunner.java:29= 3)
at org.apac= he.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at org.apache.hadoop.util.ToolRunner.ru= n(ToolRunner.java:84)
= at org.apache.giraph.hive.HiveGiraphRunner.main(HiveGiraphRunner.ja= va:286)
at sun= .reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAcces= sorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorI= mpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Metho= d.java:601)
at= org.apache.hadoop.util.RunJar.run(RunJar.java:221)
at org.apache.hadoop.util.RunJar.ma= in(RunJar.java:136)
Caused by: java.io.IOException: No more retri= es left org.apache.thrift.transport.TTransportException: java.net.SocketTim= eoutException: Read timed out
at com.facebook.hiveio.common.BackoffRetryTask.run(BackoffR= etryTask.java:111)
at com.facebook.hiveio.schema.HiveTableSchemas.lookup(HiveTableSchemas= .java:156)
at = com.facebook.hiveio.schema.HiveTableSchemas.initTableSchema(HiveTableSchema= s.java:97)
at = com.facebook.hiveio.input.HiveApiInputFormat.initialize(HiveApiInputFormat.= java:92)
... 1= 8 more


Any hints around this = ?=A0

I work at Yahoo and here are versions we are = using. Is Giraph and hive-io compatiable with these versions ?=A0
hadoop:=A02.6.0.5.1501231242
Hive:=A00.13.0.6.1502201219

--
Thanks,<= div>Kalyan
--047d7bf18428a4a6ca05100a7abb--