giraph-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Kalyan Kanuri <kalyan.ku...@gmail.com>
Subject giraph hive example and versions
Date Fri, 27 Feb 2015 04:41:16 GMT
Hi,

I'm trying out giraph with hive with instructions from
http://giraph.apache.org/hive.html and I get the following error while
reading data from hive


hadoop jar giraphstarter-jar-with-dependencies.jar
org.apache.giraph.hive.HiveGiraphRunner -hiveconf
hive.metastore.uris=thrift://..:50513 -hiveconf
hive.metastore.sasl.enabled=true -hiveconf
hive.metastore.kerberos.principal=hcat/...  -db=dancehall
-ei="com.yahoo.IdMappingsHiveToEdge, id_mappings, source=interstitials,
dt=2015022323" -o "com.yahoo.IdClustersEdgeToHive, id_giraph,
source=interstitials, dt=2015022323"
--computationClass=com.yahoo.HiveConnectedComponents  -w 1

15/02/27 04:18:58 INFO common.HiveMetastores: Connecting to metastore at
...50513
15/02/27 04:19:58 INFO common.BackoffRetryTask: Failed, but will retry in
25250 msec : org.apache.thrift.transport.TTransportException:
java.net.SocketTimeoutException: Read timed out
15/02/27 04:20:24 INFO common.HiveMetastores: Connecting to metastore at
..:50513
15/02/27 04:21:24 INFO common.BackoffRetryTask: Failed, but will retry in
51984 msec : org.apache.thrift.transport.TTransportException:
java.net.SocketTimeoutException: Read timed out
15/02/27 04:22:16 INFO common.HiveMetastores: Connecting to metastore at
...:50513
Exception in thread "main" java.lang.IllegalStateException: initialize:
IOException occurred
at
com.facebook.hiveio.input.HiveApiInputFormat.initialize(HiveApiInputFormat.java:94)
at
org.apache.giraph.hive.input.edge.HiveEdgeInputFormat.setConf(HiveEdgeInputFormat.java:73)
at
org.apache.giraph.conf.ImmutableClassesGiraphConfiguration.configureIfPossible(ImmutableClassesGiraphConfiguration.java:156)
at
org.apache.giraph.utils.ConfigurationUtils.configureIfPossible(ConfigurationUtils.java:151)
at
org.apache.giraph.utils.ReflectionUtils.newInstance(ReflectionUtils.java:111)
at
org.apache.giraph.conf.ImmutableClassesGiraphConfiguration.createEdgeInputFormat(ImmutableClassesGiraphConfiguration.java:477)
at
org.apache.giraph.conf.ImmutableClassesGiraphConfiguration.createWrappedEdgeInputFormat(ImmutableClassesGiraphConfiguration.java:488)
at
org.apache.giraph.hive.HiveGiraphRunner.prepareHiveEdgeInputs(HiveGiraphRunner.java:425)
at
org.apache.giraph.hive.HiveGiraphRunner.handleCommandLine(HiveGiraphRunner.java:615)
at org.apache.giraph.hive.HiveGiraphRunner.run(HiveGiraphRunner.java:293)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
at org.apache.giraph.hive.HiveGiraphRunner.main(HiveGiraphRunner.java:286)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
Caused by: java.io.IOException: No more retries left
org.apache.thrift.transport.TTransportException:
java.net.SocketTimeoutException: Read timed out
at
com.facebook.hiveio.common.BackoffRetryTask.run(BackoffRetryTask.java:111)
at
com.facebook.hiveio.schema.HiveTableSchemas.lookup(HiveTableSchemas.java:156)
at
com.facebook.hiveio.schema.HiveTableSchemas.initTableSchema(HiveTableSchemas.java:97)
at
com.facebook.hiveio.input.HiveApiInputFormat.initialize(HiveApiInputFormat.java:92)
... 18 more


Any hints around this ?

I work at Yahoo and here are versions we are using. Is Giraph and hive-io
compatiable with these versions ?
hadoop: 2.6.0.5.1501231242
Hive: 0.13.0.6.1502201219

-- 
Thanks,
Kalyan

Mime
View raw message