giraph-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Gianmarco De Francisci Morales <g...@apache.org>
Subject Re: Running Giraph on YARN (0.23)
Date Thu, 21 Nov 2013 13:45:04 GMT
Indeed I am expecting Giraph to do it for me. Can I ask what is the reason
for this difference in YARN?
It makes deploying Giraph much more complicated.

--
Gianmarco


On 21 November 2013 12:17, Claudio Martella <claudio.martella@gmail.com>wrote:

> Are you running with an external ZK, or are you expecting giraph to start
> one from you on the master? If the second one is the case (as it seams to
> me from the logs), then it won't work. Giraph on Yarn requires you to start
> ZK externally, and pass the address to giraph when you start the job.
>
>
> On Thu, Nov 21, 2013 at 11:49 AM, Gianmarco De Francisci Morales <
> gdfm@apache.org> wrote:
>
>> Hi,
>>
>> I am trying to run Giraph on a Hadoop 0.23 installation, but I am running
>> in a communication problem. From the log it seems that the workers are not
>> able to connect to the zookeeper instance on the master.
>> I am using the option -DuseNetty=true
>>
>> Any help is appreciated.
>> Here the error log:
>>
>>
>>
>> ************
>>
>>
>> 2013-11-20 15:22:22,000 INFO [main] org.apache.giraph.graph.GraphTaskManager:
>> setup: classpath @ /user/bordino/.staging/job_1382563758657_470916/job.jar for
>> job Giraph: org.apache.giraph.examples.RandomWalkWithRestartComputation
>> 2013-11-20 15:22:22,003 WARN [main] org.apache.hadoop.conf.Configuration:
>> mapred.task.partition is deprecated. Instead, use mapreduce.task.partition
>> 2013-11-20 15:22:22,003 WARN [main] org.apache.hadoop.conf.Configuration:mapred.job.id
is deprecated. Instead, use mapreduce.job.id
>> 2013-11-20 15:22:22,004 WARN [main] org.apache.hadoop.conf.Configuration:
>> job.local.dir is deprecated. Instead, use mapreduce.job.local.dir
>> 2013-11-20 15:22:22,007 INFO [main] org.apache.giraph.zk.ZooKeeperManager:
>> createCandidateStamp: Made the directory
>> _bsp/_defaultZkManagerDir/job_1382563758657_470916
>> 2013-11-20 15:22:22,008 INFO [main] org.apache.giraph.zk.ZooKeeperManager:
>> createCandidateStamp: Made the directory
>> _bsp/_defaultZkManagerDir/job_1382563758657_470916/_zkServer
>> 2013-11-20 15:22:22,011 INFO [main] org.apache.giraph.zk.ZooKeeperManager:
>> createCandidateStamp: Creating my filestamp
>> _bsp/_defaultZkManagerDir/job_1382563758657_470916/_task/gsta60065.tan.ygrid.yahoo.com<http://tan.ygrid.yahoo.com/>
>> 56
>> 2013-11-20 15:22:22,027 INFO [main] org.apache.giraph.zk.ZooKeeperManager:
>> getZooKeeperServerList: For task 56, got file 'null' (polling period is 3000)
>> 2013-11-20 15:22:25,030 INFO [main] org.apache.giraph.zk.ZooKeeperManager:
>> getZooKeeperServerList: For task 56, got file
>> 'zkServerList_gsta60065.tan.ygrid.yahoo.com<http://tan.ygrid.yahoo.com/> 56
'
>> (polling period is 3000)
>> 2013-11-20 15:22:25,030 INFO [main] org.apache.giraph.zk.ZooKeeperManager:
>> getZooKeeperServerList: Found
>> [gsta60065.tan.ygrid.yahoo.com<http://tan.ygrid.yahoo.com/>, 56] 2 hosts in
>> filename
>> 'zkServerList_gsta60065.tan.ygrid.yahoo.com<http://tan.ygrid.yahoo.com/> 56
'
>> 2013-11-20 15:22:25,031 INFO [main] org.apache.giraph.zk.ZooKeeperManager:
>> onlineZooKeeperServers: Trying to delete old directory
>> /grid/0/tmp/yarn-local/usercache/bordino/appcache/application_1382563758657_470916/work/_bspZooKeeper
>> 2013-11-20 15:22:25,038 INFO [main] org.apache.giraph.zk.ZooKeeperManager:
>> generateZooKeeperConfigFile: Creating file
>> /grid/0/tmp/yarn-local/usercache/bordino/appcache/application_1382563758657_470916/work/_bspZooKeeper/zoo.cfg
>> in
>> /grid/0/tmp/yarn-local/usercache/bordino/appcache/application_1382563758657_470916/work/_bspZooKeeper
>> with base port 22181
>> 2013-11-20 15:22:25,038 INFO [main] org.apache.giraph.zk.ZooKeeperManager:
>> generateZooKeeperConfigFile: Make directory of _bspZooKeeper = true
>> 2013-11-20 15:22:25,039 INFO [main] org.apache.giraph.zk.ZooKeeperManager:
>> generateZooKeeperConfigFile: Delete of zoo.cfg = false
>> 2013-11-20 15:22:25,041 INFO [main] org.apache.giraph.zk.ZooKeeperManager:
>> onlineZooKeeperServers: Attempting to start ZooKeeper server with command
>> [/home/Releases/gridjdk-1.7.0_17.1303042057-20130415-000/share/gridjdk-1.7.0_17/jre/bin/java,
>> -Xmx128M, -XX:ParallelGCThreads=4, -XX:+UseConcMarkSweepGC,
>> -XX:CMSInitiatingOccupancyFraction=70, -XX:MaxGCPauseMillis=100, -cp,
>> /user/bordino/.staging/job_1382563758657_470916/job.jar,
>> org.apache.zookeeper.server.quorum.QuorumPeerMain,
>> /grid/0/tmp/yarn-local/usercache/bordino/appcache/application_1382563758657_470916/work/_bspZooKeeper/zoo.cfg]
>> in directory
>> /grid/0/tmp/yarn-local/usercache/bordino/appcache/application_1382563758657_470916/work/_bspZooKeeper
>> 2013-11-20 15:22:25,044 INFO [main] org.apache.giraph.zk.ZooKeeperManager:
>> onlineZooKeeperServers: Shutdown hook added.
>> 2013-11-20 15:22:25,044 INFO [main] org.apache.giraph.zk.ZooKeeperManager:
>> onlineZooKeeperServers: Connect attempt 0 of 10 max trying to connect togsta60065.tan.ygrid.yahoo.com<http://tan.ygrid.yahoo.com/>:22181
with poll
>> msecs = 3000
>> 2013-11-20 15:22:25,048 WARN [main] org.apache.giraph.zk.ZooKeeperManager:
>> onlineZooKeeperServers: Got ConnectException
>> java.net.ConnectException: Connection refused
>>         at java.net.PlainSocketImpl.socketConnect(Native Method)
>>         at
>> java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
>>         at
>> java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
>>         at
>> java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
>>         at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391)
>>         at java.net.Socket.connect(Socket.java:579)
>>         at
>> org.apache.giraph.zk.ZooKeeperManager.onlineZooKeeperServers(ZooKeeperManager.java:711)
>>         at
>> org.apache.giraph.graph.GraphTaskManager.startZooKeeperManager(GraphTaskManager.java:369)
>>         at
>> org.apache.giraph.graph.GraphTaskManager.setup(GraphTaskManager.java:202)
>>         at org.apache.giraph.graph.GraphMapper.setup(GraphMapper.java:59)
>>         at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:89)
>>         at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:726)
>>         at org.apache.hadoop.mapred.MapTask.run(MapTask.java:333)
>>         at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:158)
>>         at java.security.AccessController.doPrivileged(Native Method)
>>         at javax.security.auth.Subject.doAs(Subject.java:415)
>>         at
>> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1262)
>>         at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:153)
>> 2013-11-20 15:22:28,050 INFO [main] org.apache.giraph.zk.ZooKeeperManager:
>> onlineZooKeeperServers: Connect attempt 1 of 10 max trying to connect togsta60065.tan.ygrid.yahoo.com<http://tan.ygrid.yahoo.com/>:22181
with poll
>> msecs = 3000
>> 2013-11-20 15:22:28,050 WARN [main] org.apache.giraph.zk.ZooKeeperManager:
>> onlineZooKeeperServers: Got ConnectException
>> java.net.ConnectException: Connection refused
>>         at java.net.PlainSocketImpl.socketConnect(Native Method)
>>         at
>> java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
>>         at
>> java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
>>         at
>> java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
>>         at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391)
>>         at java.net.Socket.connect(Socket.java:579)
>>         at
>> org.apache.giraph.zk.ZooKeeperManager.onlineZooKeeperServers(ZooKeeperManager.java:711)
>>         at
>> org.apache.giraph.graph.GraphTaskManager.startZooKeeperManager(GraphTaskManager.java:369)
>>         at
>> org.apache.giraph.graph.GraphTaskManager.setup(GraphTaskManager.java:202)
>>         at org.apache.giraph.graph.GraphMapper.setup(GraphMapper.java:59)
>>         at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:89)
>>         at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:726)
>>         at org.apache.hadoop.mapred.MapTask.run(MapTask.java:333)
>>         at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:158)
>>         at java.security.AccessController.doPrivileged(Native Method)
>>         at javax.security.auth.Subject.doAs(Subject.java:415)
>>         at
>> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1262)
>>         at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:153)
>> 2013-11-20 15:22:31,051 INFO [main] org.apache.giraph.zk.ZooKeeperManager:
>> onlineZooKeeperServers: Connect attempt 2 of 10 max trying to connect togsta60065.tan.ygrid.yahoo.com<http://tan.ygrid.yahoo.com/>:22181
with poll
>> msecs = 3000
>> 2013-11-20 15:22:31,051 WARN [main] org.apache.giraph.zk.ZooKeeperManager:
>> onlineZooKeeperServers: Got ConnectException
>> java.net.ConnectException: Connection refused
>>         at java.net.PlainSocketImpl.socketConnect(Native Method)
>>         at
>> java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
>>         at
>> java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
>>         at
>> java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
>>         at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391)
>>         at java.net.Socket.connect(Socket.java:579)
>>         at
>> org.apache.giraph.zk.ZooKeeperManager.onlineZooKeeperServers(ZooKeeperManager.java:711)
>>         at
>> org.apache.giraph.graph.GraphTaskManager.startZooKeeperManager(GraphTaskManager.java:369)
>>         at
>> org.apache.giraph.graph.GraphTaskManager.setup(GraphTaskManager.java:202)
>>         at org.apache.giraph.graph.GraphMapper.setup(GraphMapper.java:59)
>>         at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:89)
>>         at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:726)
>>         at org.apache.hadoop.mapred.MapTask.run(MapTask.java:333)
>>         at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:158)
>>         at java.security.AccessController.doPrivileged(Native Method)
>>         at javax.security.auth.Subject.doAs(Subject.java:415)
>>         at
>> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1262)
>>         at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:153)
>> 2013-11-20 15:22:34,052 INFO [main] org.apache.giraph.zk.ZooKeeperManager:
>> onlineZooKeeperServers: Connect attempt 3 of 10 max trying to connect togsta60065.tan.ygrid.yahoo.com<http://tan.ygrid.yahoo.com/>:22181
with poll
>> msecs = 3000
>> 2013-11-20 15:22:34,052 WARN [main] org.apache.giraph.zk.ZooKeeperManager:
>> onlineZooKeeperServers: Got ConnectException
>> java.net.ConnectException: Connection refused
>>         at java.net.PlainSocketImpl.socketConnect(Native Method)
>>         at
>> java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
>>         at
>> java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
>>         at
>> java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
>>         at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391)
>>         at java.net.Socket.connect(Socket.java:579)
>>         at
>> org.apache.giraph.zk.ZooKeeperManager.onlineZooKeeperServers(ZooKeeperManager.java:711)
>>         at
>> org.apache.giraph.graph.GraphTaskManager.startZooKeeperManager(GraphTaskManager.java:369)
>>         at
>> org.apache.giraph.graph.GraphTaskManager.setup(GraphTaskManager.java:202)
>>         at org.apache.giraph.graph.GraphMapper.setup(GraphMapper.java:59)
>>         at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:89)
>>         at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:726)
>>         at org.apache.hadoop.mapred.MapTask.run(MapTask.java:333)
>>         at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:158)
>>         at java.security.AccessController.doPrivileged(Native Method)
>>         at javax.security.auth.Subject.doAs(Subject.java:415)
>>         at
>> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1262)
>>         at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:153)
>> 2013-11-20 15:22:37,053 INFO [main] org.apache.giraph.zk.ZooKeeperManager:
>> onlineZooKeeperServers: Connect attempt 4 of 10 max trying to connect togsta60065.tan.ygrid.yahoo.com<http://tan.ygrid.yahoo.com/>:22181
with poll
>> msecs = 3000
>> 2013-11-20 15:22:37,053 WARN [main] org.apache.giraph.zk.ZooKeeperManager:
>> onlineZooKeeperServers: Got ConnectException
>> java.net.ConnectException: Connection refused
>>         at java.net.PlainSocketImpl.socketConnect(Native Method)
>>         at
>> java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
>>         at
>> java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
>>         at
>> java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
>>         at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391)
>>         at java.net.Socket.connect(Socket.java:579)
>>         at
>> org.apache.giraph.zk.ZooKeeperManager.onlineZooKeeperServers(ZooKeeperManager.java:711)
>>         at
>> org.apache.giraph.graph.GraphTaskManager.startZooKeeperManager(GraphTaskManager.java:369)
>>         at
>> org.apache.giraph.graph.GraphTaskManager.setup(GraphTaskManager.java:202)
>>         at org.apache.giraph.graph.GraphMapper.setup(GraphMapper.java:59)
>>         at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:89)
>>         at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:726)
>>         at org.apache.hadoop.mapred.MapTask.run(MapTask.java:333)
>>         at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:158)
>>         at java.security.AccessController.doPrivileged(Native Method)
>>         at javax.security.auth.Subject.doAs(Subject.java:415)
>>         at
>> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1262)
>>         at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:153)
>> 2013-11-20 15:22:40,054 INFO [main] org.apache.giraph.zk.ZooKeeperManager:
>> onlineZooKeeperServers: Connect attempt 5 of 10 max trying to connect togsta60065.tan.ygrid.yahoo.com<http://tan.ygrid.yahoo.com/>:22181
with poll
>> msecs = 3000
>> 2013-11-20 15:22:40,054 WARN [main] org.apache.giraph.zk.ZooKeeperManager:
>> onlineZooKeeperServers: Got ConnectException
>> java.net.ConnectException: Connection refused
>>         at java.net.PlainSocketImpl.socketConnect(Native Method)
>>         at
>> java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
>>         at
>> java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
>>         at
>> java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
>>         at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391)
>>         at java.net.Socket.connect(Socket.java:579)
>>         at
>> org.apache.giraph.zk.ZooKeeperManager.onlineZooKeeperServers(ZooKeeperManager.java:711)
>>         at
>> org.apache.giraph.graph.GraphTaskManager.startZooKeeperManager(GraphTaskManager.java:369)
>>         at
>> org.apache.giraph.graph.GraphTaskManager.setup(GraphTaskManager.java:202)
>>         at org.apache.giraph.graph.GraphMapper.setup(GraphMapper.java:59)
>>         at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:89)
>>         at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:726)
>>         at org.apache.hadoop.mapred.MapTask.run(MapTask.java:333)
>>         at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:158)
>>         at java.security.AccessController.doPrivileged(Native Method)
>>         at javax.security.auth.Subject.doAs(Subject.java:415)
>>         at
>> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1262)
>>         at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:153)
>> 2013-11-20 15:22:43,055 INFO [main] org.apache.giraph.zk.ZooKeeperManager:
>> onlineZooKeeperServers: Connect attempt 6 of 10 max trying to connect togsta60065.tan.ygrid.yahoo.com<http://tan.ygrid.yahoo.com/>:22181
with poll
>> msecs = 3000
>> 2013-11-20 15:22:43,055 WARN [main] org.apache.giraph.zk.ZooKeeperManager:
>> onlineZooKeeperServers: Got ConnectException
>> java.net.ConnectException: Connection refused
>>         at java.net.PlainSocketImpl.socketConnect(Native Method)
>>         at
>> java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
>>         at
>> java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
>>         at
>> java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
>>         at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391)
>>         at java.net.Socket.connect(Socket.java:579)
>>         at
>> org.apache.giraph.zk.ZooKeeperManager.onlineZooKeeperServers(ZooKeeperManager.java:711)
>>         at
>> org.apache.giraph.graph.GraphTaskManager.startZooKeeperManager(GraphTaskManager.java:369)
>>         at
>> org.apache.giraph.graph.GraphTaskManager.setup(GraphTaskManager.java:202)
>>         at org.apache.giraph.graph.GraphMapper.setup(GraphMapper.java:59)
>>         at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:89)
>>         at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:726)
>>         at org.apache.hadoop.mapred.MapTask.run(MapTask.java:333)
>>         at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:158)
>>         at java.security.AccessController.doPrivileged(Native Method)
>>         at javax.security.auth.Subject.doAs(Subject.java:415)
>>         at
>> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1262)
>>         at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:153)
>> 2013-11-20 15:22:46,055 INFO [main] org.apache.giraph.zk.ZooKeeperManager:
>> onlineZooKeeperServers: Connect attempt 7 of 10 max trying to connect togsta60065.tan.ygrid.yahoo.com<http://tan.ygrid.yahoo.com/>:22181
with poll
>> msecs = 3000
>> 2013-11-20 15:22:46,056 WARN [main] org.apache.giraph.zk.ZooKeeperManager:
>> onlineZooKeeperServers: Got ConnectException
>> java.net.ConnectException: Connection refused
>>         at java.net.PlainSocketImpl.socketConnect(Native Method)
>>         at
>> java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
>>         at
>> java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
>>         at
>> java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
>>         at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391)
>>         at java.net.Socket.connect(Socket.java:579)
>>         at
>> org.apache.giraph.zk.ZooKeeperManager.onlineZooKeeperServers(ZooKeeperManager.java:711)
>>         at
>> org.apache.giraph.graph.GraphTaskManager.startZooKeeperManager(GraphTaskManager.java:369)
>>         at
>> org.apache.giraph.graph.GraphTaskManager.setup(GraphTaskManager.java:202)
>>         at org.apache.giraph.graph.GraphMapper.setup(GraphMapper.java:59)
>>         at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:89)
>>         at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:726)
>>         at org.apache.hadoop.mapred.MapTask.run(MapTask.java:333)
>>         at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:158)
>>         at java.security.AccessController.doPrivileged(Native Method)
>>         at javax.security.auth.Subject.doAs(Subject.java:415)
>>         at
>> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1262)
>>         at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:153)
>> 2013-11-20 15:22:49,056 INFO [main] org.apache.giraph.zk.ZooKeeperManager:
>> onlineZooKeeperServers: Connect attempt 8 of 10 max trying to connect togsta60065.tan.ygrid.yahoo.com<http://tan.ygrid.yahoo.com/>:22181
with poll
>> msecs = 3000
>> 2013-11-20 15:22:49,057 WARN [main] org.apache.giraph.zk.ZooKeeperManager:
>> onlineZooKeeperServers: Got ConnectException
>> java.net.ConnectException: Connection refused
>>         at java.net.PlainSocketImpl.socketConnect(Native Method)
>>         at
>> java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
>>         at
>> java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
>>         at
>> java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
>>         at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391)
>>         at java.net.Socket.connect(Socket.java:579)
>>         at
>> org.apache.giraph.zk.ZooKeeperManager.onlineZooKeeperServers(ZooKeeperManager.java:711)
>>         at
>> org.apache.giraph.graph.GraphTaskManager.startZooKeeperManager(GraphTaskManager.java:369)
>>         at
>> org.apache.giraph.graph.GraphTaskManager.setup(GraphTaskManager.java:202)
>>         at org.apache.giraph.graph.GraphMapper.setup(GraphMapper.java:59)
>>         at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:89)
>>         at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:726)
>>         at org.apache.hadoop.mapred.MapTask.run(MapTask.java:333)
>>         at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:158)
>>         at java.security.AccessController.doPrivileged(Native Method)
>>         at javax.security.auth.Subject.doAs(Subject.java:415)
>>         at
>> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1262)
>>         at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:153)
>> 2013-11-20 15:22:52,057 INFO [main] org.apache.giraph.zk.ZooKeeperManager:
>> onlineZooKeeperServers: Connect attempt 9 of 10 max trying to connect togsta60065.tan.ygrid.yahoo.com<http://tan.ygrid.yahoo.com/>:22181
with poll
>> msecs = 3000
>> 2013-11-20 15:22:52,058 WARN [main] org.apache.giraph.zk.ZooKeeperManager:
>> onlineZooKeeperServers: Got ConnectException
>> java.net.ConnectException: Connection refused
>>         at java.net.PlainSocketImpl.socketConnect(Native Method)
>>         at
>> java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
>>         at
>> java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
>>         at
>> java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
>>         at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391)
>>         at java.net.Socket.connect(Socket.java:579)
>>         at
>> org.apache.giraph.zk.ZooKeeperManager.onlineZooKeeperServers(ZooKeeperManager.java:711)
>>         at
>> org.apache.giraph.graph.GraphTaskManager.startZooKeeperManager(GraphTaskManager.java:369)
>>         at
>> org.apache.giraph.graph.GraphTaskManager.setup(GraphTaskManager.java:202)
>>         at org.apache.giraph.graph.GraphMapper.setup(GraphMapper.java:59)
>>         at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:89)
>>         at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:726)
>>         at org.apache.hadoop.mapred.MapTask.run(MapTask.java:333)
>>         at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:158)
>>         at java.security.AccessController.doPrivileged(Native Method)
>>         at javax.security.auth.Subject.doAs(Subject.java:415)
>>         at
>> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1262)
>>         at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:153)
>> 2013-11-20 15:22:55,060 WARN [main] org.apache.hadoop.mapred.YarnChild:
>> Exception running child : java.lang.IllegalStateException: run: Caught an
>> unrecoverable exception onlineZooKeeperServers: Failed to connect in 10 tries!
>>         at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:101)
>>         at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:726)
>>         at org.apache.hadoop.mapred.MapTask.run(MapTask.java:333)
>>         at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:158)
>>         at java.security.AccessController.doPrivileged(Native Method)
>>         at javax.security.auth.Subject.doAs(Subject.java:415)
>>         at
>> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1262)
>>         at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:153)
>> Caused by: java.lang.IllegalStateException: onlineZooKeeperServers: Failed to
>> connect in 10 tries!
>>         at
>> org.apache.giraph.zk.ZooKeeperManager.onlineZooKeeperServers(ZooKeeperManager.java:737)
>>         at
>> org.apache.giraph.graph.GraphTaskManager.startZooKeeperManager(GraphTaskManager.java:369)
>>         at
>> org.apache.giraph.graph.GraphTaskManager.setup(GraphTaskManager.java:202)
>>         at org.apache.giraph.graph.GraphMapper.setup(GraphMapper.java:59)
>>         at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:89)
>>         ... 7 more
>>
>> 2013-11-20 15:22:55,063 INFO [main] org.apache.hadoop.mapred.Task: Runnning
>> cleanup for the task
>> 2013-11-20 15:22:55,110 WARN [main] org.apache.hadoop.conf.Configuration:
>> job.xml:an attempt to override final parameter: dfs.namenode.name.dir;
>> Ignoring.
>> 2013-11-20 15:22:55,110 WARN [main] org.apache.hadoop.conf.Configuration:
>> job.xml:an attempt to override final parameter:
>> hadoop.security.token.service.use_ip;  Ignoring.
>> 2013-11-20 15:22:55,110 WARN [main] org.apache.hadoop.conf.Configuration:
>> job.xml:an attempt to override final parameter:
>> mapreduce.job.end-notification.max.retry.interval;  Ignoring.
>> 2013-11-20 15:22:55,110 WARN [main] org.apache.hadoop.conf.Configuration:
>> job.xml:an attempt to override final parameter:
>> mapreduce.admin.map.child.java.opts;  Ignoring.
>> 2013-11-20 15:22:55,111 WARN [main] org.apache.hadoop.conf.Configuration:
>> job.xml:an attempt to override final parameter:
>> mapreduce.job.end-notification.max.attempts;  Ignoring.
>> 2013-11-20 15:22:55,111 WARN [main] org.apache.hadoop.conf.Configuration:
>> job.xml:an attempt to override final parameter:
>> yarn.app.mapreduce.am.job.client.port-range;  Ignoring.
>> 2013-11-20 15:22:55,111 WARN [main] org.apache.hadoop.conf.Configuration:
>> job.xml:an attempt to override final parameter:
>> mapreduce.admin.reduce.child.java.opts;  Ignoring.
>> 2013-11-20 15:22:55,111 WARN [main] org.apache.hadoop.conf.Configuration:
>> job.xml:an attempt to override final parameter: hadoop.tmp.dir;  Ignoring.
>> 2013-11-20 15:22:55,117 WARN [main]
>> org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter: Could not delete
>> hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/bordino/test-giraph-tmp/_temporary/1/_temporary/attempt_1382563758657_470916_m_000056_0
>> 2013-11-20 <http://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/bordino/test-giraph-tmp/_temporary/1/_temporary/attempt_1382563758657_470916_m_000056_02013-11-20>
15:22:55,121 INFO [main]
>> org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Stopping MapTask metrics
>> system...
>> 2013-11-20 15:22:55,122 INFO [main]
>> org.apache.hadoop.metrics2.impl.MetricsSystemImpl: MapTask metrics system
>> stopped.
>> 2013-11-20 15:22:55,122 INFO [main]
>> org.apache.hadoop.metrics2.impl.MetricsSystemImpl: MapTask metrics system
>> shutdown complete.
>>
>>
>>
>> Cheres,
>> --
>> Gianmarco
>>
>
>
>
> --
>    Claudio Martella
>    claudio.martella@gmail.com
>

Mime
View raw message