giraph-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Claudio Martella <claudio.marte...@gmail.com>
Subject Re: Running Giraph on YARN (0.23)
Date Thu, 21 Nov 2013 11:17:56 GMT
Are you running with an external ZK, or are you expecting giraph to start
one from you on the master? If the second one is the case (as it seams to
me from the logs), then it won't work. Giraph on Yarn requires you to start
ZK externally, and pass the address to giraph when you start the job.


On Thu, Nov 21, 2013 at 11:49 AM, Gianmarco De Francisci Morales <
gdfm@apache.org> wrote:

> Hi,
>
> I am trying to run Giraph on a Hadoop 0.23 installation, but I am running
> in a communication problem. From the log it seems that the workers are not
> able to connect to the zookeeper instance on the master.
> I am using the option -DuseNetty=true
>
> Any help is appreciated.
> Here the error log:
>
>
>
> ************
>
>
> 2013-11-20 15:22:22,000 INFO [main] org.apache.giraph.graph.GraphTaskManager:
> setup: classpath @ /user/bordino/.staging/job_1382563758657_470916/job.jar for
> job Giraph: org.apache.giraph.examples.RandomWalkWithRestartComputation
> 2013-11-20 15:22:22,003 WARN [main] org.apache.hadoop.conf.Configuration:
> mapred.task.partition is deprecated. Instead, use mapreduce.task.partition
> 2013-11-20 15:22:22,003 WARN [main] org.apache.hadoop.conf.Configuration:mapred.job.id
is deprecated. Instead, use mapreduce.job.id
> 2013-11-20 15:22:22,004 WARN [main] org.apache.hadoop.conf.Configuration:
> job.local.dir is deprecated. Instead, use mapreduce.job.local.dir
> 2013-11-20 15:22:22,007 INFO [main] org.apache.giraph.zk.ZooKeeperManager:
> createCandidateStamp: Made the directory
> _bsp/_defaultZkManagerDir/job_1382563758657_470916
> 2013-11-20 15:22:22,008 INFO [main] org.apache.giraph.zk.ZooKeeperManager:
> createCandidateStamp: Made the directory
> _bsp/_defaultZkManagerDir/job_1382563758657_470916/_zkServer
> 2013-11-20 15:22:22,011 INFO [main] org.apache.giraph.zk.ZooKeeperManager:
> createCandidateStamp: Creating my filestamp
> _bsp/_defaultZkManagerDir/job_1382563758657_470916/_task/gsta60065.tan.ygrid.yahoo.com<http://tan.ygrid.yahoo.com/>
> 56
> 2013-11-20 15:22:22,027 INFO [main] org.apache.giraph.zk.ZooKeeperManager:
> getZooKeeperServerList: For task 56, got file 'null' (polling period is 3000)
> 2013-11-20 15:22:25,030 INFO [main] org.apache.giraph.zk.ZooKeeperManager:
> getZooKeeperServerList: For task 56, got file
> 'zkServerList_gsta60065.tan.ygrid.yahoo.com<http://tan.ygrid.yahoo.com/> 56 '
> (polling period is 3000)
> 2013-11-20 15:22:25,030 INFO [main] org.apache.giraph.zk.ZooKeeperManager:
> getZooKeeperServerList: Found
> [gsta60065.tan.ygrid.yahoo.com<http://tan.ygrid.yahoo.com/>, 56] 2 hosts in
> filename
> 'zkServerList_gsta60065.tan.ygrid.yahoo.com<http://tan.ygrid.yahoo.com/> 56 '
> 2013-11-20 15:22:25,031 INFO [main] org.apache.giraph.zk.ZooKeeperManager:
> onlineZooKeeperServers: Trying to delete old directory
> /grid/0/tmp/yarn-local/usercache/bordino/appcache/application_1382563758657_470916/work/_bspZooKeeper
> 2013-11-20 15:22:25,038 INFO [main] org.apache.giraph.zk.ZooKeeperManager:
> generateZooKeeperConfigFile: Creating file
> /grid/0/tmp/yarn-local/usercache/bordino/appcache/application_1382563758657_470916/work/_bspZooKeeper/zoo.cfg
> in
> /grid/0/tmp/yarn-local/usercache/bordino/appcache/application_1382563758657_470916/work/_bspZooKeeper
> with base port 22181
> 2013-11-20 15:22:25,038 INFO [main] org.apache.giraph.zk.ZooKeeperManager:
> generateZooKeeperConfigFile: Make directory of _bspZooKeeper = true
> 2013-11-20 15:22:25,039 INFO [main] org.apache.giraph.zk.ZooKeeperManager:
> generateZooKeeperConfigFile: Delete of zoo.cfg = false
> 2013-11-20 15:22:25,041 INFO [main] org.apache.giraph.zk.ZooKeeperManager:
> onlineZooKeeperServers: Attempting to start ZooKeeper server with command
> [/home/Releases/gridjdk-1.7.0_17.1303042057-20130415-000/share/gridjdk-1.7.0_17/jre/bin/java,
> -Xmx128M, -XX:ParallelGCThreads=4, -XX:+UseConcMarkSweepGC,
> -XX:CMSInitiatingOccupancyFraction=70, -XX:MaxGCPauseMillis=100, -cp,
> /user/bordino/.staging/job_1382563758657_470916/job.jar,
> org.apache.zookeeper.server.quorum.QuorumPeerMain,
> /grid/0/tmp/yarn-local/usercache/bordino/appcache/application_1382563758657_470916/work/_bspZooKeeper/zoo.cfg]
> in directory
> /grid/0/tmp/yarn-local/usercache/bordino/appcache/application_1382563758657_470916/work/_bspZooKeeper
> 2013-11-20 15:22:25,044 INFO [main] org.apache.giraph.zk.ZooKeeperManager:
> onlineZooKeeperServers: Shutdown hook added.
> 2013-11-20 15:22:25,044 INFO [main] org.apache.giraph.zk.ZooKeeperManager:
> onlineZooKeeperServers: Connect attempt 0 of 10 max trying to connect togsta60065.tan.ygrid.yahoo.com<http://tan.ygrid.yahoo.com/>:22181
with poll
> msecs = 3000
> 2013-11-20 15:22:25,048 WARN [main] org.apache.giraph.zk.ZooKeeperManager:
> onlineZooKeeperServers: Got ConnectException
> java.net.ConnectException: Connection refused
>         at java.net.PlainSocketImpl.socketConnect(Native Method)
>         at
> java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
>         at
> java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
>         at
> java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
>         at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391)
>         at java.net.Socket.connect(Socket.java:579)
>         at
> org.apache.giraph.zk.ZooKeeperManager.onlineZooKeeperServers(ZooKeeperManager.java:711)
>         at
> org.apache.giraph.graph.GraphTaskManager.startZooKeeperManager(GraphTaskManager.java:369)
>         at
> org.apache.giraph.graph.GraphTaskManager.setup(GraphTaskManager.java:202)
>         at org.apache.giraph.graph.GraphMapper.setup(GraphMapper.java:59)
>         at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:89)
>         at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:726)
>         at org.apache.hadoop.mapred.MapTask.run(MapTask.java:333)
>         at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:158)
>         at java.security.AccessController.doPrivileged(Native Method)
>         at javax.security.auth.Subject.doAs(Subject.java:415)
>         at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1262)
>         at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:153)
> 2013-11-20 15:22:28,050 INFO [main] org.apache.giraph.zk.ZooKeeperManager:
> onlineZooKeeperServers: Connect attempt 1 of 10 max trying to connect togsta60065.tan.ygrid.yahoo.com<http://tan.ygrid.yahoo.com/>:22181
with poll
> msecs = 3000
> 2013-11-20 15:22:28,050 WARN [main] org.apache.giraph.zk.ZooKeeperManager:
> onlineZooKeeperServers: Got ConnectException
> java.net.ConnectException: Connection refused
>         at java.net.PlainSocketImpl.socketConnect(Native Method)
>         at
> java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
>         at
> java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
>         at
> java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
>         at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391)
>         at java.net.Socket.connect(Socket.java:579)
>         at
> org.apache.giraph.zk.ZooKeeperManager.onlineZooKeeperServers(ZooKeeperManager.java:711)
>         at
> org.apache.giraph.graph.GraphTaskManager.startZooKeeperManager(GraphTaskManager.java:369)
>         at
> org.apache.giraph.graph.GraphTaskManager.setup(GraphTaskManager.java:202)
>         at org.apache.giraph.graph.GraphMapper.setup(GraphMapper.java:59)
>         at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:89)
>         at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:726)
>         at org.apache.hadoop.mapred.MapTask.run(MapTask.java:333)
>         at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:158)
>         at java.security.AccessController.doPrivileged(Native Method)
>         at javax.security.auth.Subject.doAs(Subject.java:415)
>         at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1262)
>         at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:153)
> 2013-11-20 15:22:31,051 INFO [main] org.apache.giraph.zk.ZooKeeperManager:
> onlineZooKeeperServers: Connect attempt 2 of 10 max trying to connect togsta60065.tan.ygrid.yahoo.com<http://tan.ygrid.yahoo.com/>:22181
with poll
> msecs = 3000
> 2013-11-20 15:22:31,051 WARN [main] org.apache.giraph.zk.ZooKeeperManager:
> onlineZooKeeperServers: Got ConnectException
> java.net.ConnectException: Connection refused
>         at java.net.PlainSocketImpl.socketConnect(Native Method)
>         at
> java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
>         at
> java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
>         at
> java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
>         at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391)
>         at java.net.Socket.connect(Socket.java:579)
>         at
> org.apache.giraph.zk.ZooKeeperManager.onlineZooKeeperServers(ZooKeeperManager.java:711)
>         at
> org.apache.giraph.graph.GraphTaskManager.startZooKeeperManager(GraphTaskManager.java:369)
>         at
> org.apache.giraph.graph.GraphTaskManager.setup(GraphTaskManager.java:202)
>         at org.apache.giraph.graph.GraphMapper.setup(GraphMapper.java:59)
>         at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:89)
>         at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:726)
>         at org.apache.hadoop.mapred.MapTask.run(MapTask.java:333)
>         at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:158)
>         at java.security.AccessController.doPrivileged(Native Method)
>         at javax.security.auth.Subject.doAs(Subject.java:415)
>         at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1262)
>         at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:153)
> 2013-11-20 15:22:34,052 INFO [main] org.apache.giraph.zk.ZooKeeperManager:
> onlineZooKeeperServers: Connect attempt 3 of 10 max trying to connect togsta60065.tan.ygrid.yahoo.com<http://tan.ygrid.yahoo.com/>:22181
with poll
> msecs = 3000
> 2013-11-20 15:22:34,052 WARN [main] org.apache.giraph.zk.ZooKeeperManager:
> onlineZooKeeperServers: Got ConnectException
> java.net.ConnectException: Connection refused
>         at java.net.PlainSocketImpl.socketConnect(Native Method)
>         at
> java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
>         at
> java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
>         at
> java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
>         at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391)
>         at java.net.Socket.connect(Socket.java:579)
>         at
> org.apache.giraph.zk.ZooKeeperManager.onlineZooKeeperServers(ZooKeeperManager.java:711)
>         at
> org.apache.giraph.graph.GraphTaskManager.startZooKeeperManager(GraphTaskManager.java:369)
>         at
> org.apache.giraph.graph.GraphTaskManager.setup(GraphTaskManager.java:202)
>         at org.apache.giraph.graph.GraphMapper.setup(GraphMapper.java:59)
>         at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:89)
>         at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:726)
>         at org.apache.hadoop.mapred.MapTask.run(MapTask.java:333)
>         at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:158)
>         at java.security.AccessController.doPrivileged(Native Method)
>         at javax.security.auth.Subject.doAs(Subject.java:415)
>         at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1262)
>         at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:153)
> 2013-11-20 15:22:37,053 INFO [main] org.apache.giraph.zk.ZooKeeperManager:
> onlineZooKeeperServers: Connect attempt 4 of 10 max trying to connect togsta60065.tan.ygrid.yahoo.com<http://tan.ygrid.yahoo.com/>:22181
with poll
> msecs = 3000
> 2013-11-20 15:22:37,053 WARN [main] org.apache.giraph.zk.ZooKeeperManager:
> onlineZooKeeperServers: Got ConnectException
> java.net.ConnectException: Connection refused
>         at java.net.PlainSocketImpl.socketConnect(Native Method)
>         at
> java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
>         at
> java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
>         at
> java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
>         at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391)
>         at java.net.Socket.connect(Socket.java:579)
>         at
> org.apache.giraph.zk.ZooKeeperManager.onlineZooKeeperServers(ZooKeeperManager.java:711)
>         at
> org.apache.giraph.graph.GraphTaskManager.startZooKeeperManager(GraphTaskManager.java:369)
>         at
> org.apache.giraph.graph.GraphTaskManager.setup(GraphTaskManager.java:202)
>         at org.apache.giraph.graph.GraphMapper.setup(GraphMapper.java:59)
>         at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:89)
>         at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:726)
>         at org.apache.hadoop.mapred.MapTask.run(MapTask.java:333)
>         at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:158)
>         at java.security.AccessController.doPrivileged(Native Method)
>         at javax.security.auth.Subject.doAs(Subject.java:415)
>         at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1262)
>         at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:153)
> 2013-11-20 15:22:40,054 INFO [main] org.apache.giraph.zk.ZooKeeperManager:
> onlineZooKeeperServers: Connect attempt 5 of 10 max trying to connect togsta60065.tan.ygrid.yahoo.com<http://tan.ygrid.yahoo.com/>:22181
with poll
> msecs = 3000
> 2013-11-20 15:22:40,054 WARN [main] org.apache.giraph.zk.ZooKeeperManager:
> onlineZooKeeperServers: Got ConnectException
> java.net.ConnectException: Connection refused
>         at java.net.PlainSocketImpl.socketConnect(Native Method)
>         at
> java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
>         at
> java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
>         at
> java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
>         at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391)
>         at java.net.Socket.connect(Socket.java:579)
>         at
> org.apache.giraph.zk.ZooKeeperManager.onlineZooKeeperServers(ZooKeeperManager.java:711)
>         at
> org.apache.giraph.graph.GraphTaskManager.startZooKeeperManager(GraphTaskManager.java:369)
>         at
> org.apache.giraph.graph.GraphTaskManager.setup(GraphTaskManager.java:202)
>         at org.apache.giraph.graph.GraphMapper.setup(GraphMapper.java:59)
>         at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:89)
>         at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:726)
>         at org.apache.hadoop.mapred.MapTask.run(MapTask.java:333)
>         at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:158)
>         at java.security.AccessController.doPrivileged(Native Method)
>         at javax.security.auth.Subject.doAs(Subject.java:415)
>         at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1262)
>         at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:153)
> 2013-11-20 15:22:43,055 INFO [main] org.apache.giraph.zk.ZooKeeperManager:
> onlineZooKeeperServers: Connect attempt 6 of 10 max trying to connect togsta60065.tan.ygrid.yahoo.com<http://tan.ygrid.yahoo.com/>:22181
with poll
> msecs = 3000
> 2013-11-20 15:22:43,055 WARN [main] org.apache.giraph.zk.ZooKeeperManager:
> onlineZooKeeperServers: Got ConnectException
> java.net.ConnectException: Connection refused
>         at java.net.PlainSocketImpl.socketConnect(Native Method)
>         at
> java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
>         at
> java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
>         at
> java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
>         at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391)
>         at java.net.Socket.connect(Socket.java:579)
>         at
> org.apache.giraph.zk.ZooKeeperManager.onlineZooKeeperServers(ZooKeeperManager.java:711)
>         at
> org.apache.giraph.graph.GraphTaskManager.startZooKeeperManager(GraphTaskManager.java:369)
>         at
> org.apache.giraph.graph.GraphTaskManager.setup(GraphTaskManager.java:202)
>         at org.apache.giraph.graph.GraphMapper.setup(GraphMapper.java:59)
>         at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:89)
>         at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:726)
>         at org.apache.hadoop.mapred.MapTask.run(MapTask.java:333)
>         at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:158)
>         at java.security.AccessController.doPrivileged(Native Method)
>         at javax.security.auth.Subject.doAs(Subject.java:415)
>         at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1262)
>         at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:153)
> 2013-11-20 15:22:46,055 INFO [main] org.apache.giraph.zk.ZooKeeperManager:
> onlineZooKeeperServers: Connect attempt 7 of 10 max trying to connect togsta60065.tan.ygrid.yahoo.com<http://tan.ygrid.yahoo.com/>:22181
with poll
> msecs = 3000
> 2013-11-20 15:22:46,056 WARN [main] org.apache.giraph.zk.ZooKeeperManager:
> onlineZooKeeperServers: Got ConnectException
> java.net.ConnectException: Connection refused
>         at java.net.PlainSocketImpl.socketConnect(Native Method)
>         at
> java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
>         at
> java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
>         at
> java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
>         at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391)
>         at java.net.Socket.connect(Socket.java:579)
>         at
> org.apache.giraph.zk.ZooKeeperManager.onlineZooKeeperServers(ZooKeeperManager.java:711)
>         at
> org.apache.giraph.graph.GraphTaskManager.startZooKeeperManager(GraphTaskManager.java:369)
>         at
> org.apache.giraph.graph.GraphTaskManager.setup(GraphTaskManager.java:202)
>         at org.apache.giraph.graph.GraphMapper.setup(GraphMapper.java:59)
>         at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:89)
>         at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:726)
>         at org.apache.hadoop.mapred.MapTask.run(MapTask.java:333)
>         at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:158)
>         at java.security.AccessController.doPrivileged(Native Method)
>         at javax.security.auth.Subject.doAs(Subject.java:415)
>         at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1262)
>         at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:153)
> 2013-11-20 15:22:49,056 INFO [main] org.apache.giraph.zk.ZooKeeperManager:
> onlineZooKeeperServers: Connect attempt 8 of 10 max trying to connect togsta60065.tan.ygrid.yahoo.com<http://tan.ygrid.yahoo.com/>:22181
with poll
> msecs = 3000
> 2013-11-20 15:22:49,057 WARN [main] org.apache.giraph.zk.ZooKeeperManager:
> onlineZooKeeperServers: Got ConnectException
> java.net.ConnectException: Connection refused
>         at java.net.PlainSocketImpl.socketConnect(Native Method)
>         at
> java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
>         at
> java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
>         at
> java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
>         at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391)
>         at java.net.Socket.connect(Socket.java:579)
>         at
> org.apache.giraph.zk.ZooKeeperManager.onlineZooKeeperServers(ZooKeeperManager.java:711)
>         at
> org.apache.giraph.graph.GraphTaskManager.startZooKeeperManager(GraphTaskManager.java:369)
>         at
> org.apache.giraph.graph.GraphTaskManager.setup(GraphTaskManager.java:202)
>         at org.apache.giraph.graph.GraphMapper.setup(GraphMapper.java:59)
>         at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:89)
>         at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:726)
>         at org.apache.hadoop.mapred.MapTask.run(MapTask.java:333)
>         at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:158)
>         at java.security.AccessController.doPrivileged(Native Method)
>         at javax.security.auth.Subject.doAs(Subject.java:415)
>         at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1262)
>         at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:153)
> 2013-11-20 15:22:52,057 INFO [main] org.apache.giraph.zk.ZooKeeperManager:
> onlineZooKeeperServers: Connect attempt 9 of 10 max trying to connect togsta60065.tan.ygrid.yahoo.com<http://tan.ygrid.yahoo.com/>:22181
with poll
> msecs = 3000
> 2013-11-20 15:22:52,058 WARN [main] org.apache.giraph.zk.ZooKeeperManager:
> onlineZooKeeperServers: Got ConnectException
> java.net.ConnectException: Connection refused
>         at java.net.PlainSocketImpl.socketConnect(Native Method)
>         at
> java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
>         at
> java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
>         at
> java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
>         at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391)
>         at java.net.Socket.connect(Socket.java:579)
>         at
> org.apache.giraph.zk.ZooKeeperManager.onlineZooKeeperServers(ZooKeeperManager.java:711)
>         at
> org.apache.giraph.graph.GraphTaskManager.startZooKeeperManager(GraphTaskManager.java:369)
>         at
> org.apache.giraph.graph.GraphTaskManager.setup(GraphTaskManager.java:202)
>         at org.apache.giraph.graph.GraphMapper.setup(GraphMapper.java:59)
>         at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:89)
>         at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:726)
>         at org.apache.hadoop.mapred.MapTask.run(MapTask.java:333)
>         at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:158)
>         at java.security.AccessController.doPrivileged(Native Method)
>         at javax.security.auth.Subject.doAs(Subject.java:415)
>         at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1262)
>         at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:153)
> 2013-11-20 15:22:55,060 WARN [main] org.apache.hadoop.mapred.YarnChild:
> Exception running child : java.lang.IllegalStateException: run: Caught an
> unrecoverable exception onlineZooKeeperServers: Failed to connect in 10 tries!
>         at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:101)
>         at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:726)
>         at org.apache.hadoop.mapred.MapTask.run(MapTask.java:333)
>         at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:158)
>         at java.security.AccessController.doPrivileged(Native Method)
>         at javax.security.auth.Subject.doAs(Subject.java:415)
>         at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1262)
>         at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:153)
> Caused by: java.lang.IllegalStateException: onlineZooKeeperServers: Failed to
> connect in 10 tries!
>         at
> org.apache.giraph.zk.ZooKeeperManager.onlineZooKeeperServers(ZooKeeperManager.java:737)
>         at
> org.apache.giraph.graph.GraphTaskManager.startZooKeeperManager(GraphTaskManager.java:369)
>         at
> org.apache.giraph.graph.GraphTaskManager.setup(GraphTaskManager.java:202)
>         at org.apache.giraph.graph.GraphMapper.setup(GraphMapper.java:59)
>         at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:89)
>         ... 7 more
>
> 2013-11-20 15:22:55,063 INFO [main] org.apache.hadoop.mapred.Task: Runnning
> cleanup for the task
> 2013-11-20 15:22:55,110 WARN [main] org.apache.hadoop.conf.Configuration:
> job.xml:an attempt to override final parameter: dfs.namenode.name.dir;
> Ignoring.
> 2013-11-20 15:22:55,110 WARN [main] org.apache.hadoop.conf.Configuration:
> job.xml:an attempt to override final parameter:
> hadoop.security.token.service.use_ip;  Ignoring.
> 2013-11-20 15:22:55,110 WARN [main] org.apache.hadoop.conf.Configuration:
> job.xml:an attempt to override final parameter:
> mapreduce.job.end-notification.max.retry.interval;  Ignoring.
> 2013-11-20 15:22:55,110 WARN [main] org.apache.hadoop.conf.Configuration:
> job.xml:an attempt to override final parameter:
> mapreduce.admin.map.child.java.opts;  Ignoring.
> 2013-11-20 15:22:55,111 WARN [main] org.apache.hadoop.conf.Configuration:
> job.xml:an attempt to override final parameter:
> mapreduce.job.end-notification.max.attempts;  Ignoring.
> 2013-11-20 15:22:55,111 WARN [main] org.apache.hadoop.conf.Configuration:
> job.xml:an attempt to override final parameter:
> yarn.app.mapreduce.am.job.client.port-range;  Ignoring.
> 2013-11-20 15:22:55,111 WARN [main] org.apache.hadoop.conf.Configuration:
> job.xml:an attempt to override final parameter:
> mapreduce.admin.reduce.child.java.opts;  Ignoring.
> 2013-11-20 15:22:55,111 WARN [main] org.apache.hadoop.conf.Configuration:
> job.xml:an attempt to override final parameter: hadoop.tmp.dir;  Ignoring.
> 2013-11-20 15:22:55,117 WARN [main]
> org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter: Could not delete
> hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/bordino/test-giraph-tmp/_temporary/1/_temporary/attempt_1382563758657_470916_m_000056_0
> 2013-11-20 <http://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/bordino/test-giraph-tmp/_temporary/1/_temporary/attempt_1382563758657_470916_m_000056_02013-11-20>
15:22:55,121 INFO [main]
> org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Stopping MapTask metrics
> system...
> 2013-11-20 15:22:55,122 INFO [main]
> org.apache.hadoop.metrics2.impl.MetricsSystemImpl: MapTask metrics system
> stopped.
> 2013-11-20 15:22:55,122 INFO [main]
> org.apache.hadoop.metrics2.impl.MetricsSystemImpl: MapTask metrics system
> shutdown complete.
>
>
>
> Cheres,
> --
> Gianmarco
>



-- 
   Claudio Martella
   claudio.martella@gmail.com

Mime
View raw message