giraph-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Gianmarco De Francisci Morales <g...@apache.org>
Subject Re: Running Giraph on YARN (0.23)
Date Wed, 27 Nov 2013 13:46:55 GMT
Hi,
Any update on this issue?
Is there any way to bring ZK up automatically on YARN, or any reason why
this is not possible?

Cheers,

--
Gianmarco


On 21 November 2013 15:52, Claudio Martella <claudio.martella@gmail.com>wrote:

> AFAIK YARN does/did not allow to freely spawn new processes in the way we
> used to with M/R.
> Probably Eli and others can give more details.
>
>
>  On Thu, Nov 21, 2013 at 2:45 PM, Gianmarco De Francisci Morales <
> gdfm@apache.org> wrote:
>
>> Indeed I am expecting Giraph to do it for me. Can I ask what is the
>> reason for this difference in YARN?
>> It makes deploying Giraph much more complicated.
>>
>> --
>> Gianmarco
>>
>>
>> On 21 November 2013 12:17, Claudio Martella <claudio.martella@gmail.com>wrote:
>>
>>> Are you running with an external ZK, or are you expecting giraph to
>>> start one from you on the master? If the second one is the case (as it
>>> seams to me from the logs), then it won't work. Giraph on Yarn requires you
>>> to start ZK externally, and pass the address to giraph when you start the
>>> job.
>>>
>>>
>>> On Thu, Nov 21, 2013 at 11:49 AM, Gianmarco De Francisci Morales <
>>> gdfm@apache.org> wrote:
>>>
>>>> Hi,
>>>>
>>>> I am trying to run Giraph on a Hadoop 0.23 installation, but I am
>>>> running in a communication problem. From the log it seems that the workers
>>>> are not able to connect to the zookeeper instance on the master.
>>>> I am using the option -DuseNetty=true
>>>>
>>>> Any help is appreciated.
>>>> Here the error log:
>>>>
>>>>
>>>>
>>>>
>>>> ************
>>>>
>>>>
>>>> 2013-11-20 15:22:22,000 INFO [main] org.apache.giraph.graph.GraphTaskManager:
>>>> setup: classpath @ /user/bordino/.staging/job_1382563758657_470916/job.jar
for
>>>> job Giraph: org.apache.giraph.examples.RandomWalkWithRestartComputation
>>>> 2013-11-20 15:22:22,003 WARN [main] org.apache.hadoop.conf.Configuration:
>>>> mapred.task.partition is deprecated. Instead, use mapreduce.task.partition
>>>> 2013-11-20 15:22:22,003 WARN [main] org.apache.hadoop.conf.Configuration:mapred.job.id
is deprecated. Instead, use mapreduce.job.id
>>>> 2013-11-20 15:22:22,004 WARN [main] org.apache.hadoop.conf.Configuration:
>>>> job.local.dir is deprecated. Instead, use mapreduce.job.local.dir
>>>> 2013-11-20 15:22:22,007 INFO [main] org.apache.giraph.zk.ZooKeeperManager:
>>>> createCandidateStamp: Made the directory
>>>> _bsp/_defaultZkManagerDir/job_1382563758657_470916
>>>> 2013-11-20 15:22:22,008 INFO [main] org.apache.giraph.zk.ZooKeeperManager:
>>>> createCandidateStamp: Made the directory
>>>> _bsp/_defaultZkManagerDir/job_1382563758657_470916/_zkServer
>>>> 2013-11-20 15:22:22,011 INFO [main] org.apache.giraph.zk.ZooKeeperManager:
>>>> createCandidateStamp: Creating my filestamp
>>>> _bsp/_defaultZkManagerDir/job_1382563758657_470916/_task/gsta60065.tan.ygrid.yahoo.com<http://tan.ygrid.yahoo.com/>
>>>> 56
>>>> 2013-11-20 15:22:22,027 INFO [main] org.apache.giraph.zk.ZooKeeperManager:
>>>> getZooKeeperServerList: For task 56, got file 'null' (polling period is 3000)
>>>> 2013-11-20 15:22:25,030 INFO [main] org.apache.giraph.zk.ZooKeeperManager:
>>>> getZooKeeperServerList: For task 56, got file
>>>> 'zkServerList_gsta60065.tan.ygrid.yahoo.com<http://tan.ygrid.yahoo.com/>
56 '
>>>> (polling period is 3000)
>>>> 2013-11-20 15:22:25,030 INFO [main] org.apache.giraph.zk.ZooKeeperManager:
>>>> getZooKeeperServerList: Found
>>>> [gsta60065.tan.ygrid.yahoo.com<http://tan.ygrid.yahoo.com/>, 56] 2
hosts in
>>>> filename
>>>> 'zkServerList_gsta60065.tan.ygrid.yahoo.com<http://tan.ygrid.yahoo.com/>
56 '
>>>> 2013-11-20 15:22:25,031 INFO [main] org.apache.giraph.zk.ZooKeeperManager:
>>>> onlineZooKeeperServers: Trying to delete old directory
>>>> /grid/0/tmp/yarn-local/usercache/bordino/appcache/application_1382563758657_470916/work/_bspZooKeeper
>>>> 2013-11-20 15:22:25,038 INFO [main] org.apache.giraph.zk.ZooKeeperManager:
>>>> generateZooKeeperConfigFile: Creating file
>>>> /grid/0/tmp/yarn-local/usercache/bordino/appcache/application_1382563758657_470916/work/_bspZooKeeper/zoo.cfg
>>>> in
>>>> /grid/0/tmp/yarn-local/usercache/bordino/appcache/application_1382563758657_470916/work/_bspZooKeeper
>>>> with base port 22181
>>>> 2013-11-20 15:22:25,038 INFO [main] org.apache.giraph.zk.ZooKeeperManager:
>>>> generateZooKeeperConfigFile: Make directory of _bspZooKeeper = true
>>>> 2013-11-20 15:22:25,039 INFO [main] org.apache.giraph.zk.ZooKeeperManager:
>>>> generateZooKeeperConfigFile: Delete of zoo.cfg = false
>>>> 2013-11-20 15:22:25,041 INFO [main] org.apache.giraph.zk.ZooKeeperManager:
>>>> onlineZooKeeperServers: Attempting to start ZooKeeper server with command
>>>> [/home/Releases/gridjdk-1.7.0_17.1303042057-20130415-000/share/gridjdk-1.7.0_17/jre/bin/java,
>>>> -Xmx128M, -XX:ParallelGCThreads=4, -XX:+UseConcMarkSweepGC,
>>>> -XX:CMSInitiatingOccupancyFraction=70, -XX:MaxGCPauseMillis=100, -cp,
>>>> /user/bordino/.staging/job_1382563758657_470916/job.jar,
>>>> org.apache.zookeeper.server.quorum.QuorumPeerMain,
>>>> /grid/0/tmp/yarn-local/usercache/bordino/appcache/application_1382563758657_470916/work/_bspZooKeeper/zoo.cfg]
>>>> in directory
>>>> /grid/0/tmp/yarn-local/usercache/bordino/appcache/application_1382563758657_470916/work/_bspZooKeeper
>>>> 2013-11-20 15:22:25,044 INFO [main] org.apache.giraph.zk.ZooKeeperManager:
>>>> onlineZooKeeperServers: Shutdown hook added.
>>>> 2013-11-20 15:22:25,044 INFO [main] org.apache.giraph.zk.ZooKeeperManager:
>>>> onlineZooKeeperServers: Connect attempt 0 of 10 max trying to connect togsta60065.tan.ygrid.yahoo.com<http://tan.ygrid.yahoo.com/>:22181
with poll
>>>> msecs = 3000
>>>> 2013-11-20 15:22:25,048 WARN [main] org.apache.giraph.zk.ZooKeeperManager:
>>>> onlineZooKeeperServers: Got ConnectException
>>>> java.net.ConnectException: Connection refused
>>>>         at java.net.PlainSocketImpl.socketConnect(Native Method)
>>>>         at
>>>> java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
>>>>         at
>>>> java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
>>>>         at
>>>> java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
>>>>         at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391)
>>>>         at java.net.Socket.connect(Socket.java:579)
>>>>         at
>>>> org.apache.giraph.zk.ZooKeeperManager.onlineZooKeeperServers(ZooKeeperManager.java:711)
>>>>         at
>>>> org.apache.giraph.graph.GraphTaskManager.startZooKeeperManager(GraphTaskManager.java:369)
>>>>         at
>>>> org.apache.giraph.graph.GraphTaskManager.setup(GraphTaskManager.java:202)
>>>>         at org.apache.giraph.graph.GraphMapper.setup(GraphMapper.java:59)
>>>>         at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:89)
>>>>         at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:726)
>>>>         at org.apache.hadoop.mapred.MapTask.run(MapTask.java:333)
>>>>         at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:158)
>>>>         at java.security.AccessController.doPrivileged(Native Method)
>>>>         at javax.security.auth.Subject.doAs(Subject.java:415)
>>>>         at
>>>> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1262)
>>>>         at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:153)
>>>> 2013-11-20 15:22:28,050 INFO [main] org.apache.giraph.zk.ZooKeeperManager:
>>>> onlineZooKeeperServers: Connect attempt 1 of 10 max trying to connect togsta60065.tan.ygrid.yahoo.com<http://tan.ygrid.yahoo.com/>:22181
with poll
>>>> msecs = 3000
>>>> 2013-11-20 15:22:28,050 WARN [main] org.apache.giraph.zk.ZooKeeperManager:
>>>> onlineZooKeeperServers: Got ConnectException
>>>> java.net.ConnectException: Connection refused
>>>>         at java.net.PlainSocketImpl.socketConnect(Native Method)
>>>>         at
>>>> java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
>>>>         at
>>>> java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
>>>>         at
>>>> java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
>>>>         at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391)
>>>>         at java.net.Socket.connect(Socket.java:579)
>>>>         at
>>>> org.apache.giraph.zk.ZooKeeperManager.onlineZooKeeperServers(ZooKeeperManager.java:711)
>>>>         at
>>>> org.apache.giraph.graph.GraphTaskManager.startZooKeeperManager(GraphTaskManager.java:369)
>>>>         at
>>>> org.apache.giraph.graph.GraphTaskManager.setup(GraphTaskManager.java:202)
>>>>         at org.apache.giraph.graph.GraphMapper.setup(GraphMapper.java:59)
>>>>         at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:89)
>>>>         at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:726)
>>>>         at org.apache.hadoop.mapred.MapTask.run(MapTask.java:333)
>>>>         at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:158)
>>>>         at java.security.AccessController.doPrivileged(Native Method)
>>>>         at javax.security.auth.Subject.doAs(Subject.java:415)
>>>>         at
>>>> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1262)
>>>>         at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:153)
>>>> 2013-11-20 15:22:31,051 INFO [main] org.apache.giraph.zk.ZooKeeperManager:
>>>> onlineZooKeeperServers: Connect attempt 2 of 10 max trying to connect togsta60065.tan.ygrid.yahoo.com<http://tan.ygrid.yahoo.com/>:22181
with poll
>>>> msecs = 3000
>>>> 2013-11-20 15:22:31,051 WARN [main] org.apache.giraph.zk.ZooKeeperManager:
>>>> onlineZooKeeperServers: Got ConnectException
>>>> java.net.ConnectException: Connection refused
>>>>         at java.net.PlainSocketImpl.socketConnect(Native Method)
>>>>         at
>>>> java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
>>>>         at
>>>> java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
>>>>         at
>>>> java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
>>>>         at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391)
>>>>         at java.net.Socket.connect(Socket.java:579)
>>>>         at
>>>> org.apache.giraph.zk.ZooKeeperManager.onlineZooKeeperServers(ZooKeeperManager.java:711)
>>>>         at
>>>> org.apache.giraph.graph.GraphTaskManager.startZooKeeperManager(GraphTaskManager.java:369)
>>>>         at
>>>> org.apache.giraph.graph.GraphTaskManager.setup(GraphTaskManager.java:202)
>>>>         at org.apache.giraph.graph.GraphMapper.setup(GraphMapper.java:59)
>>>>         at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:89)
>>>>         at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:726)
>>>>         at org.apache.hadoop.mapred.MapTask.run(MapTask.java:333)
>>>>         at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:158)
>>>>         at java.security.AccessController.doPrivileged(Native Method)
>>>>         at javax.security.auth.Subject.doAs(Subject.java:415)
>>>>         at
>>>> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1262)
>>>>         at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:153)
>>>> 2013-11-20 15:22:34,052 INFO [main] org.apache.giraph.zk.ZooKeeperManager:
>>>> onlineZooKeeperServers: Connect attempt 3 of 10 max trying to connect togsta60065.tan.ygrid.yahoo.com<http://tan.ygrid.yahoo.com/>:22181
with poll
>>>> msecs = 3000
>>>> 2013-11-20 15:22:34,052 WARN [main] org.apache.giraph.zk.ZooKeeperManager:
>>>> onlineZooKeeperServers: Got ConnectException
>>>> java.net.ConnectException: Connection refused
>>>>         at java.net.PlainSocketImpl.socketConnect(Native Method)
>>>>         at
>>>> java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
>>>>         at
>>>> java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
>>>>         at
>>>> java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
>>>>         at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391)
>>>>         at java.net.Socket.connect(Socket.java:579)
>>>>         at
>>>> org.apache.giraph.zk.ZooKeeperManager.onlineZooKeeperServers(ZooKeeperManager.java:711)
>>>>         at
>>>> org.apache.giraph.graph.GraphTaskManager.startZooKeeperManager(GraphTaskManager.java:369)
>>>>         at
>>>> org.apache.giraph.graph.GraphTaskManager.setup(GraphTaskManager.java:202)
>>>>         at org.apache.giraph.graph.GraphMapper.setup(GraphMapper.java:59)
>>>>         at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:89)
>>>>         at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:726)
>>>>         at org.apache.hadoop.mapred.MapTask.run(MapTask.java:333)
>>>>         at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:158)
>>>>         at java.security.AccessController.doPrivileged(Native Method)
>>>>         at javax.security.auth.Subject.doAs(Subject.java:415)
>>>>         at
>>>> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1262)
>>>>         at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:153)
>>>> 2013-11-20 15:22:37,053 INFO [main] org.apache.giraph.zk.ZooKeeperManager:
>>>> onlineZooKeeperServers: Connect attempt 4 of 10 max trying to connect togsta60065.tan.ygrid.yahoo.com<http://tan.ygrid.yahoo.com/>:22181
with poll
>>>> msecs = 3000
>>>> 2013-11-20 15:22:37,053 WARN [main] org.apache.giraph.zk.ZooKeeperManager:
>>>> onlineZooKeeperServers: Got ConnectException
>>>> java.net.ConnectException: Connection refused
>>>>         at java.net.PlainSocketImpl.socketConnect(Native Method)
>>>>         at
>>>> java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
>>>>         at
>>>> java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
>>>>         at
>>>> java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
>>>>         at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391)
>>>>         at java.net.Socket.connect(Socket.java:579)
>>>>         at
>>>> org.apache.giraph.zk.ZooKeeperManager.onlineZooKeeperServers(ZooKeeperManager.java:711)
>>>>         at
>>>> org.apache.giraph.graph.GraphTaskManager.startZooKeeperManager(GraphTaskManager.java:369)
>>>>         at
>>>> org.apache.giraph.graph.GraphTaskManager.setup(GraphTaskManager.java:202)
>>>>         at org.apache.giraph.graph.GraphMapper.setup(GraphMapper.java:59)
>>>>         at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:89)
>>>>         at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:726)
>>>>         at org.apache.hadoop.mapred.MapTask.run(MapTask.java:333)
>>>>         at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:158)
>>>>         at java.security.AccessController.doPrivileged(Native Method)
>>>>         at javax.security.auth.Subject.doAs(Subject.java:415)
>>>>         at
>>>> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1262)
>>>>         at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:153)
>>>> 2013-11-20 15:22:40,054 INFO [main] org.apache.giraph.zk.ZooKeeperManager:
>>>> onlineZooKeeperServers: Connect attempt 5 of 10 max trying to connect togsta60065.tan.ygrid.yahoo.com<http://tan.ygrid.yahoo.com/>:22181
with poll
>>>> msecs = 3000
>>>> 2013-11-20 15:22:40,054 WARN [main] org.apache.giraph.zk.ZooKeeperManager:
>>>> onlineZooKeeperServers: Got ConnectException
>>>> java.net.ConnectException: Connection refused
>>>>         at java.net.PlainSocketImpl.socketConnect(Native Method)
>>>>         at
>>>> java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
>>>>         at
>>>> java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
>>>>         at
>>>> java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
>>>>         at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391)
>>>>         at java.net.Socket.connect(Socket.java:579)
>>>>         at
>>>> org.apache.giraph.zk.ZooKeeperManager.onlineZooKeeperServers(ZooKeeperManager.java:711)
>>>>         at
>>>> org.apache.giraph.graph.GraphTaskManager.startZooKeeperManager(GraphTaskManager.java:369)
>>>>         at
>>>> org.apache.giraph.graph.GraphTaskManager.setup(GraphTaskManager.java:202)
>>>>         at org.apache.giraph.graph.GraphMapper.setup(GraphMapper.java:59)
>>>>         at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:89)
>>>>         at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:726)
>>>>         at org.apache.hadoop.mapred.MapTask.run(MapTask.java:333)
>>>>         at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:158)
>>>>         at java.security.AccessController.doPrivileged(Native Method)
>>>>         at javax.security.auth.Subject.doAs(Subject.java:415)
>>>>         at
>>>> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1262)
>>>>         at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:153)
>>>> 2013-11-20 15:22:43,055 INFO [main] org.apache.giraph.zk.ZooKeeperManager:
>>>> onlineZooKeeperServers: Connect attempt 6 of 10 max trying to connect togsta60065.tan.ygrid.yahoo.com<http://tan.ygrid.yahoo.com/>:22181
with poll
>>>> msecs = 3000
>>>> 2013-11-20 15:22:43,055 WARN [main] org.apache.giraph.zk.ZooKeeperManager:
>>>> onlineZooKeeperServers: Got ConnectException
>>>> java.net.ConnectException: Connection refused
>>>>         at java.net.PlainSocketImpl.socketConnect(Native Method)
>>>>         at
>>>> java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
>>>>         at
>>>> java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
>>>>         at
>>>> java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
>>>>         at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391)
>>>>         at java.net.Socket.connect(Socket.java:579)
>>>>         at
>>>> org.apache.giraph.zk.ZooKeeperManager.onlineZooKeeperServers(ZooKeeperManager.java:711)
>>>>         at
>>>> org.apache.giraph.graph.GraphTaskManager.startZooKeeperManager(GraphTaskManager.java:369)
>>>>         at
>>>> org.apache.giraph.graph.GraphTaskManager.setup(GraphTaskManager.java:202)
>>>>         at org.apache.giraph.graph.GraphMapper.setup(GraphMapper.java:59)
>>>>         at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:89)
>>>>         at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:726)
>>>>         at org.apache.hadoop.mapred.MapTask.run(MapTask.java:333)
>>>>         at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:158)
>>>>         at java.security.AccessController.doPrivileged(Native Method)
>>>>         at javax.security.auth.Subject.doAs(Subject.java:415)
>>>>         at
>>>> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1262)
>>>>         at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:153)
>>>> 2013-11-20 15:22:46,055 INFO [main] org.apache.giraph.zk.ZooKeeperManager:
>>>> onlineZooKeeperServers: Connect attempt 7 of 10 max trying to connect togsta60065.tan.ygrid.yahoo.com<http://tan.ygrid.yahoo.com/>:22181
with poll
>>>> msecs = 3000
>>>> 2013-11-20 15:22:46,056 WARN [main] org.apache.giraph.zk.ZooKeeperManager:
>>>> onlineZooKeeperServers: Got ConnectException
>>>> java.net.ConnectException: Connection refused
>>>>         at java.net.PlainSocketImpl.socketConnect(Native Method)
>>>>         at
>>>> java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
>>>>         at
>>>> java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
>>>>         at
>>>> java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
>>>>         at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391)
>>>>         at java.net.Socket.connect(Socket.java:579)
>>>>         at
>>>> org.apache.giraph.zk.ZooKeeperManager.onlineZooKeeperServers(ZooKeeperManager.java:711)
>>>>         at
>>>> org.apache.giraph.graph.GraphTaskManager.startZooKeeperManager(GraphTaskManager.java:369)
>>>>         at
>>>> org.apache.giraph.graph.GraphTaskManager.setup(GraphTaskManager.java:202)
>>>>         at org.apache.giraph.graph.GraphMapper.setup(GraphMapper.java:59)
>>>>         at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:89)
>>>>         at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:726)
>>>>         at org.apache.hadoop.mapred.MapTask.run(MapTask.java:333)
>>>>         at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:158)
>>>>         at java.security.AccessController.doPrivileged(Native Method)
>>>>         at javax.security.auth.Subject.doAs(Subject.java:415)
>>>>         at
>>>> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1262)
>>>>         at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:153)
>>>> 2013-11-20 15:22:49,056 INFO [main] org.apache.giraph.zk.ZooKeeperManager:
>>>> onlineZooKeeperServers: Connect attempt 8 of 10 max trying to connect togsta60065.tan.ygrid.yahoo.com<http://tan.ygrid.yahoo.com/>:22181
with poll
>>>> msecs = 3000
>>>> 2013-11-20 15:22:49,057 WARN [main] org.apache.giraph.zk.ZooKeeperManager:
>>>> onlineZooKeeperServers: Got ConnectException
>>>> java.net.ConnectException: Connection refused
>>>>         at java.net.PlainSocketImpl.socketConnect(Native Method)
>>>>         at
>>>> java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
>>>>         at
>>>> java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
>>>>         at
>>>> java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
>>>>         at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391)
>>>>         at java.net.Socket.connect(Socket.java:579)
>>>>         at
>>>> org.apache.giraph.zk.ZooKeeperManager.onlineZooKeeperServers(ZooKeeperManager.java:711)
>>>>         at
>>>> org.apache.giraph.graph.GraphTaskManager.startZooKeeperManager(GraphTaskManager.java:369)
>>>>         at
>>>> org.apache.giraph.graph.GraphTaskManager.setup(GraphTaskManager.java:202)
>>>>         at org.apache.giraph.graph.GraphMapper.setup(GraphMapper.java:59)
>>>>         at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:89)
>>>>         at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:726)
>>>>         at org.apache.hadoop.mapred.MapTask.run(MapTask.java:333)
>>>>         at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:158)
>>>>         at java.security.AccessController.doPrivileged(Native Method)
>>>>         at javax.security.auth.Subject.doAs(Subject.java:415)
>>>>         at
>>>> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1262)
>>>>         at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:153)
>>>> 2013-11-20 15:22:52,057 INFO [main] org.apache.giraph.zk.ZooKeeperManager:
>>>> onlineZooKeeperServers: Connect attempt 9 of 10 max trying to connect togsta60065.tan.ygrid.yahoo.com<http://tan.ygrid.yahoo.com/>:22181
with poll
>>>> msecs = 3000
>>>> 2013-11-20 15:22:52,058 WARN [main] org.apache.giraph.zk.ZooKeeperManager:
>>>> onlineZooKeeperServers: Got ConnectException
>>>> java.net.ConnectException: Connection refused
>>>>         at java.net.PlainSocketImpl.socketConnect(Native Method)
>>>>         at
>>>> java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
>>>>         at
>>>> java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
>>>>         at
>>>> java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
>>>>         at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391)
>>>>         at java.net.Socket.connect(Socket.java:579)
>>>>         at
>>>> org.apache.giraph.zk.ZooKeeperManager.onlineZooKeeperServers(ZooKeeperManager.java:711)
>>>>         at
>>>> org.apache.giraph.graph.GraphTaskManager.startZooKeeperManager(GraphTaskManager.java:369)
>>>>         at
>>>> org.apache.giraph.graph.GraphTaskManager.setup(GraphTaskManager.java:202)
>>>>         at org.apache.giraph.graph.GraphMapper.setup(GraphMapper.java:59)
>>>>         at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:89)
>>>>         at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:726)
>>>>         at org.apache.hadoop.mapred.MapTask.run(MapTask.java:333)
>>>>         at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:158)
>>>>         at java.security.AccessController.doPrivileged(Native Method)
>>>>         at javax.security.auth.Subject.doAs(Subject.java:415)
>>>>         at
>>>> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1262)
>>>>         at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:153)
>>>> 2013-11-20 15:22:55,060 WARN [main] org.apache.hadoop.mapred.YarnChild:
>>>> Exception running child : java.lang.IllegalStateException: run: Caught an
>>>> unrecoverable exception onlineZooKeeperServers: Failed to connect in 10 tries!
>>>>         at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:101)
>>>>         at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:726)
>>>>         at org.apache.hadoop.mapred.MapTask.run(MapTask.java:333)
>>>>         at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:158)
>>>>         at java.security.AccessController.doPrivileged(Native Method)
>>>>         at javax.security.auth.Subject.doAs(Subject.java:415)
>>>>         at
>>>> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1262)
>>>>         at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:153)
>>>> Caused by: java.lang.IllegalStateException: onlineZooKeeperServers: Failed
to
>>>> connect in 10 tries!
>>>>         at
>>>> org.apache.giraph.zk.ZooKeeperManager.onlineZooKeeperServers(ZooKeeperManager.java:737)
>>>>         at
>>>> org.apache.giraph.graph.GraphTaskManager.startZooKeeperManager(GraphTaskManager.java:369)
>>>>         at
>>>> org.apache.giraph.graph.GraphTaskManager.setup(GraphTaskManager.java:202)
>>>>         at org.apache.giraph.graph.GraphMapper.setup(GraphMapper.java:59)
>>>>         at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:89)
>>>>         ... 7 more
>>>>
>>>> 2013-11-20 15:22:55,063 INFO [main] org.apache.hadoop.mapred.Task: Runnning
>>>> cleanup for the task
>>>> 2013-11-20 15:22:55,110 WARN [main] org.apache.hadoop.conf.Configuration:
>>>> job.xml:an attempt to override final parameter: dfs.namenode.name.dir;
>>>> Ignoring.
>>>> 2013-11-20 15:22:55,110 WARN [main] org.apache.hadoop.conf.Configuration:
>>>> job.xml:an attempt to override final parameter:
>>>> hadoop.security.token.service.use_ip;  Ignoring.
>>>> 2013-11-20 15:22:55,110 WARN [main] org.apache.hadoop.conf.Configuration:
>>>> job.xml:an attempt to override final parameter:
>>>> mapreduce.job.end-notification.max.retry.interval;  Ignoring.
>>>> 2013-11-20 15:22:55,110 WARN [main] org.apache.hadoop.conf.Configuration:
>>>> job.xml:an attempt to override final parameter:
>>>> mapreduce.admin.map.child.java.opts;  Ignoring.
>>>> 2013-11-20 15:22:55,111 WARN [main] org.apache.hadoop.conf.Configuration:
>>>> job.xml:an attempt to override final parameter:
>>>> mapreduce.job.end-notification.max.attempts;  Ignoring.
>>>> 2013-11-20 15:22:55,111 WARN [main] org.apache.hadoop.conf.Configuration:
>>>> job.xml:an attempt to override final parameter:
>>>> yarn.app.mapreduce.am.job.client.port-range;  Ignoring.
>>>> 2013-11-20 15:22:55,111 WARN [main] org.apache.hadoop.conf.Configuration:
>>>> job.xml:an attempt to override final parameter:
>>>> mapreduce.admin.reduce.child.java.opts;  Ignoring.
>>>> 2013-11-20 15:22:55,111 WARN [main] org.apache.hadoop.conf.Configuration:
>>>> job.xml:an attempt to override final parameter: hadoop.tmp.dir;  Ignoring.
>>>> 2013-11-20 15:22:55,117 WARN [main]
>>>> org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter: Could not delete
>>>> hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/bordino/test-giraph-tmp/_temporary/1/_temporary/attempt_1382563758657_470916_m_000056_0
>>>> 2013-11-20 <http://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/bordino/test-giraph-tmp/_temporary/1/_temporary/attempt_1382563758657_470916_m_000056_02013-11-20>
15:22:55,121 INFO [main]
>>>> org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Stopping MapTask metrics
>>>> system...
>>>> 2013-11-20 15:22:55,122 INFO [main]
>>>> org.apache.hadoop.metrics2.impl.MetricsSystemImpl: MapTask metrics system
>>>> stopped.
>>>> 2013-11-20 15:22:55,122 INFO [main]
>>>> org.apache.hadoop.metrics2.impl.MetricsSystemImpl: MapTask metrics system
>>>> shutdown complete.
>>>>
>>>>
>>>>
>>>> Cheres,
>>>> --
>>>> Gianmarco
>>>>
>>>
>>>
>>>
>>> --
>>>    Claudio Martella
>>>    claudio.martella@gmail.com
>>>
>>
>>
>
>
> --
>    Claudio Martella
>    claudio.martella@gmail.com
>

Mime
View raw message