giraph-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Claudio Martella <claudio.marte...@gmail.com>
Subject Re: Running Giraph on YARN (0.23)
Date Thu, 21 Nov 2013 14:52:28 GMT
AFAIK YARN does/did not allow to freely spawn new processes in the way we
used to with M/R.
Probably Eli and others can give more details.


On Thu, Nov 21, 2013 at 2:45 PM, Gianmarco De Francisci Morales <
gdfm@apache.org> wrote:

> Indeed I am expecting Giraph to do it for me. Can I ask what is the reason
> for this difference in YARN?
> It makes deploying Giraph much more complicated.
>
> --
> Gianmarco
>
>
> On 21 November 2013 12:17, Claudio Martella <claudio.martella@gmail.com>wrote:
>
>> Are you running with an external ZK, or are you expecting giraph to start
>> one from you on the master? If the second one is the case (as it seams to
>> me from the logs), then it won't work. Giraph on Yarn requires you to start
>> ZK externally, and pass the address to giraph when you start the job.
>>
>>
>> On Thu, Nov 21, 2013 at 11:49 AM, Gianmarco De Francisci Morales <
>> gdfm@apache.org> wrote:
>>
>>> Hi,
>>>
>>> I am trying to run Giraph on a Hadoop 0.23 installation, but I am
>>> running in a communication problem. From the log it seems that the workers
>>> are not able to connect to the zookeeper instance on the master.
>>> I am using the option -DuseNetty=true
>>>
>>> Any help is appreciated.
>>> Here the error log:
>>>
>>>
>>>
>>>
>>> ************
>>>
>>>
>>> 2013-11-20 15:22:22,000 INFO [main] org.apache.giraph.graph.GraphTaskManager:
>>> setup: classpath @ /user/bordino/.staging/job_1382563758657_470916/job.jar for
>>> job Giraph: org.apache.giraph.examples.RandomWalkWithRestartComputation
>>> 2013-11-20 15:22:22,003 WARN [main] org.apache.hadoop.conf.Configuration:
>>> mapred.task.partition is deprecated. Instead, use mapreduce.task.partition
>>> 2013-11-20 15:22:22,003 WARN [main] org.apache.hadoop.conf.Configuration:mapred.job.id
is deprecated. Instead, use mapreduce.job.id
>>> 2013-11-20 15:22:22,004 WARN [main] org.apache.hadoop.conf.Configuration:
>>> job.local.dir is deprecated. Instead, use mapreduce.job.local.dir
>>> 2013-11-20 15:22:22,007 INFO [main] org.apache.giraph.zk.ZooKeeperManager:
>>> createCandidateStamp: Made the directory
>>> _bsp/_defaultZkManagerDir/job_1382563758657_470916
>>> 2013-11-20 15:22:22,008 INFO [main] org.apache.giraph.zk.ZooKeeperManager:
>>> createCandidateStamp: Made the directory
>>> _bsp/_defaultZkManagerDir/job_1382563758657_470916/_zkServer
>>> 2013-11-20 15:22:22,011 INFO [main] org.apache.giraph.zk.ZooKeeperManager:
>>> createCandidateStamp: Creating my filestamp
>>> _bsp/_defaultZkManagerDir/job_1382563758657_470916/_task/gsta60065.tan.ygrid.yahoo.com<http://tan.ygrid.yahoo.com/>
>>> 56
>>> 2013-11-20 15:22:22,027 INFO [main] org.apache.giraph.zk.ZooKeeperManager:
>>> getZooKeeperServerList: For task 56, got file 'null' (polling period is 3000)
>>> 2013-11-20 15:22:25,030 INFO [main] org.apache.giraph.zk.ZooKeeperManager:
>>> getZooKeeperServerList: For task 56, got file
>>> 'zkServerList_gsta60065.tan.ygrid.yahoo.com<http://tan.ygrid.yahoo.com/>
56 '
>>> (polling period is 3000)
>>> 2013-11-20 15:22:25,030 INFO [main] org.apache.giraph.zk.ZooKeeperManager:
>>> getZooKeeperServerList: Found
>>> [gsta60065.tan.ygrid.yahoo.com<http://tan.ygrid.yahoo.com/>, 56] 2 hosts
in
>>> filename
>>> 'zkServerList_gsta60065.tan.ygrid.yahoo.com<http://tan.ygrid.yahoo.com/>
56 '
>>> 2013-11-20 15:22:25,031 INFO [main] org.apache.giraph.zk.ZooKeeperManager:
>>> onlineZooKeeperServers: Trying to delete old directory
>>> /grid/0/tmp/yarn-local/usercache/bordino/appcache/application_1382563758657_470916/work/_bspZooKeeper
>>> 2013-11-20 15:22:25,038 INFO [main] org.apache.giraph.zk.ZooKeeperManager:
>>> generateZooKeeperConfigFile: Creating file
>>> /grid/0/tmp/yarn-local/usercache/bordino/appcache/application_1382563758657_470916/work/_bspZooKeeper/zoo.cfg
>>> in
>>> /grid/0/tmp/yarn-local/usercache/bordino/appcache/application_1382563758657_470916/work/_bspZooKeeper
>>> with base port 22181
>>> 2013-11-20 15:22:25,038 INFO [main] org.apache.giraph.zk.ZooKeeperManager:
>>> generateZooKeeperConfigFile: Make directory of _bspZooKeeper = true
>>> 2013-11-20 15:22:25,039 INFO [main] org.apache.giraph.zk.ZooKeeperManager:
>>> generateZooKeeperConfigFile: Delete of zoo.cfg = false
>>> 2013-11-20 15:22:25,041 INFO [main] org.apache.giraph.zk.ZooKeeperManager:
>>> onlineZooKeeperServers: Attempting to start ZooKeeper server with command
>>> [/home/Releases/gridjdk-1.7.0_17.1303042057-20130415-000/share/gridjdk-1.7.0_17/jre/bin/java,
>>> -Xmx128M, -XX:ParallelGCThreads=4, -XX:+UseConcMarkSweepGC,
>>> -XX:CMSInitiatingOccupancyFraction=70, -XX:MaxGCPauseMillis=100, -cp,
>>> /user/bordino/.staging/job_1382563758657_470916/job.jar,
>>> org.apache.zookeeper.server.quorum.QuorumPeerMain,
>>> /grid/0/tmp/yarn-local/usercache/bordino/appcache/application_1382563758657_470916/work/_bspZooKeeper/zoo.cfg]
>>> in directory
>>> /grid/0/tmp/yarn-local/usercache/bordino/appcache/application_1382563758657_470916/work/_bspZooKeeper
>>> 2013-11-20 15:22:25,044 INFO [main] org.apache.giraph.zk.ZooKeeperManager:
>>> onlineZooKeeperServers: Shutdown hook added.
>>> 2013-11-20 15:22:25,044 INFO [main] org.apache.giraph.zk.ZooKeeperManager:
>>> onlineZooKeeperServers: Connect attempt 0 of 10 max trying to connect togsta60065.tan.ygrid.yahoo.com<http://tan.ygrid.yahoo.com/>:22181
with poll
>>> msecs = 3000
>>> 2013-11-20 15:22:25,048 WARN [main] org.apache.giraph.zk.ZooKeeperManager:
>>> onlineZooKeeperServers: Got ConnectException
>>> java.net.ConnectException: Connection refused
>>>         at java.net.PlainSocketImpl.socketConnect(Native Method)
>>>         at
>>> java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
>>>         at
>>> java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
>>>         at
>>> java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
>>>         at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391)
>>>         at java.net.Socket.connect(Socket.java:579)
>>>         at
>>> org.apache.giraph.zk.ZooKeeperManager.onlineZooKeeperServers(ZooKeeperManager.java:711)
>>>         at
>>> org.apache.giraph.graph.GraphTaskManager.startZooKeeperManager(GraphTaskManager.java:369)
>>>         at
>>> org.apache.giraph.graph.GraphTaskManager.setup(GraphTaskManager.java:202)
>>>         at org.apache.giraph.graph.GraphMapper.setup(GraphMapper.java:59)
>>>         at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:89)
>>>         at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:726)
>>>         at org.apache.hadoop.mapred.MapTask.run(MapTask.java:333)
>>>         at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:158)
>>>         at java.security.AccessController.doPrivileged(Native Method)
>>>         at javax.security.auth.Subject.doAs(Subject.java:415)
>>>         at
>>> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1262)
>>>         at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:153)
>>> 2013-11-20 15:22:28,050 INFO [main] org.apache.giraph.zk.ZooKeeperManager:
>>> onlineZooKeeperServers: Connect attempt 1 of 10 max trying to connect togsta60065.tan.ygrid.yahoo.com<http://tan.ygrid.yahoo.com/>:22181
with poll
>>> msecs = 3000
>>> 2013-11-20 15:22:28,050 WARN [main] org.apache.giraph.zk.ZooKeeperManager:
>>> onlineZooKeeperServers: Got ConnectException
>>> java.net.ConnectException: Connection refused
>>>         at java.net.PlainSocketImpl.socketConnect(Native Method)
>>>         at
>>> java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
>>>         at
>>> java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
>>>         at
>>> java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
>>>         at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391)
>>>         at java.net.Socket.connect(Socket.java:579)
>>>         at
>>> org.apache.giraph.zk.ZooKeeperManager.onlineZooKeeperServers(ZooKeeperManager.java:711)
>>>         at
>>> org.apache.giraph.graph.GraphTaskManager.startZooKeeperManager(GraphTaskManager.java:369)
>>>         at
>>> org.apache.giraph.graph.GraphTaskManager.setup(GraphTaskManager.java:202)
>>>         at org.apache.giraph.graph.GraphMapper.setup(GraphMapper.java:59)
>>>         at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:89)
>>>         at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:726)
>>>         at org.apache.hadoop.mapred.MapTask.run(MapTask.java:333)
>>>         at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:158)
>>>         at java.security.AccessController.doPrivileged(Native Method)
>>>         at javax.security.auth.Subject.doAs(Subject.java:415)
>>>         at
>>> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1262)
>>>         at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:153)
>>> 2013-11-20 15:22:31,051 INFO [main] org.apache.giraph.zk.ZooKeeperManager:
>>> onlineZooKeeperServers: Connect attempt 2 of 10 max trying to connect togsta60065.tan.ygrid.yahoo.com<http://tan.ygrid.yahoo.com/>:22181
with poll
>>> msecs = 3000
>>> 2013-11-20 15:22:31,051 WARN [main] org.apache.giraph.zk.ZooKeeperManager:
>>> onlineZooKeeperServers: Got ConnectException
>>> java.net.ConnectException: Connection refused
>>>         at java.net.PlainSocketImpl.socketConnect(Native Method)
>>>         at
>>> java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
>>>         at
>>> java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
>>>         at
>>> java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
>>>         at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391)
>>>         at java.net.Socket.connect(Socket.java:579)
>>>         at
>>> org.apache.giraph.zk.ZooKeeperManager.onlineZooKeeperServers(ZooKeeperManager.java:711)
>>>         at
>>> org.apache.giraph.graph.GraphTaskManager.startZooKeeperManager(GraphTaskManager.java:369)
>>>         at
>>> org.apache.giraph.graph.GraphTaskManager.setup(GraphTaskManager.java:202)
>>>         at org.apache.giraph.graph.GraphMapper.setup(GraphMapper.java:59)
>>>         at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:89)
>>>         at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:726)
>>>         at org.apache.hadoop.mapred.MapTask.run(MapTask.java:333)
>>>         at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:158)
>>>         at java.security.AccessController.doPrivileged(Native Method)
>>>         at javax.security.auth.Subject.doAs(Subject.java:415)
>>>         at
>>> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1262)
>>>         at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:153)
>>> 2013-11-20 15:22:34,052 INFO [main] org.apache.giraph.zk.ZooKeeperManager:
>>> onlineZooKeeperServers: Connect attempt 3 of 10 max trying to connect togsta60065.tan.ygrid.yahoo.com<http://tan.ygrid.yahoo.com/>:22181
with poll
>>> msecs = 3000
>>> 2013-11-20 15:22:34,052 WARN [main] org.apache.giraph.zk.ZooKeeperManager:
>>> onlineZooKeeperServers: Got ConnectException
>>> java.net.ConnectException: Connection refused
>>>         at java.net.PlainSocketImpl.socketConnect(Native Method)
>>>         at
>>> java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
>>>         at
>>> java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
>>>         at
>>> java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
>>>         at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391)
>>>         at java.net.Socket.connect(Socket.java:579)
>>>         at
>>> org.apache.giraph.zk.ZooKeeperManager.onlineZooKeeperServers(ZooKeeperManager.java:711)
>>>         at
>>> org.apache.giraph.graph.GraphTaskManager.startZooKeeperManager(GraphTaskManager.java:369)
>>>         at
>>> org.apache.giraph.graph.GraphTaskManager.setup(GraphTaskManager.java:202)
>>>         at org.apache.giraph.graph.GraphMapper.setup(GraphMapper.java:59)
>>>         at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:89)
>>>         at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:726)
>>>         at org.apache.hadoop.mapred.MapTask.run(MapTask.java:333)
>>>         at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:158)
>>>         at java.security.AccessController.doPrivileged(Native Method)
>>>         at javax.security.auth.Subject.doAs(Subject.java:415)
>>>         at
>>> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1262)
>>>         at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:153)
>>> 2013-11-20 15:22:37,053 INFO [main] org.apache.giraph.zk.ZooKeeperManager:
>>> onlineZooKeeperServers: Connect attempt 4 of 10 max trying to connect togsta60065.tan.ygrid.yahoo.com<http://tan.ygrid.yahoo.com/>:22181
with poll
>>> msecs = 3000
>>> 2013-11-20 15:22:37,053 WARN [main] org.apache.giraph.zk.ZooKeeperManager:
>>> onlineZooKeeperServers: Got ConnectException
>>> java.net.ConnectException: Connection refused
>>>         at java.net.PlainSocketImpl.socketConnect(Native Method)
>>>         at
>>> java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
>>>         at
>>> java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
>>>         at
>>> java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
>>>         at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391)
>>>         at java.net.Socket.connect(Socket.java:579)
>>>         at
>>> org.apache.giraph.zk.ZooKeeperManager.onlineZooKeeperServers(ZooKeeperManager.java:711)
>>>         at
>>> org.apache.giraph.graph.GraphTaskManager.startZooKeeperManager(GraphTaskManager.java:369)
>>>         at
>>> org.apache.giraph.graph.GraphTaskManager.setup(GraphTaskManager.java:202)
>>>         at org.apache.giraph.graph.GraphMapper.setup(GraphMapper.java:59)
>>>         at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:89)
>>>         at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:726)
>>>         at org.apache.hadoop.mapred.MapTask.run(MapTask.java:333)
>>>         at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:158)
>>>         at java.security.AccessController.doPrivileged(Native Method)
>>>         at javax.security.auth.Subject.doAs(Subject.java:415)
>>>         at
>>> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1262)
>>>         at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:153)
>>> 2013-11-20 15:22:40,054 INFO [main] org.apache.giraph.zk.ZooKeeperManager:
>>> onlineZooKeeperServers: Connect attempt 5 of 10 max trying to connect togsta60065.tan.ygrid.yahoo.com<http://tan.ygrid.yahoo.com/>:22181
with poll
>>> msecs = 3000
>>> 2013-11-20 15:22:40,054 WARN [main] org.apache.giraph.zk.ZooKeeperManager:
>>> onlineZooKeeperServers: Got ConnectException
>>> java.net.ConnectException: Connection refused
>>>         at java.net.PlainSocketImpl.socketConnect(Native Method)
>>>         at
>>> java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
>>>         at
>>> java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
>>>         at
>>> java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
>>>         at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391)
>>>         at java.net.Socket.connect(Socket.java:579)
>>>         at
>>> org.apache.giraph.zk.ZooKeeperManager.onlineZooKeeperServers(ZooKeeperManager.java:711)
>>>         at
>>> org.apache.giraph.graph.GraphTaskManager.startZooKeeperManager(GraphTaskManager.java:369)
>>>         at
>>> org.apache.giraph.graph.GraphTaskManager.setup(GraphTaskManager.java:202)
>>>         at org.apache.giraph.graph.GraphMapper.setup(GraphMapper.java:59)
>>>         at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:89)
>>>         at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:726)
>>>         at org.apache.hadoop.mapred.MapTask.run(MapTask.java:333)
>>>         at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:158)
>>>         at java.security.AccessController.doPrivileged(Native Method)
>>>         at javax.security.auth.Subject.doAs(Subject.java:415)
>>>         at
>>> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1262)
>>>         at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:153)
>>> 2013-11-20 15:22:43,055 INFO [main] org.apache.giraph.zk.ZooKeeperManager:
>>> onlineZooKeeperServers: Connect attempt 6 of 10 max trying to connect togsta60065.tan.ygrid.yahoo.com<http://tan.ygrid.yahoo.com/>:22181
with poll
>>> msecs = 3000
>>> 2013-11-20 15:22:43,055 WARN [main] org.apache.giraph.zk.ZooKeeperManager:
>>> onlineZooKeeperServers: Got ConnectException
>>> java.net.ConnectException: Connection refused
>>>         at java.net.PlainSocketImpl.socketConnect(Native Method)
>>>         at
>>> java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
>>>         at
>>> java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
>>>         at
>>> java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
>>>         at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391)
>>>         at java.net.Socket.connect(Socket.java:579)
>>>         at
>>> org.apache.giraph.zk.ZooKeeperManager.onlineZooKeeperServers(ZooKeeperManager.java:711)
>>>         at
>>> org.apache.giraph.graph.GraphTaskManager.startZooKeeperManager(GraphTaskManager.java:369)
>>>         at
>>> org.apache.giraph.graph.GraphTaskManager.setup(GraphTaskManager.java:202)
>>>         at org.apache.giraph.graph.GraphMapper.setup(GraphMapper.java:59)
>>>         at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:89)
>>>         at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:726)
>>>         at org.apache.hadoop.mapred.MapTask.run(MapTask.java:333)
>>>         at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:158)
>>>         at java.security.AccessController.doPrivileged(Native Method)
>>>         at javax.security.auth.Subject.doAs(Subject.java:415)
>>>         at
>>> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1262)
>>>         at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:153)
>>> 2013-11-20 15:22:46,055 INFO [main] org.apache.giraph.zk.ZooKeeperManager:
>>> onlineZooKeeperServers: Connect attempt 7 of 10 max trying to connect togsta60065.tan.ygrid.yahoo.com<http://tan.ygrid.yahoo.com/>:22181
with poll
>>> msecs = 3000
>>> 2013-11-20 15:22:46,056 WARN [main] org.apache.giraph.zk.ZooKeeperManager:
>>> onlineZooKeeperServers: Got ConnectException
>>> java.net.ConnectException: Connection refused
>>>         at java.net.PlainSocketImpl.socketConnect(Native Method)
>>>         at
>>> java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
>>>         at
>>> java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
>>>         at
>>> java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
>>>         at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391)
>>>         at java.net.Socket.connect(Socket.java:579)
>>>         at
>>> org.apache.giraph.zk.ZooKeeperManager.onlineZooKeeperServers(ZooKeeperManager.java:711)
>>>         at
>>> org.apache.giraph.graph.GraphTaskManager.startZooKeeperManager(GraphTaskManager.java:369)
>>>         at
>>> org.apache.giraph.graph.GraphTaskManager.setup(GraphTaskManager.java:202)
>>>         at org.apache.giraph.graph.GraphMapper.setup(GraphMapper.java:59)
>>>         at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:89)
>>>         at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:726)
>>>         at org.apache.hadoop.mapred.MapTask.run(MapTask.java:333)
>>>         at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:158)
>>>         at java.security.AccessController.doPrivileged(Native Method)
>>>         at javax.security.auth.Subject.doAs(Subject.java:415)
>>>         at
>>> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1262)
>>>         at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:153)
>>> 2013-11-20 15:22:49,056 INFO [main] org.apache.giraph.zk.ZooKeeperManager:
>>> onlineZooKeeperServers: Connect attempt 8 of 10 max trying to connect togsta60065.tan.ygrid.yahoo.com<http://tan.ygrid.yahoo.com/>:22181
with poll
>>> msecs = 3000
>>> 2013-11-20 15:22:49,057 WARN [main] org.apache.giraph.zk.ZooKeeperManager:
>>> onlineZooKeeperServers: Got ConnectException
>>> java.net.ConnectException: Connection refused
>>>         at java.net.PlainSocketImpl.socketConnect(Native Method)
>>>         at
>>> java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
>>>         at
>>> java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
>>>         at
>>> java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
>>>         at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391)
>>>         at java.net.Socket.connect(Socket.java:579)
>>>         at
>>> org.apache.giraph.zk.ZooKeeperManager.onlineZooKeeperServers(ZooKeeperManager.java:711)
>>>         at
>>> org.apache.giraph.graph.GraphTaskManager.startZooKeeperManager(GraphTaskManager.java:369)
>>>         at
>>> org.apache.giraph.graph.GraphTaskManager.setup(GraphTaskManager.java:202)
>>>         at org.apache.giraph.graph.GraphMapper.setup(GraphMapper.java:59)
>>>         at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:89)
>>>         at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:726)
>>>         at org.apache.hadoop.mapred.MapTask.run(MapTask.java:333)
>>>         at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:158)
>>>         at java.security.AccessController.doPrivileged(Native Method)
>>>         at javax.security.auth.Subject.doAs(Subject.java:415)
>>>         at
>>> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1262)
>>>         at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:153)
>>> 2013-11-20 15:22:52,057 INFO [main] org.apache.giraph.zk.ZooKeeperManager:
>>> onlineZooKeeperServers: Connect attempt 9 of 10 max trying to connect togsta60065.tan.ygrid.yahoo.com<http://tan.ygrid.yahoo.com/>:22181
with poll
>>> msecs = 3000
>>> 2013-11-20 15:22:52,058 WARN [main] org.apache.giraph.zk.ZooKeeperManager:
>>> onlineZooKeeperServers: Got ConnectException
>>> java.net.ConnectException: Connection refused
>>>         at java.net.PlainSocketImpl.socketConnect(Native Method)
>>>         at
>>> java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
>>>         at
>>> java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
>>>         at
>>> java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
>>>         at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391)
>>>         at java.net.Socket.connect(Socket.java:579)
>>>         at
>>> org.apache.giraph.zk.ZooKeeperManager.onlineZooKeeperServers(ZooKeeperManager.java:711)
>>>         at
>>> org.apache.giraph.graph.GraphTaskManager.startZooKeeperManager(GraphTaskManager.java:369)
>>>         at
>>> org.apache.giraph.graph.GraphTaskManager.setup(GraphTaskManager.java:202)
>>>         at org.apache.giraph.graph.GraphMapper.setup(GraphMapper.java:59)
>>>         at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:89)
>>>         at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:726)
>>>         at org.apache.hadoop.mapred.MapTask.run(MapTask.java:333)
>>>         at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:158)
>>>         at java.security.AccessController.doPrivileged(Native Method)
>>>         at javax.security.auth.Subject.doAs(Subject.java:415)
>>>         at
>>> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1262)
>>>         at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:153)
>>> 2013-11-20 15:22:55,060 WARN [main] org.apache.hadoop.mapred.YarnChild:
>>> Exception running child : java.lang.IllegalStateException: run: Caught an
>>> unrecoverable exception onlineZooKeeperServers: Failed to connect in 10 tries!
>>>         at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:101)
>>>         at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:726)
>>>         at org.apache.hadoop.mapred.MapTask.run(MapTask.java:333)
>>>         at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:158)
>>>         at java.security.AccessController.doPrivileged(Native Method)
>>>         at javax.security.auth.Subject.doAs(Subject.java:415)
>>>         at
>>> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1262)
>>>         at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:153)
>>> Caused by: java.lang.IllegalStateException: onlineZooKeeperServers: Failed to
>>> connect in 10 tries!
>>>         at
>>> org.apache.giraph.zk.ZooKeeperManager.onlineZooKeeperServers(ZooKeeperManager.java:737)
>>>         at
>>> org.apache.giraph.graph.GraphTaskManager.startZooKeeperManager(GraphTaskManager.java:369)
>>>         at
>>> org.apache.giraph.graph.GraphTaskManager.setup(GraphTaskManager.java:202)
>>>         at org.apache.giraph.graph.GraphMapper.setup(GraphMapper.java:59)
>>>         at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:89)
>>>         ... 7 more
>>>
>>> 2013-11-20 15:22:55,063 INFO [main] org.apache.hadoop.mapred.Task: Runnning
>>> cleanup for the task
>>> 2013-11-20 15:22:55,110 WARN [main] org.apache.hadoop.conf.Configuration:
>>> job.xml:an attempt to override final parameter: dfs.namenode.name.dir;
>>> Ignoring.
>>> 2013-11-20 15:22:55,110 WARN [main] org.apache.hadoop.conf.Configuration:
>>> job.xml:an attempt to override final parameter:
>>> hadoop.security.token.service.use_ip;  Ignoring.
>>> 2013-11-20 15:22:55,110 WARN [main] org.apache.hadoop.conf.Configuration:
>>> job.xml:an attempt to override final parameter:
>>> mapreduce.job.end-notification.max.retry.interval;  Ignoring.
>>> 2013-11-20 15:22:55,110 WARN [main] org.apache.hadoop.conf.Configuration:
>>> job.xml:an attempt to override final parameter:
>>> mapreduce.admin.map.child.java.opts;  Ignoring.
>>> 2013-11-20 15:22:55,111 WARN [main] org.apache.hadoop.conf.Configuration:
>>> job.xml:an attempt to override final parameter:
>>> mapreduce.job.end-notification.max.attempts;  Ignoring.
>>> 2013-11-20 15:22:55,111 WARN [main] org.apache.hadoop.conf.Configuration:
>>> job.xml:an attempt to override final parameter:
>>> yarn.app.mapreduce.am.job.client.port-range;  Ignoring.
>>> 2013-11-20 15:22:55,111 WARN [main] org.apache.hadoop.conf.Configuration:
>>> job.xml:an attempt to override final parameter:
>>> mapreduce.admin.reduce.child.java.opts;  Ignoring.
>>> 2013-11-20 15:22:55,111 WARN [main] org.apache.hadoop.conf.Configuration:
>>> job.xml:an attempt to override final parameter: hadoop.tmp.dir;  Ignoring.
>>> 2013-11-20 15:22:55,117 WARN [main]
>>> org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter: Could not delete
>>> hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/bordino/test-giraph-tmp/_temporary/1/_temporary/attempt_1382563758657_470916_m_000056_0
>>> 2013-11-20 <http://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/bordino/test-giraph-tmp/_temporary/1/_temporary/attempt_1382563758657_470916_m_000056_02013-11-20>
15:22:55,121 INFO [main]
>>> org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Stopping MapTask metrics
>>> system...
>>> 2013-11-20 15:22:55,122 INFO [main]
>>> org.apache.hadoop.metrics2.impl.MetricsSystemImpl: MapTask metrics system
>>> stopped.
>>> 2013-11-20 15:22:55,122 INFO [main]
>>> org.apache.hadoop.metrics2.impl.MetricsSystemImpl: MapTask metrics system
>>> shutdown complete.
>>>
>>>
>>>
>>> Cheres,
>>> --
>>> Gianmarco
>>>
>>
>>
>>
>> --
>>    Claudio Martella
>>    claudio.martella@gmail.com
>>
>
>


-- 
   Claudio Martella
   claudio.martella@gmail.com

Mime
View raw message