giraph-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Vishal Mishra <mishravis...@gmail.com>
Subject Re: Running Giraph on YARN (0.23)
Date Wed, 27 Nov 2013 19:39:14 GMT
Maybe this stackoverflow thread answers your question on why it doesn't
work yet.
http://stackoverflow.com/questions/20108196/zookeeper-configs-for-giraph-1-0-on-hadoop-2-2-0


On Wed, Nov 27, 2013 at 5:46 AM, Gianmarco De Francisci Morales <
gdfm@apache.org> wrote:

> Hi,
> Any update on this issue?
> Is there any way to bring ZK up automatically on YARN, or any reason why
> this is not possible?
>
> Cheers,
>
> --
> Gianmarco
>
>
> On 21 November 2013 15:52, Claudio Martella <claudio.martella@gmail.com>wrote:
>
>> AFAIK YARN does/did not allow to freely spawn new processes in the way we
>> used to with M/R.
>> Probably Eli and others can give more details.
>>
>>
>>  On Thu, Nov 21, 2013 at 2:45 PM, Gianmarco De Francisci Morales <
>> gdfm@apache.org> wrote:
>>
>>> Indeed I am expecting Giraph to do it for me. Can I ask what is the
>>> reason for this difference in YARN?
>>> It makes deploying Giraph much more complicated.
>>>
>>> --
>>> Gianmarco
>>>
>>>
>>> On 21 November 2013 12:17, Claudio Martella <claudio.martella@gmail.com>wrote:
>>>
>>>> Are you running with an external ZK, or are you expecting giraph to
>>>> start one from you on the master? If the second one is the case (as it
>>>> seams to me from the logs), then it won't work. Giraph on Yarn requires you
>>>> to start ZK externally, and pass the address to giraph when you start the
>>>> job.
>>>>
>>>>
>>>> On Thu, Nov 21, 2013 at 11:49 AM, Gianmarco De Francisci Morales <
>>>> gdfm@apache.org> wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> I am trying to run Giraph on a Hadoop 0.23 installation, but I am
>>>>> running in a communication problem. From the log it seems that the workers
>>>>> are not able to connect to the zookeeper instance on the master.
>>>>> I am using the option -DuseNetty=true
>>>>>
>>>>> Any help is appreciated.
>>>>> Here the error log:
>>>>>
>>>>>
>>>>> ************
>>>>>
>>>>>
>>>>> 2013-11-20 15:22:22,000 INFO [main] org.apache.giraph.graph.GraphTaskManager:
>>>>> setup: classpath @ /user/bordino/.staging/job_1382563758657_470916/job.jar
for
>>>>> job Giraph: org.apache.giraph.examples.RandomWalkWithRestartComputation
>>>>> 2013-11-20 15:22:22,003 WARN [main] org.apache.hadoop.conf.Configuration:
>>>>> mapred.task.partition is deprecated. Instead, use mapreduce.task.partition
>>>>> 2013-11-20 15:22:22,003 WARN [main] org.apache.hadoop.conf.Configuration:mapred.job.id
is deprecated. Instead, use mapreduce.job.id
>>>>> 2013-11-20 15:22:22,004 WARN [main] org.apache.hadoop.conf.Configuration:
>>>>> job.local.dir is deprecated. Instead, use mapreduce.job.local.dir
>>>>> 2013-11-20 15:22:22,007 INFO [main] org.apache.giraph.zk.ZooKeeperManager:
>>>>> createCandidateStamp: Made the directory
>>>>> _bsp/_defaultZkManagerDir/job_1382563758657_470916
>>>>> 2013-11-20 15:22:22,008 INFO [main] org.apache.giraph.zk.ZooKeeperManager:
>>>>> createCandidateStamp: Made the directory
>>>>> _bsp/_defaultZkManagerDir/job_1382563758657_470916/_zkServer
>>>>> 2013-11-20 15:22:22,011 INFO [main] org.apache.giraph.zk.ZooKeeperManager:
>>>>> createCandidateStamp: Creating my filestamp
>>>>> _bsp/_defaultZkManagerDir/job_1382563758657_470916/_task/gsta60065.tan.ygrid.yahoo.com<http://tan.ygrid.yahoo.com/>
>>>>> 56
>>>>> 2013-11-20 15:22:22,027 INFO [main] org.apache.giraph.zk.ZooKeeperManager:
>>>>> getZooKeeperServerList: For task 56, got file 'null' (polling period
is 3000)
>>>>> 2013-11-20 15:22:25,030 INFO [main] org.apache.giraph.zk.ZooKeeperManager:
>>>>> getZooKeeperServerList: For task 56, got file
>>>>> 'zkServerList_gsta60065.tan.ygrid.yahoo.com<http://tan.ygrid.yahoo.com/>
56 '
>>>>> (polling period is 3000)
>>>>> 2013-11-20 15:22:25,030 INFO [main] org.apache.giraph.zk.ZooKeeperManager:
>>>>> getZooKeeperServerList: Found
>>>>> [gsta60065.tan.ygrid.yahoo.com<http://tan.ygrid.yahoo.com/>, 56]
2 hosts in
>>>>> filename
>>>>> 'zkServerList_gsta60065.tan.ygrid.yahoo.com<http://tan.ygrid.yahoo.com/>
56 '
>>>>> 2013-11-20 15:22:25,031 INFO [main] org.apache.giraph.zk.ZooKeeperManager:
>>>>> onlineZooKeeperServers: Trying to delete old directory
>>>>> /grid/0/tmp/yarn-local/usercache/bordino/appcache/application_1382563758657_470916/work/_bspZooKeeper
>>>>> 2013-11-20 15:22:25,038 INFO [main] org.apache.giraph.zk.ZooKeeperManager:
>>>>> generateZooKeeperConfigFile: Creating file
>>>>> /grid/0/tmp/yarn-local/usercache/bordino/appcache/application_1382563758657_470916/work/_bspZooKeeper/zoo.cfg
>>>>> in
>>>>> /grid/0/tmp/yarn-local/usercache/bordino/appcache/application_1382563758657_470916/work/_bspZooKeeper
>>>>> with base port 22181
>>>>> 2013-11-20 15:22:25,038 INFO [main] org.apache.giraph.zk.ZooKeeperManager:
>>>>> generateZooKeeperConfigFile: Make directory of _bspZooKeeper = true
>>>>> 2013-11-20 15:22:25,039 INFO [main] org.apache.giraph.zk.ZooKeeperManager:
>>>>> generateZooKeeperConfigFile: Delete of zoo.cfg = false
>>>>> 2013-11-20 15:22:25,041 INFO [main] org.apache.giraph.zk.ZooKeeperManager:
>>>>> onlineZooKeeperServers: Attempting to start ZooKeeper server with command
>>>>> [/home/Releases/gridjdk-1.7.0_17.1303042057-20130415-000/share/gridjdk-1.7.0_17/jre/bin/java,
>>>>> -Xmx128M, -XX:ParallelGCThreads=4, -XX:+UseConcMarkSweepGC,
>>>>> -XX:CMSInitiatingOccupancyFraction=70, -XX:MaxGCPauseMillis=100, -cp,
>>>>> /user/bordino/.staging/job_1382563758657_470916/job.jar,
>>>>> org.apache.zookeeper.server.quorum.QuorumPeerMain,
>>>>> /grid/0/tmp/yarn-local/usercache/bordino/appcache/application_1382563758657_470916/work/_bspZooKeeper/zoo.cfg]
>>>>> in directory
>>>>> /grid/0/tmp/yarn-local/usercache/bordino/appcache/application_1382563758657_470916/work/_bspZooKeeper
>>>>> 2013-11-20 15:22:25,044 INFO [main] org.apache.giraph.zk.ZooKeeperManager:
>>>>> onlineZooKeeperServers: Shutdown hook added.
>>>>> 2013-11-20 15:22:25,044 INFO [main] org.apache.giraph.zk.ZooKeeperManager:
>>>>> onlineZooKeeperServers: Connect attempt 0 of 10 max trying to connect
togsta60065.tan.ygrid.yahoo.com<http://tan.ygrid.yahoo.com/>:22181 with poll
>>>>> msecs = 3000
>>>>> 2013-11-20 15:22:25,048 WARN [main] org.apache.giraph.zk.ZooKeeperManager:
>>>>> onlineZooKeeperServers: Got ConnectException
>>>>> java.net.ConnectException: Connection refused
>>>>>         at java.net.PlainSocketImpl.socketConnect(Native Method)
>>>>>         at
>>>>> java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
>>>>>         at
>>>>> java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
>>>>>         at
>>>>> java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
>>>>>         at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391)
>>>>>         at java.net.Socket.connect(Socket.java:579)
>>>>>         at
>>>>> org.apache.giraph.zk.ZooKeeperManager.onlineZooKeeperServers(ZooKeeperManager.java:711)
>>>>>         at
>>>>> org.apache.giraph.graph.GraphTaskManager.startZooKeeperManager(GraphTaskManager.java:369)
>>>>>         at
>>>>> org.apache.giraph.graph.GraphTaskManager.setup(GraphTaskManager.java:202)
>>>>>         at org.apache.giraph.graph.GraphMapper.setup(GraphMapper.java:59)
>>>>>         at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:89)
>>>>>         at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:726)
>>>>>         at org.apache.hadoop.mapred.MapTask.run(MapTask.java:333)
>>>>>         at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:158)
>>>>>         at java.security.AccessController.doPrivileged(Native Method)
>>>>>         at javax.security.auth.Subject.doAs(Subject.java:415)
>>>>>         at
>>>>> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1262)
>>>>>         at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:153)
>>>>> 2013-11-20 15:22:28,050 INFO [main] org.apache.giraph.zk.ZooKeeperManager:
>>>>> onlineZooKeeperServers: Connect attempt 1 of 10 max trying to connect
togsta60065.tan.ygrid.yahoo.com<http://tan.ygrid.yahoo.com/>:22181 with poll
>>>>> msecs = 3000
>>>>> 2013-11-20 15:22:28,050 WARN [main] org.apache.giraph.zk.ZooKeeperManager:
>>>>> onlineZooKeeperServers: Got ConnectException
>>>>> java.net.ConnectException: Connection refused
>>>>>         at java.net.PlainSocketImpl.socketConnect(Native Method)
>>>>>         at
>>>>> java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
>>>>>         at
>>>>> java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
>>>>>         at
>>>>> java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
>>>>>         at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391)
>>>>>         at java.net.Socket.connect(Socket.java:579)
>>>>>         at
>>>>> org.apache.giraph.zk.ZooKeeperManager.onlineZooKeeperServers(ZooKeeperManager.java:711)
>>>>>         at
>>>>> org.apache.giraph.graph.GraphTaskManager.startZooKeeperManager(GraphTaskManager.java:369)
>>>>>         at
>>>>> org.apache.giraph.graph.GraphTaskManager.setup(GraphTaskManager.java:202)
>>>>>         at org.apache.giraph.graph.GraphMapper.setup(GraphMapper.java:59)
>>>>>         at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:89)
>>>>>         at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:726)
>>>>>         at org.apache.hadoop.mapred.MapTask.run(MapTask.java:333)
>>>>>         at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:158)
>>>>>         at java.security.AccessController.doPrivileged(Native Method)
>>>>>         at javax.security.auth.Subject.doAs(Subject.java:415)
>>>>>         at
>>>>> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1262)
>>>>>         at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:153)
>>>>> 2013-11-20 15:22:31,051 INFO [main] org.apache.giraph.zk.ZooKeeperManager:
>>>>> onlineZooKeeperServers: Connect attempt 2 of 10 max trying to connect
togsta60065.tan.ygrid.yahoo.com<http://tan.ygrid.yahoo.com/>:22181 with poll
>>>>> msecs = 3000
>>>>> 2013-11-20 15:22:31,051 WARN [main] org.apache.giraph.zk.ZooKeeperManager:
>>>>> onlineZooKeeperServers: Got ConnectException
>>>>> java.net.ConnectException: Connection refused
>>>>>         at java.net.PlainSocketImpl.socketConnect(Native Method)
>>>>>         at
>>>>> java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
>>>>>         at
>>>>> java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
>>>>>         at
>>>>> java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
>>>>>         at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391)
>>>>>         at java.net.Socket.connect(Socket.java:579)
>>>>>         at
>>>>> org.apache.giraph.zk.ZooKeeperManager.onlineZooKeeperServers(ZooKeeperManager.java:711)
>>>>>         at
>>>>> org.apache.giraph.graph.GraphTaskManager.startZooKeeperManager(GraphTaskManager.java:369)
>>>>>         at
>>>>> org.apache.giraph.graph.GraphTaskManager.setup(GraphTaskManager.java:202)
>>>>>         at org.apache.giraph.graph.GraphMapper.setup(GraphMapper.java:59)
>>>>>         at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:89)
>>>>>         at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:726)
>>>>>         at org.apache.hadoop.mapred.MapTask.run(MapTask.java:333)
>>>>>         at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:158)
>>>>>         at java.security.AccessController.doPrivileged(Native Method)
>>>>>         at javax.security.auth.Subject.doAs(Subject.java:415)
>>>>>         at
>>>>> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1262)
>>>>>         at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:153)
>>>>> 2013-11-20 15:22:34,052 INFO [main] org.apache.giraph.zk.ZooKeeperManager:
>>>>> onlineZooKeeperServers: Connect attempt 3 of 10 max trying to connect
togsta60065.tan.ygrid.yahoo.com<http://tan.ygrid.yahoo.com/>:22181 with poll
>>>>> msecs = 3000
>>>>> 2013-11-20 15:22:34,052 WARN [main] org.apache.giraph.zk.ZooKeeperManager:
>>>>> onlineZooKeeperServers: Got ConnectException
>>>>> java.net.ConnectException: Connection refused
>>>>>         at java.net.PlainSocketImpl.socketConnect(Native Method)
>>>>>         at
>>>>> java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
>>>>>         at
>>>>> java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
>>>>>         at
>>>>> java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
>>>>>         at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391)
>>>>>         at java.net.Socket.connect(Socket.java:579)
>>>>>         at
>>>>> org.apache.giraph.zk.ZooKeeperManager.onlineZooKeeperServers(ZooKeeperManager.java:711)
>>>>>         at
>>>>> org.apache.giraph.graph.GraphTaskManager.startZooKeeperManager(GraphTaskManager.java:369)
>>>>>         at
>>>>> org.apache.giraph.graph.GraphTaskManager.setup(GraphTaskManager.java:202)
>>>>>         at org.apache.giraph.graph.GraphMapper.setup(GraphMapper.java:59)
>>>>>         at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:89)
>>>>>         at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:726)
>>>>>         at org.apache.hadoop.mapred.MapTask.run(MapTask.java:333)
>>>>>         at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:158)
>>>>>         at java.security.AccessController.doPrivileged(Native Method)
>>>>>         at javax.security.auth.Subject.doAs(Subject.java:415)
>>>>>         at
>>>>> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1262)
>>>>>         at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:153)
>>>>> 2013-11-20 15:22:37,053 INFO [main] org.apache.giraph.zk.ZooKeeperManager:
>>>>> onlineZooKeeperServers: Connect attempt 4 of 10 max trying to connect
togsta60065.tan.ygrid.yahoo.com<http://tan.ygrid.yahoo.com/>:22181 with poll
>>>>> msecs = 3000
>>>>> 2013-11-20 15:22:37,053 WARN [main] org.apache.giraph.zk.ZooKeeperManager:
>>>>> onlineZooKeeperServers: Got ConnectException
>>>>> java.net.ConnectException: Connection refused
>>>>>         at java.net.PlainSocketImpl.socketConnect(Native Method)
>>>>>         at
>>>>> java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
>>>>>         at
>>>>> java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
>>>>>         at
>>>>> java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
>>>>>         at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391)
>>>>>         at java.net.Socket.connect(Socket.java:579)
>>>>>         at
>>>>> org.apache.giraph.zk.ZooKeeperManager.onlineZooKeeperServers(ZooKeeperManager.java:711)
>>>>>         at
>>>>> org.apache.giraph.graph.GraphTaskManager.startZooKeeperManager(GraphTaskManager.java:369)
>>>>>         at
>>>>> org.apache.giraph.graph.GraphTaskManager.setup(GraphTaskManager.java:202)
>>>>>         at org.apache.giraph.graph.GraphMapper.setup(GraphMapper.java:59)
>>>>>         at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:89)
>>>>>         at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:726)
>>>>>         at org.apache.hadoop.mapred.MapTask.run(MapTask.java:333)
>>>>>         at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:158)
>>>>>         at java.security.AccessController.doPrivileged(Native Method)
>>>>>         at javax.security.auth.Subject.doAs(Subject.java:415)
>>>>>         at
>>>>> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1262)
>>>>>         at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:153)
>>>>> 2013-11-20 15:22:40,054 INFO [main] org.apache.giraph.zk.ZooKeeperManager:
>>>>> onlineZooKeeperServers: Connect attempt 5 of 10 max trying to connect
togsta60065.tan.ygrid.yahoo.com<http://tan.ygrid.yahoo.com/>:22181 with poll
>>>>> msecs = 3000
>>>>> 2013-11-20 15:22:40,054 WARN [main] org.apache.giraph.zk.ZooKeeperManager:
>>>>> onlineZooKeeperServers: Got ConnectException
>>>>> java.net.ConnectException: Connection refused
>>>>>         at java.net.PlainSocketImpl.socketConnect(Native Method)
>>>>>         at
>>>>> java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
>>>>>         at
>>>>> java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
>>>>>         at
>>>>> java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
>>>>>         at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391)
>>>>>         at java.net.Socket.connect(Socket.java:579)
>>>>>         at
>>>>> org.apache.giraph.zk.ZooKeeperManager.onlineZooKeeperServers(ZooKeeperManager.java:711)
>>>>>         at
>>>>> org.apache.giraph.graph.GraphTaskManager.startZooKeeperManager(GraphTaskManager.java:369)
>>>>>         at
>>>>> org.apache.giraph.graph.GraphTaskManager.setup(GraphTaskManager.java:202)
>>>>>         at org.apache.giraph.graph.GraphMapper.setup(GraphMapper.java:59)
>>>>>         at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:89)
>>>>>         at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:726)
>>>>>         at org.apache.hadoop.mapred.MapTask.run(MapTask.java:333)
>>>>>         at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:158)
>>>>>         at java.security.AccessController.doPrivileged(Native Method)
>>>>>         at javax.security.auth.Subject.doAs(Subject.java:415)
>>>>>         at
>>>>> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1262)
>>>>>         at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:153)
>>>>> 2013-11-20 15:22:43,055 INFO [main] org.apache.giraph.zk.ZooKeeperManager:
>>>>> onlineZooKeeperServers: Connect attempt 6 of 10 max trying to connect
togsta60065.tan.ygrid.yahoo.com<http://tan.ygrid.yahoo.com/>:22181 with poll
>>>>> msecs = 3000
>>>>> 2013-11-20 15:22:43,055 WARN [main] org.apache.giraph.zk.ZooKeeperManager:
>>>>> onlineZooKeeperServers: Got ConnectException
>>>>> java.net.ConnectException: Connection refused
>>>>>         at java.net.PlainSocketImpl.socketConnect(Native Method)
>>>>>         at
>>>>> java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
>>>>>         at
>>>>> java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
>>>>>         at
>>>>> java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
>>>>>         at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391)
>>>>>         at java.net.Socket.connect(Socket.java:579)
>>>>>         at
>>>>> org.apache.giraph.zk.ZooKeeperManager.onlineZooKeeperServers(ZooKeeperManager.java:711)
>>>>>         at
>>>>> org.apache.giraph.graph.GraphTaskManager.startZooKeeperManager(GraphTaskManager.java:369)
>>>>>         at
>>>>> org.apache.giraph.graph.GraphTaskManager.setup(GraphTaskManager.java:202)
>>>>>         at org.apache.giraph.graph.GraphMapper.setup(GraphMapper.java:59)
>>>>>         at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:89)
>>>>>         at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:726)
>>>>>         at org.apache.hadoop.mapred.MapTask.run(MapTask.java:333)
>>>>>         at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:158)
>>>>>         at java.security.AccessController.doPrivileged(Native Method)
>>>>>         at javax.security.auth.Subject.doAs(Subject.java:415)
>>>>>         at
>>>>> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1262)
>>>>>         at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:153)
>>>>> 2013-11-20 15:22:46,055 INFO [main] org.apache.giraph.zk.ZooKeeperManager:
>>>>> onlineZooKeeperServers: Connect attempt 7 of 10 max trying to connect
togsta60065.tan.ygrid.yahoo.com<http://tan.ygrid.yahoo.com/>:22181 with poll
>>>>> msecs = 3000
>>>>> 2013-11-20 15:22:46,056 WARN [main] org.apache.giraph.zk.ZooKeeperManager:
>>>>> onlineZooKeeperServers: Got ConnectException
>>>>> java.net.ConnectException: Connection refused
>>>>>         at java.net.PlainSocketImpl.socketConnect(Native Method)
>>>>>         at
>>>>> java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
>>>>>         at
>>>>> java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
>>>>>         at
>>>>> java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
>>>>>         at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391)
>>>>>         at java.net.Socket.connect(Socket.java:579)
>>>>>         at
>>>>> org.apache.giraph.zk.ZooKeeperManager.onlineZooKeeperServers(ZooKeeperManager.java:711)
>>>>>         at
>>>>> org.apache.giraph.graph.GraphTaskManager.startZooKeeperManager(GraphTaskManager.java:369)
>>>>>         at
>>>>> org.apache.giraph.graph.GraphTaskManager.setup(GraphTaskManager.java:202)
>>>>>         at org.apache.giraph.graph.GraphMapper.setup(GraphMapper.java:59)
>>>>>         at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:89)
>>>>>         at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:726)
>>>>>         at org.apache.hadoop.mapred.MapTask.run(MapTask.java:333)
>>>>>         at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:158)
>>>>>         at java.security.AccessController.doPrivileged(Native Method)
>>>>>         at javax.security.auth.Subject.doAs(Subject.java:415)
>>>>>         at
>>>>> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1262)
>>>>>         at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:153)
>>>>> 2013-11-20 15:22:49,056 INFO [main] org.apache.giraph.zk.ZooKeeperManager:
>>>>> onlineZooKeeperServers: Connect attempt 8 of 10 max trying to connect
togsta60065.tan.ygrid.yahoo.com<http://tan.ygrid.yahoo.com/>:22181 with poll
>>>>> msecs = 3000
>>>>> 2013-11-20 15:22:49,057 WARN [main] org.apache.giraph.zk.ZooKeeperManager:
>>>>> onlineZooKeeperServers: Got ConnectException
>>>>> java.net.ConnectException: Connection refused
>>>>>         at java.net.PlainSocketImpl.socketConnect(Native Method)
>>>>>         at
>>>>> java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
>>>>>         at
>>>>> java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
>>>>>         at
>>>>> java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
>>>>>         at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391)
>>>>>         at java.net.Socket.connect(Socket.java:579)
>>>>>         at
>>>>> org.apache.giraph.zk.ZooKeeperManager.onlineZooKeeperServers(ZooKeeperManager.java:711)
>>>>>         at
>>>>> org.apache.giraph.graph.GraphTaskManager.startZooKeeperManager(GraphTaskManager.java:369)
>>>>>         at
>>>>> org.apache.giraph.graph.GraphTaskManager.setup(GraphTaskManager.java:202)
>>>>>         at org.apache.giraph.graph.GraphMapper.setup(GraphMapper.java:59)
>>>>>         at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:89)
>>>>>         at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:726)
>>>>>         at org.apache.hadoop.mapred.MapTask.run(MapTask.java:333)
>>>>>         at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:158)
>>>>>         at java.security.AccessController.doPrivileged(Native Method)
>>>>>         at javax.security.auth.Subject.doAs(Subject.java:415)
>>>>>         at
>>>>> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1262)
>>>>>         at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:153)
>>>>> 2013-11-20 15:22:52,057 INFO [main] org.apache.giraph.zk.ZooKeeperManager:
>>>>> onlineZooKeeperServers: Connect attempt 9 of 10 max trying to connect
togsta60065.tan.ygrid.yahoo.com<http://tan.ygrid.yahoo.com/>:22181 with poll
>>>>> msecs = 3000
>>>>> 2013-11-20 15:22:52,058 WARN [main] org.apache.giraph.zk.ZooKeeperManager:
>>>>> onlineZooKeeperServers: Got ConnectException
>>>>> java.net.ConnectException: Connection refused
>>>>>         at java.net.PlainSocketImpl.socketConnect(Native Method)
>>>>>         at
>>>>> java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
>>>>>         at
>>>>> java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
>>>>>         at
>>>>> java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
>>>>>         at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391)
>>>>>         at java.net.Socket.connect(Socket.java:579)
>>>>>         at
>>>>> org.apache.giraph.zk.ZooKeeperManager.onlineZooKeeperServers(ZooKeeperManager.java:711)
>>>>>         at
>>>>> org.apache.giraph.graph.GraphTaskManager.startZooKeeperManager(GraphTaskManager.java:369)
>>>>>         at
>>>>> org.apache.giraph.graph.GraphTaskManager.setup(GraphTaskManager.java:202)
>>>>>         at org.apache.giraph.graph.GraphMapper.setup(GraphMapper.java:59)
>>>>>         at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:89)
>>>>>         at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:726)
>>>>>         at org.apache.hadoop.mapred.MapTask.run(MapTask.java:333)
>>>>>         at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:158)
>>>>>         at java.security.AccessController.doPrivileged(Native Method)
>>>>>         at javax.security.auth.Subject.doAs(Subject.java:415)
>>>>>         at
>>>>> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1262)
>>>>>         at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:153)
>>>>> 2013-11-20 15:22:55,060 WARN [main] org.apache.hadoop.mapred.YarnChild:
>>>>> Exception running child : java.lang.IllegalStateException: run: Caught
an
>>>>> unrecoverable exception onlineZooKeeperServers: Failed to connect in
10 tries!
>>>>>         at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:101)
>>>>>         at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:726)
>>>>>         at org.apache.hadoop.mapred.MapTask.run(MapTask.java:333)
>>>>>         at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:158)
>>>>>         at java.security.AccessController.doPrivileged(Native Method)
>>>>>         at javax.security.auth.Subject.doAs(Subject.java:415)
>>>>>         at
>>>>> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1262)
>>>>>         at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:153)
>>>>> Caused by: java.lang.IllegalStateException: onlineZooKeeperServers: Failed
to
>>>>> connect in 10 tries!
>>>>>         at
>>>>> org.apache.giraph.zk.ZooKeeperManager.onlineZooKeeperServers(ZooKeeperManager.java:737)
>>>>>         at
>>>>> org.apache.giraph.graph.GraphTaskManager.startZooKeeperManager(GraphTaskManager.java:369)
>>>>>         at
>>>>> org.apache.giraph.graph.GraphTaskManager.setup(GraphTaskManager.java:202)
>>>>>         at org.apache.giraph.graph.GraphMapper.setup(GraphMapper.java:59)
>>>>>         at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:89)
>>>>>         ... 7 more
>>>>>
>>>>> 2013-11-20 15:22:55,063 INFO [main] org.apache.hadoop.mapred.Task: Runnning
>>>>> cleanup for the task
>>>>> 2013-11-20 15:22:55,110 WARN [main] org.apache.hadoop.conf.Configuration:
>>>>> job.xml:an attempt to override final parameter: dfs.namenode.name.dir;
>>>>> Ignoring.
>>>>> 2013-11-20 15:22:55,110 WARN [main] org.apache.hadoop.conf.Configuration:
>>>>> job.xml:an attempt to override final parameter:
>>>>> hadoop.security.token.service.use_ip;  Ignoring.
>>>>> 2013-11-20 15:22:55,110 WARN [main] org.apache.hadoop.conf.Configuration:
>>>>> job.xml:an attempt to override final parameter:
>>>>> mapreduce.job.end-notification.max.retry.interval;  Ignoring.
>>>>> 2013-11-20 15:22:55,110 WARN [main] org.apache.hadoop.conf.Configuration:
>>>>> job.xml:an attempt to override final parameter:
>>>>> mapreduce.admin.map.child.java.opts;  Ignoring.
>>>>> 2013-11-20 15:22:55,111 WARN [main] org.apache.hadoop.conf.Configuration:
>>>>> job.xml:an attempt to override final parameter:
>>>>> mapreduce.job.end-notification.max.attempts;  Ignoring.
>>>>> 2013-11-20 15:22:55,111 WARN [main] org.apache.hadoop.conf.Configuration:
>>>>> job.xml:an attempt to override final parameter:
>>>>> yarn.app.mapreduce.am.job.client.port-range;  Ignoring.
>>>>> 2013-11-20 15:22:55,111 WARN [main] org.apache.hadoop.conf.Configuration:
>>>>> job.xml:an attempt to override final parameter:
>>>>> mapreduce.admin.reduce.child.java.opts;  Ignoring.
>>>>> 2013-11-20 15:22:55,111 WARN [main] org.apache.hadoop.conf.Configuration:
>>>>> job.xml:an attempt to override final parameter: hadoop.tmp.dir;  Ignoring.
>>>>> 2013-11-20 15:22:55,117 WARN [main]
>>>>> org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter: Could not
delete
>>>>> hdfs://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/bordino/test-giraph-tmp/_temporary/1/_temporary/attempt_1382563758657_470916_m_000056_0
>>>>> 2013-11-20 <http://zaniumtan-nn1.tan.ygrid.yahoo.com:8020/user/bordino/test-giraph-tmp/_temporary/1/_temporary/attempt_1382563758657_470916_m_000056_02013-11-20>
15:22:55,121 INFO [main]
>>>>> org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Stopping MapTask metrics
>>>>> system...
>>>>> 2013-11-20 15:22:55,122 INFO [main]
>>>>> org.apache.hadoop.metrics2.impl.MetricsSystemImpl: MapTask metrics system
>>>>> stopped.
>>>>> 2013-11-20 15:22:55,122 INFO [main]
>>>>> org.apache.hadoop.metrics2.impl.MetricsSystemImpl: MapTask metrics system
>>>>> shutdown complete.
>>>>>
>>>>>
>>>>>
>>>>> Cheres,
>>>>> --
>>>>> Gianmarco
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>>    Claudio Martella
>>>>    claudio.martella@gmail.com
>>>>
>>>
>>>
>>
>>
>> --
>>    Claudio Martella
>>    claudio.martella@gmail.com
>>
>
>

Mime
View raw message