giraph-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Eli Reisman <apache.mail...@gmail.com>
Subject Re: Is Zookeeper a must for Giraph?
Date Wed, 26 Jun 2013 00:58:24 GMT
Hi all,

I put up a patch to add support for 2.0.4 and 2.0.5 but (sadly) have not
checked back in to see about its review status. It builds, I know that :)

Chaun it looks like you are using Hadoop-2.0.5-alpha but using our default
MapReduce v2 interface. So its a YARN cluster underneath, but your tasks
are still hosted in Mapper processes.

So. It seems like you need Zookeeper installed and then you'll be on your
way. Let us know if you need any more advice etc.


On Tue, Jun 25, 2013 at 4:00 PM, Chuan Lei <leichuan@gmail.com> wrote:

> Hi Avery,
>
> Thanks for your quick answer. Just for your information. I was able to fix
> the second issue (running pagerank example). I believe it has been
> addressed in GIRAPH-442 (https://issues.apache.org/jira/browse/GIRAPH-442).
> What confuses me is that I don't know why this patch is not applied to the
> latest release (1.0.0).
>
> Again, I appreciate your help.
>
> Best,
> Chuan
>
>
>
> On Mon, Jun 24, 2013 at 10:17 PM, Avery Ching <aching@apache.org> wrote:
>
>> Zookeeper is required.  That being said, you can have an external
>> Zookeeper or Giraph can start one for you.  It's your choice.
>>
>> Eli is the one to contact regarding Giraph on Hadoop 2.0.5.  Any thoughts
>> Eli?
>>
>> Avery
>>
>>
>> On 6/24/13 5:22 PM, Chuan Lei wrote:
>>
>>> It is not clear to me that whether Zookeeper is required or optional to
>>> Giraph. I wonder if it is possible to run Giraph without Zookeeper. If not
>>> the case, would the default Zookeeper work with Giraph? Is there anything
>>> has to be changed on Zookeeper.
>>>
>>> Another question is that I have the following error message when I ran
>>> Giraph on Hadoop-2.0.5 when I ran the PageRankBenchmark program. I saw
>>> similar posts on the mailing list, but it seems no clear answer to it yet.
>>> I would be grateful if someone can answer my question and resolve the issue.
>>>
>>> Error: java.lang.**IllegalStateException: run: Caught an unrecoverable
>>> exception java.io.FileNotFoundException: File _bsp/_defaultZkManagerDir/job_
>>> **1372108933881_0002/_zkServer does not exist. at
>>> org.apache.giraph.graph.**GraphMapper.run(GraphMapper.**java:102) at
>>> org.apache.hadoop.mapred.**MapTask.runNewMapper(MapTask.**java:757) at
>>> org.apache.hadoop.mapred.**MapTask.run(MapTask.java:339) at
>>> org.apache.hadoop.mapred.**YarnChild$2.run(YarnChild.**java:158) at
>>> java.security.**AccessController.doPrivileged(**Native Method) at
>>> javax.security.auth.Subject.**doAs(Subject.java:396) at
>>> org.apache.hadoop.security.**UserGroupInformation.doAs(**
>>> UserGroupInformation.java:**1478) at org.apache.hadoop.mapred.**
>>> YarnChild.main(YarnChild.java:**153) Caused by:
>>> java.lang.RuntimeException: java.io.FileNotFoundException: File
>>> _bsp/_defaultZkManagerDir/job_**1372108933881_0002/_zkServer does not
>>> exist. at org.apache.giraph.zk.**ZooKeeperManager.**
>>> onlineZooKeeperServers(**ZooKeeperManager.java:790) at
>>> org.apache.giraph.graph.**GraphTaskManager.**startZooKeeperManager(**GraphTaskManager.java:357)
>>> at org.apache.giraph.graph.**GraphTaskManager.setup(**GraphTaskManager.java:188)
>>> at org.apache.giraph.graph.**GraphMapper.setup(GraphMapper.**java:60)
>>> at org.apache.giraph.graph.**GraphMapper.run(GraphMapper.**java:90) ...
>>> 7 more Caused by: java.io.FileNotFoundException: File
>>> _bsp/_defaultZkManagerDir/job_**1372108933881_0002/_zkServer does not
>>> exist. at org.apache.hadoop.hdfs.**DistributedFileSystem.**listStatus(**
>>> DistributedFileSystem.java:**405) at org.apache.giraph.zk.**
>>> ZooKeeperManager.**onlineZooKeeperServers(**ZooKeeperManager.java:749)
>>> ... 11 more
>>>
>>> Regards,
>>> Chuan
>>>
>>
>>
>

Mime
View raw message