Mailing-List: contact user-help@giraph.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@giraph.apache.org
Received-SPF: pass (athena.apache.org: domain of kyle.r.orlando@gmail.com
 designates 209.85.214.180 as permitted sender)
MIME-Version: 1.0
Date: Wed, 31 Jul 2013 19:13:37 -0400
Message-ID: 
 <CAN-tzkwMZTS+EUo8FTME9JLiCDfPuvShng3Dqk42V-ZwvSbptA@mail.gmail.com>
Subject: Getting Giraph to work with HBase
From: Kyle Orlando <kyle.r.orlando@gmail.com>
To: user@giraph.apache.org
Content-Type: text/plain; charset=ISO-8859-1

Hello,

I've been having some problems getting Giraph to work with a table
that is stored in HBase.  I've created my own classes that extend
HBaseVertexInputFormat and HBaseVertexReader (respectively) and
implemented nextVertex() and getVertex() accordingly, but I still
can't get it to work. Here is what I typed into the command line after
specifying the GiraphRunner class:

org.apache.giraph.examples.SimplePageRankComputation -vif
HBasePractice.LongDoubleFloatHBaseVertexInputFormat -of
org.apache.giraph.io.formats.IdWithValueTextOutputFormat -op /hbaseOut
-w 2 -mc org.apache.giraph.examples.SimplePageRankComputation\$SimplePageRankMasterCompute

This is the error that I'm getting from the logs:

2013-07-31 17:20:28,880 ERROR org.apache.giraph.master.MasterThread:
masterThread: Master algorithm failed with NullPointerException
java.lang.NullPointerException
	at org.apache.hadoop.hbase.mapreduce.TableInputFormat.setConf(TableInputFormat.java:94)
	at org.apache.giraph.io.hbase.HBaseVertexInputFormat.getSplits(HBaseVertexInputFormat.java:169)
	at org.apache.giraph.io.internal.WrappedVertexInputFormat.getSplits(WrappedVertexInputFormat.java:72)
	at org.apache.giraph.master.BspServiceMaster.generateInputSplits(BspServiceMaster.java:315)
	at org.apache.giraph.master.BspServiceMaster.createInputSplits(BspServiceMaster.java:627)
	at org.apache.giraph.master.BspServiceMaster.createVertexInputSplits(BspServiceMaster.java:694)
	at org.apache.giraph.master.MasterThread.run(MasterThread.java:100)
2013-07-31 17:20:28,881 FATAL org.apache.giraph.graph.GraphMapper:
uncaughtException: OverrideExceptionHandler on thread
org.apache.giraph.master.MasterThread, msg =
java.lang.NullPointerException, exiting...
java.lang.IllegalStateException: java.lang.NullPointerException
	at org.apache.giraph.master.MasterThread.run(MasterThread.java:185)
Caused by: java.lang.NullPointerException
	at org.apache.hadoop.hbase.mapreduce.TableInputFormat.setConf(TableInputFormat.java:94)
	at org.apache.giraph.io.hbase.HBaseVertexInputFormat.getSplits(HBaseVertexInputFormat.java:169)
	at org.apache.giraph.io.internal.WrappedVertexInputFormat.getSplits(WrappedVertexInputFormat.java:72)
	at org.apache.giraph.master.BspServiceMaster.generateInputSplits(BspServiceMaster.java:315)
	at org.apache.giraph.master.BspServiceMaster.createInputSplits(BspServiceMaster.java:627)
	at org.apache.giraph.master.BspServiceMaster.createVertexInputSplits(BspServiceMaster.java:694)
	at org.apache.giraph.master.MasterThread.run(MasterThread.java:100)

 I don't know how/why a NullPointerException is being thrown.  Do I
need to specify the -vip option? I didn't think so, because in my code
for my class that extends HBaseVertexInputFormat I do the following:

private ImmutableClassesGiraphConfiguration<LongWritable,
DoubleWritable, FloatWritable> conf;

@Override
	  public void setConf(ImmutableClassesGiraphConfiguration<LongWritable,
DoubleWritable, FloatWritable> conf) {
		conf.set(TableInputFormat.INPUT_TABLE, "edges");
		this.conf = conf;
	  }

Help would be much appreciated.  I actually was a bit hesitant to post
this at first; I was determined to figure it out on my own, but then I
noticed that there have been virtually no questions asked regarding
HBase + Giraph on this mailing list, so I figured that posting this
might help someone out with similar problems later on in the future.

Thanks,
-- 
Kyle Orlando
Computer Engineering Major
University of Maryland