incubator-giraph-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sebastian Schelter (Updated) (JIRA)" <>
Subject [jira] [Updated] (GIRAPH-51) Provide unit testing tool for Giraph algorithms
Date Fri, 11 Nov 2011 10:14:52 GMT


Sebastian Schelter updated GIRAPH-51:

    Attachment: GIRAPH-51.patch

Hi, I'm currently trying to find a way to easily unit test giraph algorithms as suggested
here. I plan to implement some complicated stuff on top of Giraph and definitely need this
functionality for debugging and development.

I want to be able to repeatedly run a Vertex class in a unit test from my IDE without having
to cleanup directories or rely on an external zookeeper instance.

I managed to get a working prototype of a class that can internally run a Vertex. I had to
add configuration options to disable the starting of a zookeeper instance via ProcessBuilder
in ZooKeeperManager as this didn't work out of my IDE (Intellij). My testing class simply
starts its own zookeeper in an extra thread for the duration of the test. This works pretty
well and I was already able to unit the SimpleShortestPathVertex. 

Although everything seems to work, I'm still getting some strange warnings from the local
zookeeper instance. As I'm not that familiar with zookeeper yet, I wanted ask here whether
these warning can simply be ignored?

11/11/11 09:55:16 INFO server.PrepRequestProcessor: Got user-level KeeperException when processing
sessionid:0x13391d6887b0001 type:create cxid:0x1 zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a
Error Path:/_hadoopBsp/job_local_0001 Error:KeeperErrorCode = NoNode for /_hadoopBsp/job_local_0001

WARN graph.BspService: process: Unknown and unprocessed event (path=/_hadoopBsp/job_local_0001/_applicationAttemptsDir/0/_superstepDir,
type=NodeChildrenChanged, state=SyncConnected)

Already putting up a first patch for clarity.


> Provide unit testing tool for Giraph algorithms
> -----------------------------------------------
>                 Key: GIRAPH-51
>                 URL:
>             Project: Giraph
>          Issue Type: Improvement
>            Reporter: Jakob Homan
>         Attachments: GIRAPH-51.patch
> It would be nice to have a little tool, similar to MRUnit, that would allow Giraph application
writers to quickly unit test their algorithms.  The tool could take a Vertex implementation,
a set of input and expected output and verify that after the specified number of supersteps,
we've gotten what we expect.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:!default.jspa
For more information on JIRA, see:


View raw message