giraph-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Cristian Vidal Silva <vidal...@msu.edu>
Subject A Giraph Input Problem: Reading Two One Value from a Vertex
Date Sun, 07 Jun 2015 16:41:48 GMT

Hey Everione.

I need to read vertices in the format [Id, Val1, Val2,
[[taget1, weight1], [target2, weight2]..]], and I want to get out simply
the sum of its vertex values... For example, for the tiny_graph.txt file:

[0,1,2,[[1,1],[3,3]]]
[1,3,4,[[0,1],[2,2],[3,1]]]
[2,5,5,[[1,2],[4,4]]]
[3,6,7,[[0,3],[1,1],[4,4]]]
[4,8,9,[[3,4],[2,4]]]

So, the output would be like:
0,3
1,7
2,10
3,13
4,17

What must I do to achieve this goal? What classes Do I have to define?
How Do I link these classes? I followed adives of http://stackoverflow.com/questions/24800957/vertices-with-complex-values-in-apache-giraph,
so I created a few new classes TripletVertexInputFormat.java and
TripletVertexWritable.java in the Giraph examples folder.

1.tar.gz presents a code to show each Vertex Id and its value.
This code functions to read the tiny_graph.txt file. In Eclipse, I use
as an Run As - Run Configuration - Argument
"org.apache.giraph.examples.SimpleShortestPathsComputation -vif
org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputFormat
-vip /usr/local/tiny_graph.txt -vof
org.apache.giraph.io.formats.IdWithValueTextOutputFormat -op
/usr/local/output/shortestpaths -w 1 -ca giraph.SplitMasterWorker=false".

Nertheless, in 2.tar.gz, when I define new classes to read a 2nd argument
in the tiny_graph2.txt file, the used Run As - Run Configuration -
Argument is "org.apache.giraph.examples.SimpleShortestPathsComputation
-vif org.apache.giraph.examples.TripletVertexInputFormat -vip
/usr/local/tiny_graph2.txt -vof
org.apache.giraph.io.formats.IdWithValueTextOutputFormat -op
/usr/local/output/shortestpaths -w 1 -ca
giraph.SplitMasterWorker=false",  shows me the following error messages:

- 2015-06-03 00:36:19,985 ERROR [load-0] utils.LogStacktraceCallable
(LogStacktraceCallable.java:call(57)) - Execution of callable failed
java.lang.IllegalStateException: newInstance: Illegal access
org.apache.giraph.examples.TripletVertexWritable
      at
org.apache.giraph.utils.ReflectionUtils.newInstance(ReflectionUtils.java:86)
      at
org.apache.giraph.utils.WritableUtils.createWritable(WritableUtils.java:84)
      at
org.apache.giraph.factories.DefaultVertexValueFactory.newInstance(DefaultVertexValueFactory.java:47)
      at
org.apache.giraph.conf.ImmutableClassesGiraphConfiguration.createVertexValue(ImmutableClassesGiraphConfiguration.java:717)
      at
org.apache.giraph.utils.VertexIterator.resetEmptyVertex(VertexIterator.java:69)
      at
org.apache.giraph.utils.VertexIterator.<init>(VertexIterator.java:60)
      at
org.apache.giraph.comm.requests.SendWorkerVerticesRequest.doRequest(SendWorkerVerticesRequest.java:108)
      at
org.apache.giraph.comm.netty.NettyWorkerClientRequestProcessor.doRequest(NettyWorkerClientRequestProcessor.java:467)
      at
org.apache.giraph.comm.netty.NettyWorkerClientRequestProcessor.flush(NettyWorkerClientRequestProcessor.java:413)
      at
org.apache.giraph.worker.InputSplitsCallable.call(InputSplitsCallable.java:242)
      at
org.apache.giraph.worker.InputSplitsCallable.call(InputSplitsCallable.java:60)
      at
org.apache.giraph.utils.LogStacktraceCallable.call(LogStacktraceCallable.java:51)
      at java.util.concurrent.FutureTask.run(FutureTask.java:262)
      at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
      at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
      at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalAccessException: Class
org.apache.giraph.utils.ReflectionUtils can not access a member of class
org.apache.giraph.examples.TripletVertexWritable with modifiers "public"
      at sun.reflect.Reflection.ensureMemberAccess(Reflection.java:109)
      at java.lang.Class.newInstance(Class.java:373)
      at
org.apache.giraph.utils.ReflectionUtils.newInstance(ReflectionUtils.java:81)
      ... 15 more
2015-06-03 00:36:19,998 ERROR [LocalJobRunner Map Task Executor #0]
graph.GraphMapper (GraphMapper.java:run(99)) - Caught an unrecoverable
exception waitFor: ExecutionException occurred while waiting for
org.apache.giraph.utils.ProgressableUtils$FutureWaitable@3c1de3e3
java.lang.IllegalStateException: waitFor: ExecutionException occurred
while waiting for
org.apache.giraph.utils.ProgressableUtils$FutureWaitable@3c1de3e3
      at
org.apache.giraph.utils.ProgressableUtils.waitFor(ProgressableUtils.java:193)
      at
org.apache.giraph.utils.ProgressableUtils.waitForever(ProgressableUtils.java:151)
      at
org.apache.giraph.utils.ProgressableUtils.waitForever(ProgressableUtils.java:136)
      at
org.apache.giraph.utils.ProgressableUtils.getFutureResult(ProgressableUtils.java:99)
      at
org.apache.giraph.utils.ProgressableUtils.getResultsWithNCallables(ProgressableUtils.java:233)
      at
org.apache.giraph.worker.BspServiceWorker.loadInputSplits(BspServiceWorker.java:323)
      at
org.apache.giraph.worker.BspServiceWorker.loadVertices(BspServiceWorker.java:416)
      at
org.apache.giraph.worker.BspServiceWorker.setup(BspServiceWorker.java:636)
      at
org.apache.giraph.graph.GraphTaskManager.execute(GraphTaskManager.java:296)
      at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:91)
      at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)
      at org.apache.hadoop.mapred.MapTask.run(MapTask.java:340)
      at
org.apache.hadoop.mapred.LocalJobRunner$Job$MapTaskRunnable.run(LocalJobRunner.java:243)
      at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
      at java.util.concurrent.FutureTask.run(FutureTask.java:262)
      at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
      at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
      at java.lang.Thread.run(Thread.java:745)
Caused by: java.util.concurrent.ExecutionException:
java.lang.IllegalStateException: newInstance: Illegal access
org.apache.giraph.examples.TripletVertexWritable
      at java.util.concurrent.FutureTask.report(FutureTask.java:122)
      at java.util.concurrent.FutureTask.get(FutureTask.java:202)
      at
org.apache.giraph.utils.ProgressableUtils$FutureWaitable.waitFor(ProgressableUtils.java:312)
      at
org.apache.giraph.utils.ProgressableUtils.waitFor(ProgressableUtils.java:185)
      ... 17 more
Caused by: java.lang.IllegalStateException: newInstance: Illegal access
org.apache.giraph.examples.TripletVertexWritable
      at
org.apache.giraph.utils.ReflectionUtils.newInstance(ReflectionUtils.java:86)
      at
org.apache.giraph.utils.WritableUtils.createWritable(WritableUtils.java:84)
      at
org.apache.giraph.factories.DefaultVertexValueFactory.newInstance(DefaultVertexValueFactory.java:47)
      at
org.apache.giraph.conf.ImmutableClassesGiraphConfiguration.createVertexValue(ImmutableClassesGiraphConfiguration.java:717)
      at
org.apache.giraph.utils.VertexIterator.resetEmptyVertex(VertexIterator.java:69)
      at
org.apache.giraph.utils.VertexIterator.<init>(VertexIterator.java:60)
      at
org.apache.giraph.comm.requests.SendWorkerVerticesRequest.doRequest(SendWorkerVerticesRequest.java:108)
      at
org.apache.giraph.comm.netty.NettyWorkerClientRequestProcessor.doRequest(NettyWorkerClientRequestProcessor.java:467)
      at
org.apache.giraph.comm.netty.NettyWorkerClientRequestProcessor.flush(NettyWorkerClientRequestProcessor.java:413)
      at
org.apache.giraph.worker.InputSplitsCallable.call(InputSplitsCallable.java:242)
      at
org.apache.giraph.worker.InputSplitsCallable.call(InputSplitsCallable.java:60)
      at
org.apache.giraph.utils.LogStacktraceCallable.call(LogStacktraceCallable.java:51)
      ... 4 more
Caused by: java.lang.IllegalAccessException: Class
org.apache.giraph.utils.ReflectionUtils can not access a member of class
org.apache.giraph.examples.TripletVertexWritable with modifiers "public"
      at sun.reflect.Reflection.ensureMemberAccess(Reflection.java:109)
      at java.lang.Class.newInstance(Class.java:373)
      at
org.apache.giraph.utils.ReflectionUtils.newInstance(ReflectionUtils.java:81)
      ... 15 more
2015-06-03 00:36:20,000 ERROR [LocalJobRunner Map Task Executor #0]
worker.BspServiceWorker (BspServiceWorker.java:unregisterHealth(784)) -
unregisterHealth: Got failure, unregistering health on
/_hadoopBsp/job_local869455398_0001/_applicationAttemptsDir/0/_superstepDir/-1/_workerHealthyDir/ubuntu_0
on superstep -1
2015-06-03 00:36:20,011 INFO  [Thread-13] mapred.LocalJobRunner
(LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-06-03 00:36:20,016 WARN  [Thread-13] mapred.LocalJobRunner
(LocalJobRunner.java:run(560)) - job_local869455398_0001
java.lang.Exception: java.lang.IllegalStateException: run: Caught an
unrecoverable exception waitFor: ExecutionException occurred while
waiting for
org.apache.giraph.utils.ProgressableUtils$FutureWaitable@3c1de3e3
      at
org.apache.hadoop.mapred.LocalJobRunner$Job.runTasks(LocalJobRunner.java:462)
      at
org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:522)
Caused by: java.lang.IllegalStateException: run: Caught an unrecoverable
exception waitFor: ExecutionException occurred while waiting for
org.apache.giraph.utils.ProgressableUtils$FutureWaitable@3c1de3e3
      at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:102)
      at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)
      at org.apache.hadoop.mapred.MapTask.run(MapTask.java:340)
      at
org.apache.hadoop.mapred.LocalJobRunner$Job$MapTaskRunnable.run(LocalJobRunner.java:243)
      at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
      at java.util.concurrent.FutureTask.run(FutureTask.java:262)
      at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
      at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
      at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalStateException: waitFor: ExecutionException
occurred while waiting for
org.apache.giraph.utils.ProgressableUtils$FutureWaitable@3c1de3e3
      at
org.apache.giraph.utils.ProgressableUtils.waitFor(ProgressableUtils.java:193)
      at
org.apache.giraph.utils.ProgressableUtils.waitForever(ProgressableUtils.java:151)
      at
org.apache.giraph.utils.ProgressableUtils.waitForever(ProgressableUtils.java:136)
      at
org.apache.giraph.utils.ProgressableUtils.getFutureResult(ProgressableUtils.java:99)
      at
org.apache.giraph.utils.ProgressableUtils.getResultsWithNCallables(ProgressableUtils.java:233)
      at
org.apache.giraph.worker.BspServiceWorker.loadInputSplits(BspServiceWorker.java:323)
      at
org.apache.giraph.worker.BspServiceWorker.loadVertices(BspServiceWorker.java:416)
      at
org.apache.giraph.worker.BspServiceWorker.setup(BspServiceWorker.java:636)
      at
org.apache.giraph.graph.GraphTaskManager.execute(GraphTaskManager.java:296)
      at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:91)
      ... 8 more
Caused by: java.util.concurrent.ExecutionException:
java.lang.IllegalStateException: newInstance: Illegal access
org.apache.giraph.examples.TripletVertexWritable
      at java.util.concurrent.FutureTask.report(FutureTask.java:122)
      at java.util.concurrent.FutureTask.get(FutureTask.java:202)
      at
org.apache.giraph.utils.ProgressableUtils$FutureWaitable.waitFor(ProgressableUtils.java:312)
      at
org.apache.giraph.utils.ProgressableUtils.waitFor(ProgressableUtils.java:185)
      ... 17 more
Caused by: java.lang.IllegalStateException: newInstance: Illegal access
org.apache.giraph.examples.TripletVertexWritable
      at
org.apache.giraph.utils.ReflectionUtils.newInstance(ReflectionUtils.java:86)
      at
org.apache.giraph.utils.WritableUtils.createWritable(WritableUtils.java:84)
      at
org.apache.giraph.factories.DefaultVertexValueFactory.newInstance(DefaultVertexValueFactory.java:47)
      at
org.apache.giraph.conf.ImmutableClassesGiraphConfiguration.createVertexValue(ImmutableClassesGiraphConfiguration.java:717)
      at
org.apache.giraph.utils.VertexIterator.resetEmptyVertex(VertexIterator.java:69)
      at
org.apache.giraph.utils.VertexIterator.<init>(VertexIterator.java:60)
      at
org.apache.giraph.comm.requests.SendWorkerVerticesRequest.doRequest(SendWorkerVerticesRequest.java:108)
      at
org.apache.giraph.comm.netty.NettyWorkerClientRequestProcessor.doRequest(NettyWorkerClientRequestProcessor.java:467)
      at
org.apache.giraph.comm.netty.NettyWorkerClientRequestProcessor.flush(NettyWorkerClientRequestProcessor.java:413)
      at
org.apache.giraph.worker.InputSplitsCallable.call(InputSplitsCallable.java:242)
      at
org.apache.giraph.worker.InputSplitsCallable.call(InputSplitsCallable.java:60)
      at
org.apache.giraph.utils.LogStacktraceCallable.call(LogStacktraceCallable.java:51)
      ... 4 more
Caused by: java.lang.IllegalAccessException: Class
org.apache.giraph.utils.ReflectionUtils can not access a member of class
org.apache.giraph.examples.TripletVertexWritable with modifiers "public"
      at sun.reflect.Reflection.ensureMemberAccess(Reflection.java:109)
      at java.lang.Class.newInstance(Class.java:373)
      at
org.apache.giraph.utils.ReflectionUtils.newInstance(ReflectionUtils.java:81)
      ... 15 more

Thanks for your assistance, and I wait for your comments.

-- 
Cristian Vidal Silva
Computer Engineer
Master in Computer Science, University of Concepcion - Chile
Master of Science in Computer Science, Michigan State University, MI - USA
Ph.D(c) in Software Engineering and Technology, University of Seville - Spain

Chilean Fulbright Scholar





Mime
View raw message