giraph-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sergey Edunov <edu...@gmail.com>
Subject Re: A Giraph Input Problem: Reading Two One Value from a Vertex
Date Sun, 07 Jun 2015 17:03:34 GMT
Hi Cristian,

You need to make TripletVertexWritable class public. It is package
protected now, so reflection utils can't create an instance. In general,
all writable classes should be public with public default constructor.

Regards,
Sergey Edunov

On Sun, Jun 7, 2015 at 9:41 AM, Cristian Vidal Silva <vidalsil@msu.edu>
wrote:

>
> Hey Everione.
>
> I need to read vertices in the format [Id, Val1, Val2,
> [[taget1, weight1], [target2, weight2]..]], and I want to get out simply
> the sum of its vertex values... For example, for the tiny_graph.txt file:
>
> [0,1,2,[[1,1],[3,3]]]
> [1,3,4,[[0,1],[2,2],[3,1]]]
> [2,5,5,[[1,2],[4,4]]]
> [3,6,7,[[0,3],[1,1],[4,4]]]
> [4,8,9,[[3,4],[2,4]]]
>
> So, the output would be like:
> 0,3
> 1,7
> 2,10
> 3,13
> 4,17
>
> What must I do to achieve this goal? What classes Do I have to define?
> How Do I link these classes? I followed adives of
> http://stackoverflow.com/questions/24800957/vertices-with-complex-values-in-apache-giraph
> ,
> so I created a few new classes TripletVertexInputFormat.java and
> TripletVertexWritable.java in the Giraph examples folder.
>
> 1.tar.gz presents a code to show each Vertex Id and its value.
> This code functions to read the tiny_graph.txt file. In Eclipse, I use
> as an Run As - Run Configuration - Argument
> "org.apache.giraph.examples.SimpleShortestPathsComputation -vif
> org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputFormat
> -vip /usr/local/tiny_graph.txt -vof
> org.apache.giraph.io.formats.IdWithValueTextOutputFormat -op
> /usr/local/output/shortestpaths -w 1 -ca giraph.SplitMasterWorker=false".
>
> Nertheless, in 2.tar.gz, when I define new classes to read a 2nd argument
> in the tiny_graph2.txt file, the used Run As - Run Configuration -
> Argument is "org.apache.giraph.examples.SimpleShortestPathsComputation
> -vif org.apache.giraph.examples.TripletVertexInputFormat -vip
> /usr/local/tiny_graph2.txt -vof
> org.apache.giraph.io.formats.IdWithValueTextOutputFormat -op
> /usr/local/output/shortestpaths -w 1 -ca
> giraph.SplitMasterWorker=false",  shows me the following error messages:
>
> - 2015-06-03 00:36:19,985 ERROR [load-0] utils.LogStacktraceCallable
> (LogStacktraceCallable.java:call(57)) - Execution of callable failed
> java.lang.IllegalStateException: newInstance: Illegal access
> org.apache.giraph.examples.TripletVertexWritable
>      at
>
> org.apache.giraph.utils.ReflectionUtils.newInstance(ReflectionUtils.java:86)
>      at
> org.apache.giraph.utils.WritableUtils.createWritable(WritableUtils.java:84)
>      at
>
> org.apache.giraph.factories.DefaultVertexValueFactory.newInstance(DefaultVertexValueFactory.java:47)
>      at
>
> org.apache.giraph.conf.ImmutableClassesGiraphConfiguration.createVertexValue(ImmutableClassesGiraphConfiguration.java:717)
>      at
>
> org.apache.giraph.utils.VertexIterator.resetEmptyVertex(VertexIterator.java:69)
>      at
> org.apache.giraph.utils.VertexIterator.<init>(VertexIterator.java:60)
>      at
>
> org.apache.giraph.comm.requests.SendWorkerVerticesRequest.doRequest(SendWorkerVerticesRequest.java:108)
>      at
>
> org.apache.giraph.comm.netty.NettyWorkerClientRequestProcessor.doRequest(NettyWorkerClientRequestProcessor.java:467)
>      at
>
> org.apache.giraph.comm.netty.NettyWorkerClientRequestProcessor.flush(NettyWorkerClientRequestProcessor.java:413)
>      at
>
> org.apache.giraph.worker.InputSplitsCallable.call(InputSplitsCallable.java:242)
>      at
>
> org.apache.giraph.worker.InputSplitsCallable.call(InputSplitsCallable.java:60)
>      at
>
> org.apache.giraph.utils.LogStacktraceCallable.call(LogStacktraceCallable.java:51)
>      at java.util.concurrent.FutureTask.run(FutureTask.java:262)
>      at
>
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>      at
>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>      at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.IllegalAccessException: Class
> org.apache.giraph.utils.ReflectionUtils can not access a member of class
> org.apache.giraph.examples.TripletVertexWritable with modifiers "public"
>      at sun.reflect.Reflection.ensureMemberAccess(Reflection.java:109)
>      at java.lang.Class.newInstance(Class.java:373)
>      at
>
> org.apache.giraph.utils.ReflectionUtils.newInstance(ReflectionUtils.java:81)
>      ... 15 more
> 2015-06-03 00:36:19,998 ERROR [LocalJobRunner Map Task Executor #0]
> graph.GraphMapper (GraphMapper.java:run(99)) - Caught an unrecoverable
> exception waitFor: ExecutionException occurred while waiting for
> org.apache.giraph.utils.ProgressableUtils$FutureWaitable@3c1de3e3
> java.lang.IllegalStateException: waitFor: ExecutionException occurred
> while waiting for
> org.apache.giraph.utils.ProgressableUtils$FutureWaitable@3c1de3e3
>      at
>
> org.apache.giraph.utils.ProgressableUtils.waitFor(ProgressableUtils.java:193)
>      at
>
> org.apache.giraph.utils.ProgressableUtils.waitForever(ProgressableUtils.java:151)
>      at
>
> org.apache.giraph.utils.ProgressableUtils.waitForever(ProgressableUtils.java:136)
>      at
>
> org.apache.giraph.utils.ProgressableUtils.getFutureResult(ProgressableUtils.java:99)
>      at
>
> org.apache.giraph.utils.ProgressableUtils.getResultsWithNCallables(ProgressableUtils.java:233)
>      at
>
> org.apache.giraph.worker.BspServiceWorker.loadInputSplits(BspServiceWorker.java:323)
>      at
>
> org.apache.giraph.worker.BspServiceWorker.loadVertices(BspServiceWorker.java:416)
>      at
> org.apache.giraph.worker.BspServiceWorker.setup(BspServiceWorker.java:636)
>      at
> org.apache.giraph.graph.GraphTaskManager.execute(GraphTaskManager.java:296)
>      at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:91)
>      at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)
>      at org.apache.hadoop.mapred.MapTask.run(MapTask.java:340)
>      at
>
> org.apache.hadoop.mapred.LocalJobRunner$Job$MapTaskRunnable.run(LocalJobRunner.java:243)
>      at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
>      at java.util.concurrent.FutureTask.run(FutureTask.java:262)
>      at
>
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>      at
>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>      at java.lang.Thread.run(Thread.java:745)
> Caused by: java.util.concurrent.ExecutionException:
> java.lang.IllegalStateException: newInstance: Illegal access
> org.apache.giraph.examples.TripletVertexWritable
>      at java.util.concurrent.FutureTask.report(FutureTask.java:122)
>      at java.util.concurrent.FutureTask.get(FutureTask.java:202)
>      at
>
> org.apache.giraph.utils.ProgressableUtils$FutureWaitable.waitFor(ProgressableUtils.java:312)
>      at
>
> org.apache.giraph.utils.ProgressableUtils.waitFor(ProgressableUtils.java:185)
>      ... 17 more
> Caused by: java.lang.IllegalStateException: newInstance: Illegal access
> org.apache.giraph.examples.TripletVertexWritable
>      at
>
> org.apache.giraph.utils.ReflectionUtils.newInstance(ReflectionUtils.java:86)
>      at
> org.apache.giraph.utils.WritableUtils.createWritable(WritableUtils.java:84)
>      at
>
> org.apache.giraph.factories.DefaultVertexValueFactory.newInstance(DefaultVertexValueFactory.java:47)
>      at
>
> org.apache.giraph.conf.ImmutableClassesGiraphConfiguration.createVertexValue(ImmutableClassesGiraphConfiguration.java:717)
>      at
>
> org.apache.giraph.utils.VertexIterator.resetEmptyVertex(VertexIterator.java:69)
>      at
> org.apache.giraph.utils.VertexIterator.<init>(VertexIterator.java:60)
>      at
>
> org.apache.giraph.comm.requests.SendWorkerVerticesRequest.doRequest(SendWorkerVerticesRequest.java:108)
>      at
>
> org.apache.giraph.comm.netty.NettyWorkerClientRequestProcessor.doRequest(NettyWorkerClientRequestProcessor.java:467)
>      at
>
> org.apache.giraph.comm.netty.NettyWorkerClientRequestProcessor.flush(NettyWorkerClientRequestProcessor.java:413)
>      at
>
> org.apache.giraph.worker.InputSplitsCallable.call(InputSplitsCallable.java:242)
>      at
>
> org.apache.giraph.worker.InputSplitsCallable.call(InputSplitsCallable.java:60)
>      at
>
> org.apache.giraph.utils.LogStacktraceCallable.call(LogStacktraceCallable.java:51)
>      ... 4 more
> Caused by: java.lang.IllegalAccessException: Class
> org.apache.giraph.utils.ReflectionUtils can not access a member of class
> org.apache.giraph.examples.TripletVertexWritable with modifiers "public"
>      at sun.reflect.Reflection.ensureMemberAccess(Reflection.java:109)
>      at java.lang.Class.newInstance(Class.java:373)
>      at
>
> org.apache.giraph.utils.ReflectionUtils.newInstance(ReflectionUtils.java:81)
>      ... 15 more
> 2015-06-03 00:36:20,000 ERROR [LocalJobRunner Map Task Executor #0]
> worker.BspServiceWorker (BspServiceWorker.java:unregisterHealth(784)) -
> unregisterHealth: Got failure, unregistering health on
>
> /_hadoopBsp/job_local869455398_0001/_applicationAttemptsDir/0/_superstepDir/-1/_workerHealthyDir/ubuntu_0
> on superstep -1
> 2015-06-03 00:36:20,011 INFO  [Thread-13] mapred.LocalJobRunner
> (LocalJobRunner.java:runTasks(456)) - map task executor complete.
> 2015-06-03 00:36:20,016 WARN  [Thread-13] mapred.LocalJobRunner
> (LocalJobRunner.java:run(560)) - job_local869455398_0001
> java.lang.Exception: java.lang.IllegalStateException: run: Caught an
> unrecoverable exception waitFor: ExecutionException occurred while
> waiting for
> org.apache.giraph.utils.ProgressableUtils$FutureWaitable@3c1de3e3
>      at
>
> org.apache.hadoop.mapred.LocalJobRunner$Job.runTasks(LocalJobRunner.java:462)
>      at
> org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:522)
> Caused by: java.lang.IllegalStateException: run: Caught an unrecoverable
> exception waitFor: ExecutionException occurred while waiting for
> org.apache.giraph.utils.ProgressableUtils$FutureWaitable@3c1de3e3
>      at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:102)
>      at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)
>      at org.apache.hadoop.mapred.MapTask.run(MapTask.java:340)
>      at
>
> org.apache.hadoop.mapred.LocalJobRunner$Job$MapTaskRunnable.run(LocalJobRunner.java:243)
>      at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
>      at java.util.concurrent.FutureTask.run(FutureTask.java:262)
>      at
>
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>      at
>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>      at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.IllegalStateException: waitFor: ExecutionException
> occurred while waiting for
> org.apache.giraph.utils.ProgressableUtils$FutureWaitable@3c1de3e3
>      at
>
> org.apache.giraph.utils.ProgressableUtils.waitFor(ProgressableUtils.java:193)
>      at
>
> org.apache.giraph.utils.ProgressableUtils.waitForever(ProgressableUtils.java:151)
>      at
>
> org.apache.giraph.utils.ProgressableUtils.waitForever(ProgressableUtils.java:136)
>      at
>
> org.apache.giraph.utils.ProgressableUtils.getFutureResult(ProgressableUtils.java:99)
>      at
>
> org.apache.giraph.utils.ProgressableUtils.getResultsWithNCallables(ProgressableUtils.java:233)
>      at
>
> org.apache.giraph.worker.BspServiceWorker.loadInputSplits(BspServiceWorker.java:323)
>      at
>
> org.apache.giraph.worker.BspServiceWorker.loadVertices(BspServiceWorker.java:416)
>      at
> org.apache.giraph.worker.BspServiceWorker.setup(BspServiceWorker.java:636)
>      at
> org.apache.giraph.graph.GraphTaskManager.execute(GraphTaskManager.java:296)
>      at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:91)
>      ... 8 more
> Caused by: java.util.concurrent.ExecutionException:
> java.lang.IllegalStateException: newInstance: Illegal access
> org.apache.giraph.examples.TripletVertexWritable
>      at java.util.concurrent.FutureTask.report(FutureTask.java:122)
>      at java.util.concurrent.FutureTask.get(FutureTask.java:202)
>      at
>
> org.apache.giraph.utils.ProgressableUtils$FutureWaitable.waitFor(ProgressableUtils.java:312)
>      at
>
> org.apache.giraph.utils.ProgressableUtils.waitFor(ProgressableUtils.java:185)
>      ... 17 more
> Caused by: java.lang.IllegalStateException: newInstance: Illegal access
> org.apache.giraph.examples.TripletVertexWritable
>      at
>
> org.apache.giraph.utils.ReflectionUtils.newInstance(ReflectionUtils.java:86)
>      at
> org.apache.giraph.utils.WritableUtils.createWritable(WritableUtils.java:84)
>      at
>
> org.apache.giraph.factories.DefaultVertexValueFactory.newInstance(DefaultVertexValueFactory.java:47)
>      at
>
> org.apache.giraph.conf.ImmutableClassesGiraphConfiguration.createVertexValue(ImmutableClassesGiraphConfiguration.java:717)
>      at
>
> org.apache.giraph.utils.VertexIterator.resetEmptyVertex(VertexIterator.java:69)
>      at
> org.apache.giraph.utils.VertexIterator.<init>(VertexIterator.java:60)
>      at
>
> org.apache.giraph.comm.requests.SendWorkerVerticesRequest.doRequest(SendWorkerVerticesRequest.java:108)
>      at
>
> org.apache.giraph.comm.netty.NettyWorkerClientRequestProcessor.doRequest(NettyWorkerClientRequestProcessor.java:467)
>      at
>
> org.apache.giraph.comm.netty.NettyWorkerClientRequestProcessor.flush(NettyWorkerClientRequestProcessor.java:413)
>      at
>
> org.apache.giraph.worker.InputSplitsCallable.call(InputSplitsCallable.java:242)
>      at
>
> org.apache.giraph.worker.InputSplitsCallable.call(InputSplitsCallable.java:60)
>      at
>
> org.apache.giraph.utils.LogStacktraceCallable.call(LogStacktraceCallable.java:51)
>      ... 4 more
> Caused by: java.lang.IllegalAccessException: Class
> org.apache.giraph.utils.ReflectionUtils can not access a member of class
> org.apache.giraph.examples.TripletVertexWritable with modifiers "public"
>      at sun.reflect.Reflection.ensureMemberAccess(Reflection.java:109)
>      at java.lang.Class.newInstance(Class.java:373)
>      at
>
> org.apache.giraph.utils.ReflectionUtils.newInstance(ReflectionUtils.java:81)
>      ... 15 more
>
> Thanks for your assistance, and I wait for your comments.
>
> --
> Cristian Vidal Silva
> Computer Engineer
> Master in Computer Science, University of Concepcion - Chile
> Master of Science in Computer Science, Michigan State University, MI - USA
> Ph.D(c) in Software Engineering and Technology, University of Seville -
> Spain
>
> Chilean Fulbright Scholar
>
>
>
>
>

Mime
View raw message