giraph-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andro Bermúdez Serrano <andro.bermu...@gmail.com>
Subject Re: Running the example in http://giraph.apache.org/quick_start.html
Date Fri, 11 Oct 2013 13:14:02 GMT
Thanks, it works perfectly. I added:
-mc org.apache.giraph.examples.SimplePageRankVertex\$SimplePageRankVertexMasterCompute

Is there a tutorial that explains GiraphRunner parameters and how to
customize them?
(i.e., -vif, -ve, -mc, -eif, -aw, etc., and all needed classes)

2013/10/9, Wei Zhang <weiz@us.ibm.com>:
>
> This is how I used the mc option that I used, not sure if it helps.
>
> -mc org.apache.giraph.examples.SimplePageRankComputation
> \$SimplePageRankMasterCompute
>
> Wei
>
>
>
> From:	Claudio Martella <claudio.martella@gmail.com>
> To:	"user@giraph.apache.org" <user@giraph.apache.org>,
> Date:	10/09/2013 05:00 PM
> Subject:	Re: Running the example in
>             http://giraph.apache.org/quick_start.html
>
>
>
> you need to define the mastercompute class (-mc option) which for your
> example is that SimplePageRankVertexMasterCompute class within that same
> vertex class.
>
>
> On Wed, Oct 9, 2013 at 10:44 PM, Andro Bermúdez Serrano <
> andro.bermudez@gmail.com> wrote:
>   Maybe the problem is with agregators. This is one of the exception throw
>
>   Caused by: java.lang.IllegalStateException: aggregate: Tried to access
>   aggregator which wasn't registered max; Aggregators can be registered
>   in MasterCompute.initialize by calling
>   registerAggregator(aggregatorName, aggregatorClass). Also be sure that
>   you are correctly setting MasterCompute class, currently using
>   org.apache.giraph.master.DefaultMasterCompute
>           at org.apache.giraph.worker.WorkerAggregatorHandler.aggregate
>   (WorkerAggregatorHandler.java:103)
>           at org.apache.giraph.graph.Vertex.aggregate(Vertex.java:502)
>           at org.apache.giraph.examples.SimplePageRankVertex.compute
>   (SimplePageRankVertex.java:74)
>           at org.apache.giraph.graph.ComputeCallable.computePartition
>   (ComputeCallable.java:234)
>           at org.apache.giraph.graph.ComputeCallable.call
>   (ComputeCallable.java:162)
>           at org.apache.giraph.graph.ComputeCallable.call
>   (ComputeCallable.java:70)
>           at org.apache.giraph.utils.LogStacktraceCallable.call
>   (LogStacktraceCallable.java:51)
>           at java.util.concurrent.FutureTask$Sync.innerRun
>   (FutureTask.java:334)
>           at java.util.concurrent.FutureTask.run(FutureTask.java:166)
>           at java.util.concurrent.ThreadPoolExecutor.runWorker
>   (ThreadPoolExecutor.java:1145)
>           at java.util.concurrent.ThreadPoolExecutor$Worker.run
>   (ThreadPoolExecutor.java:615)
>           at java.lang.Thread.run(Thread.java:724)
>
>   2013/10/9, Silvio Di gregorio <silvio.digregorio@gmail.com>:
>   > There must be another error . See all file. Looking for ERROR no WARN.
>   > Il giorno 09/ott/2013 22:06, "Andro Bermúdez Serrano" <
>   > andro.bermudez@gmail.com> ha scritto:
>   >
>   >> there are two failed map:
>   >> java.lang.Throwable: Child Error at
>   >> org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:271) Caused
>   >> by: java.io.IOException: Task process exit with nonzero status of 1.
>   >> at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:258)
>   >>
>   >> and
>   >>
>   >> java.lang.IllegalStateException: run: Caught an unrecoverable
>   >> exception waitFor: ExecutionException occurred while waiting for
>   >> org.apache.giraph.utils.ProgressableUtils$FutureWaitable@6ba789fd at
>   >> org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:102)
>   >> (.......)
>   >>
>   >>
>   >>
>   >>
>   >>
>   >> 2013/10/9, Silvio Di gregorio <silvio.digregorio@gmail.com>:
>   >> > Try TO the jobtracker UI.
>   >> > http://nameofthemaster:50030/.
>   >> > You can find the job band a detailed log ...
>   >> > Il giorno 09/ott/2013 21:18, "Silvio Di gregorio" <
>   >> > silvio.digregorio@gmail.com> ha scritto:
>   >> >
>   >> >> Can you see the log of the job? Try TO /var/log or find w/ the job
>   >> >> name,
>   >> >> job_date_002.
>   >> >> Il giorno 09/ott/2013 21:02, "Wei Zhang" <weiz@us.ibm.com>
ha
>   scritto:
>   >> >>
>   >> >>>  Hi Andro,
>   >> >>>
>   >> >>> Not sure if it helps -- maybe you can try to change the "-of
" to
>   >> >>> "-vof
>   >> >>> " on your command line. (i.e, the option flag in front of your
>   >> >>> org.apache.giraph.io.formats.IdWithValueTextOutputFormat).
>   >> >>>
>   >> >>> Wei
>   >> >>>
>   >> >>> [image: Inactive hide details for Andro Bermúdez Serrano
>   >> >>> ---10/09/2013
>   >> >>> 02:45:02 PM---I am new with Giraph and I am willing to using
>   it]Andro
>   >> >>> Bermúdez Serrano ---10/09/2013 02:45:02 PM---I am new with Giraph
>   and
>   >> >>> I
>   >> >>> am
>   >> >>> willing to using it in my project. I am using hadoop 1.2.1.
>   >> >>>
>   >> >>> From: Andro Bermúdez Serrano <andro.bermudez@gmail.com>
>   >> >>> To: user@giraph.apache.org,
>   >> >>> Date: 10/09/2013 02:45 PM
>   >> >>> Subject: Running the example in
>   >> >>> http://giraph.apache.org/quick_start.html
>   >> >>> ------------------------------
>   >> >>>
>   >> >>>
>   >> >>>
>   >> >>> I am new with Giraph and I am willing to using it in my project.
I
>   am
>   >> >>> using hadoop 1.2.1.
>   >> >>> I builded giraph with mvn package -Phadoop_1.0
>   >> >>> -Dhadoop.version=1.2.1 -DskipTests --offline. I ran the
>   >> >>> SimpleShortestPathsVertex example and all OK. With the same input
>   I
>   >> >>> tried the SimplePageRankVertex example and failed:
>   >> >>>
>   >> >>> This is the command I used:
>   >> >>> $HADOOP_HOME/bin/hadoop jar
>   >> >>>
>   >> >>>
>   >>
>
> $GIRAPH_HOME/giraph-examples/target/giraph-examples-1.0.0-for-hadoop-1.2.1-jar-with-dependencies.jar
>
>   >> >>> org.apache.giraph.GiraphRunner
>   >> >>> org.apache.giraph.examples.SimplePageRankVertex -vif
>   >> >>>
>   org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputFormat
>   >> >>> -vip /user/andro/input/tiny_graph.txt -of
>   >> >>> org.apache.giraph.io.formats.IdWithValueTextOutputFormat -op
>   >> >>> /user/andro/output/pagerank -w 1
>   >> >>>
>   >> >>> This is the output:
>   >> >>> 13/10/09 14:21:36 INFO utils.ConfigurationUtils: No edge input
>   format
>   >> >>> specified. Ensure your InputFormat does not require one.
>   >> >>> 13/10/09 14:21:36 WARN job.GiraphConfigurationValidator: Output
>   >> >>> format
>   >> >>> vertex index type is not known
>   >> >>> 13/10/09 14:21:36 WARN job.GiraphConfigurationValidator: Output
>   >> >>> format
>   >> >>> vertex value type is not known
>   >> >>> 13/10/09 14:21:36 WARN job.GiraphConfigurationValidator: Output
>   >> >>> format
>   >> >>> edge value type is not known
>   >> >>> 13/10/09 14:21:36 INFO job.GiraphJob: run: Since checkpointing
is
>   >> >>> disabled (default), do not allow any task retries (setting
>   >> >>> mapred.map.max.attempts = 0, old value = 4)
>   >> >>> 13/10/09 14:21:37 INFO mapred.JobClient: Running job:
>   >> >>> job_201310091401_0002
>   >> >>> 13/10/09 14:21:38 INFO mapred.JobClient:  map 0% reduce 0%
>   >> >>> 13/10/09 14:21:52 INFO mapred.JobClient:  map 50% reduce 0%
>   >> >>> 13/10/09 14:21:58 INFO mapred.JobClient:  map 100% reduce 0%
>   >> >>> 13/10/09 14:21:59 INFO mapred.JobClient:  map 50% reduce 0%
>   >> >>> 13/10/09 14:32:01 INFO mapred.JobClient:  map 0% reduce 0%
>   >> >>> 13/10/09 14:32:02 INFO mapred.JobClient: Job complete:
>   >> >>> job_201310091401_0002
>   >> >>> 13/10/09 14:32:02 INFO mapred.JobClient: Counters: 6
>   >> >>> 13/10/09 14:32:02 INFO mapred.JobClient:   Job Counters
>   >> >>> 13/10/09 14:32:02 INFO mapred.JobClient:
>   SLOTS_MILLIS_MAPS=622821
>   >> >>> 13/10/09 14:32:02 INFO mapred.JobClient:     Total time spent
by
>   all
>   >> >>> reduces waiting after reserving slots (ms)=0
>   >> >>> 13/10/09 14:32:02 INFO mapred.JobClient:     Total time spent
by
>   all
>   >> >>> maps waiting after reserving slots (ms)=0
>   >> >>> 13/10/09 14:32:02 INFO mapred.JobClient:     Launched map tasks=2
>   >> >>> 13/10/09 14:32:02 INFO mapred.JobClient:
>   SLOTS_MILLIS_REDUCES=0
>   >> >>> 13/10/09 14:32:02 INFO mapred.JobClient:     Failed map tasks=1
>   >> >>>
>   >> >>>
>   >> >>> I appreciate any help. Maybe I did it wrong.
>   >> >>> Andro.
>   >> >>>
>   >> >>>
>   >> >>>
>   >> >
>   >>
>   >
>
>
>
> --
>    Claudio Martella
>    claudio.martella@gmail.com

Mime
View raw message