giraph-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Puneet Agarwal <puagar...@yahoo.com>
Subject Re: GiraphJob Vs InternalVertexRunner
Date Fri, 07 Jun 2013 04:35:28 GMT
Dear Yazan,

The problem was that I was using Hadoop in Standalone mode.


Then configured Hadoop in Pseudo Distributed more / Single Node Set-up, and it worked without
any problem.

Regards
Puneet


----- Original Message -----
From: Yazan Boshmaf <boshmaf@ece.ubc.ca>
To: user@giraph.apache.org; Puneet Agarwal <puagarwal@yahoo.com>
Cc: 
Sent: Sunday, June 2, 2013 8:24 AM
Subject: Re: GiraphJob Vs InternalVertexRunner

Puneet, concerning the examples, recently there was a new diff to
decouple vertex data and computation which was a significant change.
Check issue GIRAPH-667 for more info
(https://issues.apache.org/jira/browse/GIRAPH-667).

On Sat, Jun 1, 2013 at 7:50 PM, Yazan Boshmaf <boshmaf@ece.ubc.ca> wrote:
> I think the SimpleShortestPathsVertex.java class got renamed to
> SimpleShortestPathsComputation.java, as can be seen in trunk (check
> here: https://github.com/apache/giraph/tree/trunk/giraph-examples/src/main/java/org/apache/giraph/examples).
>
> Concerning the issue you stated, you seem to have run Giraph using a
> local configuration and as far as I know, the LocalJobRunner has some
> restrictions (only one task at a time, and thus no split master/worker
> mode). What's your Hadoop setup? Are you running a local
> pseudo-distributed Hadoop instance? If so, have you updated
> /directory-to-hadoop/conf/mapred-site.xml with:
>
> <property>
>   <name>mapred.tasktracker.map.tasks.maximum</name>
>   <value>4</value>
> </property>
>
> <property>
>   <name>mapred.map.tasks</name>
>   <value>4</value>
> </property>
>
> Please give it a shot and let me know.
>
> Best,
> Yazan
>
> On Sat, Jun 1, 2013 at 6:03 AM, Puneet Agarwal <puagarwal@yahoo.com> wrote:
>> Hi Yazan,
>>
>> This indeed if of great help especially the help command:
>>
>> "/directory-to-hadoop/bin/hadoop jar/directory-to-giraph/giraph-examples/target/giraph-examples-1.1.0-SNAPSHOT-for-hadoop-0.20.203.0-jar-with-dependencies.jar
>> org.apache.giraph.GiraphRunner -h"
>>
>> But there seems to be some problem in your command because there is no class named
"org.apache.giraph.examples.SimpleShortestPathsComputation"
>>
>> I then looked up in the path "org/apache/giraph/examples/" and found that there is
a class named "org.apache.giraph.examples.SimpleShortestPathsVertex" so I tried this instead
of yours. But this does not work
>>
>> It gives following error. Any guidance will be helpful
>>
>> 13/06/01 18:28:22 INFO utils.ConfigurationUtils: No edge input format specified.
Ensure your InputFormat does not require one.
>> 13/06/01 18:28:22 WARN job.GiraphConfigurationValidator: Output format vertex index
type is not known
>> 13/06/01 18:28:22 WARN job.GiraphConfigurationValidator: Output format vertex value
type is not known
>> 13/06/01 18:28:22 WARN job.GiraphConfigurationValidator: Output format edge value
type is not known
>> 13/06/01 18:28:22 INFO job.GiraphJob: run: Since checkpointing is disabled (default),
do not allow any task retries (setting mapred.map.max.attempts = 0, old value = 4)
>> Exception in thread "main" java.lang.IllegalArgumentException: checkLocalJobRunnerConfiguration:
When using LocalJobRunner, you cannot run in split master / worker mode since there is only
1 task at a time!
>> at org.apache.giraph.job.GiraphJob.checkLocalJobRunnerConfiguration(GiraphJob.java:157)
>> at org.apache.giraph.job.GiraphJob.run(GiraphJob.java:225)
>> at org.apache.giraph.GiraphRunner.run(GiraphRunner.java:94)
>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
>> at org.apache.giraph.GiraphRunner.main(GiraphRunner.java:124)
>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>> at java.lang.reflect.Method.invoke(Method.java:616)
>> at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
>>
>>
>>
>>
>> Regards
>>
>> Puneet
>>
>>
>>
>> ----- Original Message -----
>> From: Yazan Boshmaf <boshmaf@ece.ubc.ca>
>> To: user@giraph.apache.org; Puneet Agarwal <puagarwal@yahoo.com>
>> Cc:
>> Sent: Saturday, June 1, 2013 7:59 AM
>> Subject: Re: GiraphJob Vs InternalVertexRunner
>>
>> After packaging Giraph (i.e., you can locate the JAR files under
>> "target" folder in each module), you can run one of the included
>> examples under /directory-to-giraph/giraph-examples as follows:
>>
>> /directory-to-hadoop/bin/hadoop jar
>> /directory-to-giraph/giraph-examples/target/giraph-examples-1.1.0-SNAPSHOT-for-hadoop-0.20.203.0-jar-with-dependencies.jar
>> org.apache.giraph.GiraphRunner
>> org.apache.giraph.examples.SimpleShortestPathsComputation -vif
>> org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputFormat
>> -vip /dfs-user-direcory/some-input-json -of
>> org.apache.giraph.io.formats.IdWithValueTextOutputFormat -op
>> /dfs-user-direcory/some-output-text -w 1
>>
>> This runs the SimpleShortestPathsComputation using the input file
>> /dfs-user-direcory/some-input-json, which has the format [source_id,
>> source_value, [ [dest_id, edge_weight], ...] ]. It comutes the
>> shortest paths to all nodes from a given source, which is the first
>> source_id in the input file. The output file is
>> /dfs-user-direcory/some-output-text, and its format is "source_id
>> distance_value". The computation is done using one worker.
>>
>> You can run he following for more info:
>>
>> /directory-to-hadoop/bin/hadoop jar
>> /directory-to-giraph/giraph-examples/target/giraph-examples-1.1.0-SNAPSHOT-for-hadoop-0.20.203.0-jar-with-dependencies.jar
>> org.apache.giraph.GiraphRunner -h
>>
>> Hope this helps.
>>
>> Cheers,
>> Yazan
>>
>> On Fri, May 31, 2013 at 6:07 PM, Puneet Agarwal <puagarwal@yahoo.com> wrote:
>>> It seems there are two ways to run a Giraph job.
>>>
>>> a) using the class InternalVertexRunner
>>> b) using the class GiraphJob
>>>
>>> Which one should be used where ?
>>>
>>> Thanks
>>> Puneet
>>


Mime
View raw message