giraph-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jyoti Yadav <rao.jyoti26ya...@gmail.com>
Subject Re: about retaining the variable value...
Date Mon, 18 Nov 2013 11:58:59 GMT
Hi folks..

To give the vertex value as (double,double), I created my own class
MyVertexValue.java and to give vertex input format,i also created
MyVertexInputFormat.java...Program is compiling successfully...
But while running, it is showing some strange behaviour....

Below is the command given...


hadoop jar
$GIRAPH_HOME/giraph-examples/target/giraph-examples-1.1.0-SNAPSHOT-for-hadoop-0.20.203.0-jar-with-dependencies.jar
org.apache.giraph.GiraphRunner org.apache.giraph.examples.LineRank -vif
org.apache.giraph.examples.io.formats.MyVertexInputFormat -vip
/user/hduser/sp_input/linerank_input1.txt -vof
org.apache.giraph.io.formats.IdWithValueTextOutputFormat -op
/user/hduser/sp_output70/linerank -w 1


output shown is ....


13/11/18 16:59:25 INFO utils.ConfigurationUtils: No edge input format
specified. Ensure your InputFormat does not require one.
13/11/18 16:59:25 INFO utils.ConfigurationUtils: No edge output format
specified. Ensure your OutputFormat does not require one.
13/11/18 16:59:26 INFO job.GiraphJob: run: Since checkpointing is disabled
(default), do not allow any task retries (setting mapred.map.max.attempts =
0, old value = 4)
13/11/18 16:59:28 INFO job.GiraphJob: run: Tracking URL:
http://localhost:50030/jobdetails.jsp?jobid=job_201311181111_0023
13/11/18 16:59:58 INFO
job.HaltApplicationUtils$DefaultHaltInstructionsWriter:
writeHaltInstructions: To halt after next superstep execute:
'bin/halt-application --zkServer kanha-Vostro-1014:22181 --zkNode
/_hadoopBsp/job_201311181111_0023/_haltComputation'
13/11/18 16:59:58 INFO mapred.JobClient: Running job: job_201311181111_0023
13/11/18 16:59:59 INFO mapred.JobClient:  map 100% reduce 0%
13/11/18 17:00:50 INFO mapred.JobClient:  map 50% reduce 0%
13/11/18 17:11:24 INFO mapred.JobClient:  map 0% reduce 0%
13/11/18 17:11:29 INFO mapred.JobClient: Job complete: job_201311181111_0023
13/11/18 17:11:29 INFO mapred.JobClient: Counters: 6
13/11/18 17:11:29 INFO mapred.JobClient:   Job Counters
13/11/18 17:11:29 INFO mapred.JobClient:     SLOTS_MILLIS_MAPS=707902
13/11/18 17:11:29 INFO mapred.JobClient:     Total time spent by all
reduces waiting after reserving slots (ms)=0
13/11/18 17:11:29 INFO mapred.JobClient:     Total time spent by all maps
waiting after reserving slots (ms)=0
13/11/18 17:11:29 INFO mapred.JobClient:     Launched map tasks=2
13/11/18 17:11:29 INFO mapred.JobClient:     SLOTS_MILLIS_REDUCES=0
13/11/18 17:11:29 INFO mapred.JobClient:     Failed map tasks=1


Any ideas related to this bug is really appreciated......

Thanks...





On Mon, Nov 18, 2013 at 2:23 AM, <betim.beja@studenti.unipg.it> wrote:

>  I think  you have to implement your own input format similar to the
>
> org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputFormat
>
> where the vertex value Type is MyVertexValueWritable
>
> and the input graph should be like:
>
> [0,0,0,[[1,1],[3,3]]][1,0,0,[[0,1],[2,2],[3,1]]][2,0,0,[[1,2],[4,4]]][3,0,0,[[0,3],[1,1],[4,4]]][4,0,0,[[3,4],[2,4]]]
>
> like [ID,Double,Double,[..Edges..]] where Double,Double is the
> MyVertexValueWritable serialization…
>
> Best Regards!
> Betim!
>
>
>
> *Da:* Jyoti Yadav <rao.jyoti26yadav@gmail.com>
> *Data invio:* domenica 17 novembre 2013 17:22
> *A:* user@giraph.apache.org
>
> Thanks all for your ideas...
>
> I have one further query..
> Scenario...
>
> Vertex_id-->LongWritable
> Vertex_value-->MyVertexValueWritable( it consists of composite value-->
> double,double )
>  Edge_value->FloatWritable
>
>
> Input given to giraph is in following format...
>
> [0,0,[[1,1],[3,3]]][1,0,[[0,1],[2,2],[3,1]]][2,0,[[1,2],[4,4]]][3,0,[[0,3],[1,1],[4,4]]][4,0,[[3,4],[2,4]]]
>
> What vertex input format should i give to execute the program???
>
> Any Ideas??
>
> Thanks
>
>
>
>
>
>
>
>
>
>
>
> On Sun, Nov 17, 2013 at 5:12 PM, <ikapoura@csd.auth.gr> wrote:
>
>> You can also set a variable outside of the compute function, at the
>> private part of the class. You will have to instantiate it at superstep 0
>> though, else you will have a NullPointerException thrown, in case you use
>> it at superstep 1.
>>
>>
>>
>>
>> Quoting Jyoti Yadav <rao.jyoti26yadav@gmail.com>:
>>
>>  Thanks Ameya...I will give a try to your ideas...
>>>
>>>
>>> On Sun, Nov 17, 2013 at 3:17 PM, Ameya Vilankar <
>>> ameya.vilankar@gmail.com>wrote:
>>>
>>>  In your scenario, you want to store some value that is obtained in
>>>> Superstep 1, so that it is available to a vertex in all the subsequent
>>>> supersteps. The best place to do that is to store it on the vertex
>>>> itself.
>>>> The simplest example is ShortesPath Example where you store the current
>>>> lowest distance on the vertex, and when you receive a message you
>>>> update it
>>>> with the new lowest mistake. I assume that in your program you don't
>>>> want
>>>> to update the value after Superstep 1. So don't. Just keep an extra
>>>> variable in your vertex value class and set it in superstep 1 so that it
>>>> can be used by the vertex in all the following steps.
>>>>
>>>> But if you want to store global information i.e in superstep 1, every
>>>> vertex computes and stores a value and in the subsequent supersteps, a
>>>> vertex needs access to the stored values of all the vertices, you need
>>>> to
>>>> make those values global. The only way to do it is to use a persistent
>>>> aggregator. In superstep 1, every vertex computes the value and then
>>>> sends
>>>> it to the persistent aggregator that can contain a hashmap from the
>>>> vertex
>>>> id to the value. This value will persist across all the supersteps since
>>>> the aggregator is a persistent one. Will be available to every vertex
>>>> too.
>>>>
>>>>
>>>> On Sun, Nov 17, 2013 at 1:11 AM, Mirko Kämpf <mirko.kaempf@cloudera.com
>>>> >wrote:
>>>>
>>>>  Hi,
>>>>>
>>>>> if it is a local value, just important for that given vertex, it might
>>>>> work with a local variable, maybe even a more complex data structure,
>>>>> e.g.
>>>>> a Stack or a FIFO buffer, to handle "old" data. I do not consider to
>>>>> exchange data between vertexes, only within one vertex over time.
>>>>>
>>>>> I write this here, to share my idea, but I am not really sure if a
>>>>> Giraph
>>>>> vertex works like this. It's just how I would try it ...
>>>>>
>>>>> But I think we will see a definitive answer soon.
>>>>>
>>>>> Have a nice day!
>>>>> Mirko
>>>>>
>>>>>
>>>>> On Sun, Nov 17, 2013 at 6:55 AM, Jyoti Yadav <
>>>>> rao.jyoti26yadav@gmail.com>wrote:
>>>>>
>>>>>  Hi folks...
>>>>>>
>>>>>> while executing my program ,i came across a doubt which is creating
>>>>>> problem...
>>>>>>
>>>>>> Scenario....
>>>>>>
>>>>>> In compute() function,
>>>>>> Suppose in superstep1 each vertex want to save some value so that
>>>>>> every
>>>>>> vertex can use its  previously saved value in upcoming
>>>>>> supersteps...Is it
>>>>>> possible???
>>>>>>
>>>>>>
>>>>>> Any ideas are really appreciated...
>>>>>>
>>>>>> Thanks...
>>>>>> Jyoti
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>
>>
>>
>>
>

Mime
View raw message