giraph-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Avery Ching <ach...@apache.org>
Subject Re: List of Algos implemented on Giraph
Date Wed, 28 Sep 2011 00:10:55 GMT
Thanks for your help Aapo.  Definitely fee free to open/tackle issues on 
the JIRA as you find them.  =)

Avery

On 9/27/11 5:05 PM, Aapo Kyrola wrote:
>
> Glad to hear about the progress. I am probably going to work on my algo
> soon and try to run it with very big graphs. I plan to do some 
> profiling and
> experimental modifications to Giraph code as well to get it to scale. 
> (Don't
> worry, I am not committing anything :)). Will keep you updated.
>
>
> On Sep 27, 2011, at 4:33 PM, Jake Mannix wrote:
>
>>
>>
>> On Tue, Sep 27, 2011 at 1:29 PM, Aapo Kyrola <akyrola@cs.cmu.edu 
>> <mailto:akyrola@cs.cmu.edu>> wrote:
>>
>>
>>
>>     The code is still in draft stage, but I attached it.
>>
>>
>> Cool, thanks.  The best place to usually attach it is to a JIRA 
>> ticket which describes what
>> the code does, usually.
>>
>>     it is actually quite optimized code with regards to data
>>     serialization etc.
>>
>>     It is just that Giraph currently takes a lot of memory (I guess
>>     it is the RPC), which makes it difficult
>>     to run algos like this that cannot use a combiner.
>>
>>
>> https://issues.apache.org/jira/browse/GIRAPH-28
>>
>> is tracking some work toward reducing the memory footprint, but it 
>> requires some work on
>>
>> https://issues.apache.org/jira/browse/GIRAPH-36
>>
>> before it'll work as well as it can.  Similarly, the RPC (including 
>> memory overhead) is being improved in explorations in
>>
>> https://issues.apache.org/jira/browse/GIRAPH-12
>>
>> and
>>
>> https://issues.apache.org/jira/browse/GIRAPH-37
>>
>> Watch those spaces for upcoming improvements!
>>
>>   -jake
>>
>>
>>     Aapo
>>
>>     On Sep 27, 2011, at 4:14 PM, Jake Mannix wrote:
>>
>>>
>>>
>>>     On Tue, Sep 27, 2011 at 12:31 PM, Aapo Kyrola
>>>     <akyrola@cs.cmu.edu <mailto:akyrola@cs.cmu.edu>> wrote:
>>>
>>>
>>>         I have written a very simple Belief Propagation algorithm
>>>         for binary
>>>         variables.
>>>
>>>         Not ready for prime-time either, though :).
>>>
>>>
>>>     That's awesome!   Where's the code? :)
>>>
>>>     It shouldn't be construed that "not ready for prime-time" is
>>>     *bad*, in case that's what it looked like I was saying.
>>>
>>>     More examples the better, so we can see where the bottlenecks
>>>     are, and move them toward productionalized stage!
>>>       -jake
>>>
>>>
>>>         Aapo
>>>
>>>         On Sep 27, 2011, at 3:03 PM, Jake Mannix wrote:
>>>
>>>>         Not really.  It's really really early, and they're in the
>>>>         "examples" stage - nothing is
>>>>         really productionized.  There's things like PageRank,
>>>>         finding shortest path, but
>>>>         nothing is really ready for prime time yet.
>>>>
>>>>         On Tue, Sep 27, 2011 at 11:56 AM, Josh Patterson
>>>>         <josh@cloudera.com <mailto:josh@cloudera.com>> wrote:
>>>>
>>>>             Is there a list of known algorithms that have been
>>>>             implemented on the
>>>>             Giraph framework?
>>>>
>>>>             JP
>>>>
>>>>             --
>>>>             Twitter: @jpatanooga
>>>>             Solution Architect @ Cloudera
>>>>             hadoop: http://www.cloudera.com <http://www.cloudera.com/>
>>>>
>>>>
>>>
>>>         Aapo Kyrola
>>>         Ph.D. student, http://www.cs.cmu.edu/~akyrola
>>>         <http://www.cs.cmu.edu/%7Eakyrola>
>>>
>>>
>>
>>     Aapo Kyrola
>>     Ph.D. student, http://www.cs.cmu.edu/~akyrola
>>     <http://www.cs.cmu.edu/%7Eakyrola>
>>
>>
>>
>
> Aapo Kyrola
> Ph.D. student, http://www.cs.cmu.edu/~akyrola 
> <http://www.cs.cmu.edu/%7Eakyrola>
>


Mime
View raw message