hama-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Edward J. Yoon" <edwardy...@apache.org>
Subject Re: I plan on cutting a release this week for Hama 0.6.1
Date Thu, 28 Mar 2013 10:35:37 GMT
> I remember we had some benchmarks on executing some jobs on clusters, maybe
> I'm wrong.

I’m ashamed to say that MR-based generator programs (for SSSP,
PageRank, KMeans examples) was used to avoid partitioning/memory
issues[1].

1. https://issues.apache.org/jira/browse/HAMA-531

On Thu, Mar 28, 2013 at 7:30 PM, Tommaso Teofili
<tommaso.teofili@gmail.com> wrote:
> 2013/3/28 Edward J. Yoon <edwardyoon@apache.org>
>
>> > Can you elaborate some more on this? Why do you think that's needed? Is
>> > that related to MRQL?
>>
>> It's not related to MRQL. The problem is that every Hama jobs, which
>> requires partitioning, doesn't work for only few ten gigabytes. As we
>> already know, input partitioning is very important for distributed
>> matrix or graph computations.
>>
>> Moreover, we didn't provide any random generator for examples. (I
>> guess) newbies' behavior (before considering adopt Hama) pattern is:
>>
>> 1. Download, a latest Hama release.
>> 2. Setup, a Hama Cluster.
>> 3. Evaluation, using examples.
>>
>> Except Pi and Bench examples, they need to study/do a lot of things.
>>
>
> ok great Edward, now it's much clearer :-)
> thanks for explaining it further.
>
>
>>
>> > I'm not sure I understand sorry :) what do you mean here exactly?
>>
>> I mean, Hama is a large scale BSP computing platform but didn't tested
>> on large cluster yet.
>>
>
> I remember we had some benchmarks on executing some jobs on clusters, maybe
> I'm wrong.
>
> Tommaso
>
>
>>
>> On Thu, Mar 28, 2013 at 5:30 PM, Tommaso Teofili
>> <tommaso.teofili@gmail.com> wrote:
>> > 2013/3/28 Edward J. Yoon <edwardyoon@apache.org>
>> >
>> >> > Sure, can we ensure the new downstream application MRQL is fine with
>> the
>> >> > changes in 0.6.1?
>> >>
>> >> Maybe, at least, I can test large input with this :)
>> >>
>> >> I can understand your concerns but, a proof of concept (on semi-large
>> >> cluster) is needed (ASAP).
>> >
>> >
>> > Can you elaborate some more on this? Why do you think that's needed? Is
>> > that related to MRQL?
>> >
>> >
>> >> IMO, there is the difference between
>> >> (somewhat) proven status and (work in progress) conceptual status.
>> >>
>> >
>> > I'm not sure I understand sorry :) what do you mean here exactly?
>> >
>> > Thanks,
>> > Tommaso
>> >
>> >
>> >>
>> >> On Thu, Mar 28, 2013 at 4:14 PM, Suraj Menon <surajsmenon@apache.org>
>> >> wrote:
>> >> > Sure, can we ensure the new downstream application MRQL is fine with
>> the
>> >> > changes in 0.6.1?
>> >> >
>> >> > -Suraj
>> >> >
>> >> > On Thu, Mar 28, 2013 at 2:57 AM, Tommaso Teofili
>> >> > <tommaso.teofili@gmail.com>wrote:
>> >> >
>> >> >> sounds good to me
>> >> >> Tommaso
>> >> >>
>> >> >>
>> >> >> 2013/3/28 Edward J. Yoon <edwardyoon@apache.org>
>> >> >>
>> >> >> > Hi all,
>> >> >> >
>> >> >> > I plan on cutting a release this week for Hama 0.6.1. We fixed
>> input
>> >> >> > splitter and made few improvements.
>> >> >> >
>> >> >> > Thanks.
>> >> >> > --
>> >> >> > Best Regards, Edward J. Yoon
>> >> >> > @eddieyoon
>> >> >> >
>> >> >>
>> >>
>> >>
>> >>
>> >> --
>> >> Best Regards, Edward J. Yoon
>> >> @eddieyoon
>> >>
>>
>>
>>
>> --
>> Best Regards, Edward J. Yoon
>> @eddieyoon
>>



-- 
Best Regards, Edward J. Yoon
@eddieyoon

Mime
View raw message