hama-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Edward J. Yoon" <edwardy...@apache.org>
Subject Re: Roadmap and Release Manager
Date Thu, 09 Aug 2012 11:44:27 GMT
>> - Data repartitioning
>> - Hadoop 2.0 compatibility

P.S., My summer vacation will start from Aug 11th, so I need more time.

On Mon, Jul 9, 2012 at 10:49 PM, Tommaso Teofili
<tommaso.teofili@gmail.com> wrote:
> 2012/7/9 Edward J. Yoon <edwardyoon@apache.org>
>
>> Then, let's try to release 0.6 by end of August or September 2012. I
>> heard/saw some CDH4 plans/activities, we should not hesitate. :)
>>
>> Our TODO list:
>>
>> >> - Fault tolerance
>> >> - Data repartitioning
>> >> - Hadoop 2.0 compatibility
>> >> - Hama documentation book in html
>> >> - Bug fixes if any reported and few code improvements.
>> > adding the streaming API.
>>
>
> +1 for this roadmap.
> I'd like to put some machine learning algorithm in there but I'm not sure
> about the timing for it.
> Regards,
> Tommaso
>
>
>>
>> I'll work on yarn module and (large input data) re-partitioning issue.
>>
>> If you need my help such as test on large cluster, Just let me know.
>>
>> On Mon, Jul 9, 2012 at 9:37 PM, Thomas Jungblut
>> <thomas.jungblut@gmail.com> wrote:
>> > Suraj's plan would also be my favorite.
>> > We also could split two faster releases from fault tolerance and Hadoop
>> 2.0.
>> >
>> > If we have sorted out the fault tolerance I would like to spend my time
>> on
>> > adding the streaming API.
>> >
>> > 2012/7/9 Suraj Menon <surajsmenon@apache.org>
>> >
>> >> Sorry, I was looking at more time (~6 months) for the next release. This
>> >> implies that are we are looking for a release in August end. The desired
>> >> feature set:
>> >>
>> >> - Fault tolerance
>> >> - Data repartitioning
>> >> - Hadoop 2.0 compatibility
>> >> - Hama documentation book in html
>> >> - Bug fixes if any reported and few code improvements.
>> >>
>> >> I have to work on my Jenkins permissions. The last time I tried, it did
>> not
>> >> work for me.
>> >>
>> >> -Suraj
>> >>
>> >> On Mon, Jul 9, 2012 at 6:02 AM, Edward J. Yoon <edward.yoon@oracle.com
>> >> >wrote:
>> >>
>> >> > We only need to discuss about 0.6 release roadmap. Any committer may
>> >> > create a release candidate. Please feel free to create your own RC
and
>> >> open
>> >> > the vote in future.
>> >> >
>> >> > Personally, I would like to release along with support for Hadoop 2.0,
>> >> > ASAP. Hope I can finish by this end of month or next month.
>> >> >
>> >> > And IMO, the current high-priority tasks are Input re-partitioning
and
>> >> > exception handling.
>> >> >
>> >> > Thanks,
>> >> > Ed
>> >> >
>> >> >
>> >> > On 7/9/2012 5:15 PM, Suraj Menon wrote:
>> >> >
>> >> >> Hello,
>> >> >>
>> >> >> Fault tolerance patch should be in this week.
>> >> >>
>> >> >> The other features I want to throw for consideration are written
>> below.
>> >> >> Please feel free to prioritize for 0.6:
>> >> >>
>> >> >> BSPPeer API Improvements
>> >> >> - Broadcast API
>> >> >>    We have to experiment a broadcast API that does not use the
>> Messaging
>> >> >> queue. Few strategies for the purpose could be brainstormed.
>> >> >> - HAMA-546
>> >> >>
>> >> >> Task Scheduler Improvements
>> >> >> - Task Allocation as feature of BSP job instead of system (Coming
>> with
>> >> the
>> >> >> Fault tolerance patch) Today the task allocation responsibility
is
>> >> wrongly
>> >> >> implemented in TaskInProgress class. With this feature we can have
>> >> >> pluggable task allocation strategy with data-local allocation as
>> default
>> >> >> strategy.
>> >> >> - To support real time jobs, we would need to implement resource
>> local
>> >> >> scheduler, where resource here is more than the file input split.
>> Based
>> >> on
>> >> >> the nature of job, it could be nodes with bigger pipe for
>> communication,
>> >> >> nature of the machine(GPU), Nodes with SSDs, etc.
>> >> >>
>> >> >> Job Submission Improvements
>> >> >> - A Job Submission API, that gives users the flexibility to define
>> the
>> >> >> nature of messaging queue and other attributes per superstep
>> >> >>
>> >> >> Improve Performance And Scalability
>> >> >> - Benchmark on more nodes
>> >> >> - Try Asynchronous communication
>> >> >> - HAMA-593
>> >> >> - Spilling message queue and External Sorting Queue.
>> >> >>
>> >> >> Apart from this, there could be more graph algorithms to be
>> implemented,
>> >> >> ml
>> >> >> algorithms to be implemented and Mikalai's work integrated in the
>> >> release.
>> >> >> When are we planning to have next release?
>> >> >>
>> >> >> I can take a shot at release management, but I would definitely
need
>> >> help
>> >> >> for the first time.
>> >> >>
>> >> >> Thanks,
>> >> >> Suraj
>> >> >>
>> >> >> On Mon, Jul 9, 2012 at 3:22 AM, Thomas Jungblut
>> >> >> <thomas.jungblut@gmail.com>**wrote:
>> >> >>
>> >> >>  So now we released 0.5.0 what are our plans for 0.6.0?
>> >> >>> And who is going to do the release managing?
>> >> >>>
>> >> >>>
>> >> > --
>> >> > Best Regards, Edward J. Yoon
>> >> > @eddieyoon
>> >> >
>> >> >
>> >>
>>
>>
>>
>> --
>> Best Regards, Edward J. Yoon
>> @eddieyoon
>>



-- 
Best Regards, Edward J. Yoon
@eddieyoon

Mime
View raw message