hama-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tommaso Teofili <tommaso.teof...@gmail.com>
Subject Re: Roadmap and Release Manager
Date Thu, 09 Aug 2012 11:46:54 GMT
same here, I'll most likely submit my work w.r.t. ML module in September.
Tommaso

2012/8/9 Edward J. Yoon <edwardyoon@apache.org>

> >> - Data repartitioning
> >> - Hadoop 2.0 compatibility
>
> P.S., My summer vacation will start from Aug 11th, so I need more time.
>
> On Mon, Jul 9, 2012 at 10:49 PM, Tommaso Teofili
> <tommaso.teofili@gmail.com> wrote:
> > 2012/7/9 Edward J. Yoon <edwardyoon@apache.org>
> >
> >> Then, let's try to release 0.6 by end of August or September 2012. I
> >> heard/saw some CDH4 plans/activities, we should not hesitate. :)
> >>
> >> Our TODO list:
> >>
> >> >> - Fault tolerance
> >> >> - Data repartitioning
> >> >> - Hadoop 2.0 compatibility
> >> >> - Hama documentation book in html
> >> >> - Bug fixes if any reported and few code improvements.
> >> > adding the streaming API.
> >>
> >
> > +1 for this roadmap.
> > I'd like to put some machine learning algorithm in there but I'm not sure
> > about the timing for it.
> > Regards,
> > Tommaso
> >
> >
> >>
> >> I'll work on yarn module and (large input data) re-partitioning issue.
> >>
> >> If you need my help such as test on large cluster, Just let me know.
> >>
> >> On Mon, Jul 9, 2012 at 9:37 PM, Thomas Jungblut
> >> <thomas.jungblut@gmail.com> wrote:
> >> > Suraj's plan would also be my favorite.
> >> > We also could split two faster releases from fault tolerance and
> Hadoop
> >> 2.0.
> >> >
> >> > If we have sorted out the fault tolerance I would like to spend my
> time
> >> on
> >> > adding the streaming API.
> >> >
> >> > 2012/7/9 Suraj Menon <surajsmenon@apache.org>
> >> >
> >> >> Sorry, I was looking at more time (~6 months) for the next release.
> This
> >> >> implies that are we are looking for a release in August end. The
> desired
> >> >> feature set:
> >> >>
> >> >> - Fault tolerance
> >> >> - Data repartitioning
> >> >> - Hadoop 2.0 compatibility
> >> >> - Hama documentation book in html
> >> >> - Bug fixes if any reported and few code improvements.
> >> >>
> >> >> I have to work on my Jenkins permissions. The last time I tried, it
> did
> >> not
> >> >> work for me.
> >> >>
> >> >> -Suraj
> >> >>
> >> >> On Mon, Jul 9, 2012 at 6:02 AM, Edward J. Yoon <
> edward.yoon@oracle.com
> >> >> >wrote:
> >> >>
> >> >> > We only need to discuss about 0.6 release roadmap. Any committer
> may
> >> >> > create a release candidate. Please feel free to create your own
RC
> and
> >> >> open
> >> >> > the vote in future.
> >> >> >
> >> >> > Personally, I would like to release along with support for Hadoop
> 2.0,
> >> >> > ASAP. Hope I can finish by this end of month or next month.
> >> >> >
> >> >> > And IMO, the current high-priority tasks are Input re-partitioning
> and
> >> >> > exception handling.
> >> >> >
> >> >> > Thanks,
> >> >> > Ed
> >> >> >
> >> >> >
> >> >> > On 7/9/2012 5:15 PM, Suraj Menon wrote:
> >> >> >
> >> >> >> Hello,
> >> >> >>
> >> >> >> Fault tolerance patch should be in this week.
> >> >> >>
> >> >> >> The other features I want to throw for consideration are written
> >> below.
> >> >> >> Please feel free to prioritize for 0.6:
> >> >> >>
> >> >> >> BSPPeer API Improvements
> >> >> >> - Broadcast API
> >> >> >>    We have to experiment a broadcast API that does not use
the
> >> Messaging
> >> >> >> queue. Few strategies for the purpose could be brainstormed.
> >> >> >> - HAMA-546
> >> >> >>
> >> >> >> Task Scheduler Improvements
> >> >> >> - Task Allocation as feature of BSP job instead of system
(Coming
> >> with
> >> >> the
> >> >> >> Fault tolerance patch) Today the task allocation responsibility
is
> >> >> wrongly
> >> >> >> implemented in TaskInProgress class. With this feature we
can have
> >> >> >> pluggable task allocation strategy with data-local allocation
as
> >> default
> >> >> >> strategy.
> >> >> >> - To support real time jobs, we would need to implement resource
> >> local
> >> >> >> scheduler, where resource here is more than the file input
split.
> >> Based
> >> >> on
> >> >> >> the nature of job, it could be nodes with bigger pipe for
> >> communication,
> >> >> >> nature of the machine(GPU), Nodes with SSDs, etc.
> >> >> >>
> >> >> >> Job Submission Improvements
> >> >> >> - A Job Submission API, that gives users the flexibility to
define
> >> the
> >> >> >> nature of messaging queue and other attributes per superstep
> >> >> >>
> >> >> >> Improve Performance And Scalability
> >> >> >> - Benchmark on more nodes
> >> >> >> - Try Asynchronous communication
> >> >> >> - HAMA-593
> >> >> >> - Spilling message queue and External Sorting Queue.
> >> >> >>
> >> >> >> Apart from this, there could be more graph algorithms to be
> >> implemented,
> >> >> >> ml
> >> >> >> algorithms to be implemented and Mikalai's work integrated
in the
> >> >> release.
> >> >> >> When are we planning to have next release?
> >> >> >>
> >> >> >> I can take a shot at release management, but I would definitely
> need
> >> >> help
> >> >> >> for the first time.
> >> >> >>
> >> >> >> Thanks,
> >> >> >> Suraj
> >> >> >>
> >> >> >> On Mon, Jul 9, 2012 at 3:22 AM, Thomas Jungblut
> >> >> >> <thomas.jungblut@gmail.com>**wrote:
> >> >> >>
> >> >> >>  So now we released 0.5.0 what are our plans for 0.6.0?
> >> >> >>> And who is going to do the release managing?
> >> >> >>>
> >> >> >>>
> >> >> > --
> >> >> > Best Regards, Edward J. Yoon
> >> >> > @eddieyoon
> >> >> >
> >> >> >
> >> >>
> >>
> >>
> >>
> >> --
> >> Best Regards, Edward J. Yoon
> >> @eddieyoon
> >>
>
>
>
> --
> Best Regards, Edward J. Yoon
> @eddieyoon
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message