hama-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Suraj Menon <surajsme...@apache.org>
Subject Re: Roadmap and Release Manager
Date Thu, 09 Aug 2012 12:02:11 GMT
I suggest we do a code freeze in the last week of August for 0.6.0. and
spend the next week fixing and hardening the release candidate(s).
And then we release 0.6.1 by the end of September. Version 0.6.1 would have
our examples with Fault tolerance and other new features and API changes we
put by then.

This gives a chance to wider audiences to try our release early and give
their feedback.
Based on our progress till Aug 20th, we can decide the JIRA issues that
would get fixed in 0.6.0 before the code freeze and which of them would be
shipped in 0.6.1.
Let me know your thoughts on this.

Thanks,
Suraj


On Thu, Aug 9, 2012 at 7:46 AM, Tommaso Teofili
<tommaso.teofili@gmail.com>wrote:

> same here, I'll most likely submit my work w.r.t. ML module in September.
> Tommaso
>
> 2012/8/9 Edward J. Yoon <edwardyoon@apache.org>
>
> > >> - Data repartitioning
> > >> - Hadoop 2.0 compatibility
> >
> > P.S., My summer vacation will start from Aug 11th, so I need more time.
> >
> > On Mon, Jul 9, 2012 at 10:49 PM, Tommaso Teofili
> > <tommaso.teofili@gmail.com> wrote:
> > > 2012/7/9 Edward J. Yoon <edwardyoon@apache.org>
> > >
> > >> Then, let's try to release 0.6 by end of August or September 2012. I
> > >> heard/saw some CDH4 plans/activities, we should not hesitate. :)
> > >>
> > >> Our TODO list:
> > >>
> > >> >> - Fault tolerance
> > >> >> - Data repartitioning
> > >> >> - Hadoop 2.0 compatibility
> > >> >> - Hama documentation book in html
> > >> >> - Bug fixes if any reported and few code improvements.
> > >> > adding the streaming API.
> > >>
> > >
> > > +1 for this roadmap.
> > > I'd like to put some machine learning algorithm in there but I'm not
> sure
> > > about the timing for it.
> > > Regards,
> > > Tommaso
> > >
> > >
> > >>
> > >> I'll work on yarn module and (large input data) re-partitioning issue.
> > >>
> > >> If you need my help such as test on large cluster, Just let me know.
> > >>
> > >> On Mon, Jul 9, 2012 at 9:37 PM, Thomas Jungblut
> > >> <thomas.jungblut@gmail.com> wrote:
> > >> > Suraj's plan would also be my favorite.
> > >> > We also could split two faster releases from fault tolerance and
> > Hadoop
> > >> 2.0.
> > >> >
> > >> > If we have sorted out the fault tolerance I would like to spend my
> > time
> > >> on
> > >> > adding the streaming API.
> > >> >
> > >> > 2012/7/9 Suraj Menon <surajsmenon@apache.org>
> > >> >
> > >> >> Sorry, I was looking at more time (~6 months) for the next release.
> > This
> > >> >> implies that are we are looking for a release in August end. The
> > desired
> > >> >> feature set:
> > >> >>
> > >> >> - Fault tolerance
> > >> >> - Data repartitioning
> > >> >> - Hadoop 2.0 compatibility
> > >> >> - Hama documentation book in html
> > >> >> - Bug fixes if any reported and few code improvements.
> > >> >>
> > >> >> I have to work on my Jenkins permissions. The last time I tried,
it
> > did
> > >> not
> > >> >> work for me.
> > >> >>
> > >> >> -Suraj
> > >> >>
> > >> >> On Mon, Jul 9, 2012 at 6:02 AM, Edward J. Yoon <
> > edward.yoon@oracle.com
> > >> >> >wrote:
> > >> >>
> > >> >> > We only need to discuss about 0.6 release roadmap. Any committer
> > may
> > >> >> > create a release candidate. Please feel free to create your
own
> RC
> > and
> > >> >> open
> > >> >> > the vote in future.
> > >> >> >
> > >> >> > Personally, I would like to release along with support for
Hadoop
> > 2.0,
> > >> >> > ASAP. Hope I can finish by this end of month or next month.
> > >> >> >
> > >> >> > And IMO, the current high-priority tasks are Input
> re-partitioning
> > and
> > >> >> > exception handling.
> > >> >> >
> > >> >> > Thanks,
> > >> >> > Ed
> > >> >> >
> > >> >> >
> > >> >> > On 7/9/2012 5:15 PM, Suraj Menon wrote:
> > >> >> >
> > >> >> >> Hello,
> > >> >> >>
> > >> >> >> Fault tolerance patch should be in this week.
> > >> >> >>
> > >> >> >> The other features I want to throw for consideration
are written
> > >> below.
> > >> >> >> Please feel free to prioritize for 0.6:
> > >> >> >>
> > >> >> >> BSPPeer API Improvements
> > >> >> >> - Broadcast API
> > >> >> >>    We have to experiment a broadcast API that does not
use the
> > >> Messaging
> > >> >> >> queue. Few strategies for the purpose could be brainstormed.
> > >> >> >> - HAMA-546
> > >> >> >>
> > >> >> >> Task Scheduler Improvements
> > >> >> >> - Task Allocation as feature of BSP job instead of system
> (Coming
> > >> with
> > >> >> the
> > >> >> >> Fault tolerance patch) Today the task allocation responsibility
> is
> > >> >> wrongly
> > >> >> >> implemented in TaskInProgress class. With this feature
we can
> have
> > >> >> >> pluggable task allocation strategy with data-local allocation
as
> > >> default
> > >> >> >> strategy.
> > >> >> >> - To support real time jobs, we would need to implement
resource
> > >> local
> > >> >> >> scheduler, where resource here is more than the file
input
> split.
> > >> Based
> > >> >> on
> > >> >> >> the nature of job, it could be nodes with bigger pipe
for
> > >> communication,
> > >> >> >> nature of the machine(GPU), Nodes with SSDs, etc.
> > >> >> >>
> > >> >> >> Job Submission Improvements
> > >> >> >> - A Job Submission API, that gives users the flexibility
to
> define
> > >> the
> > >> >> >> nature of messaging queue and other attributes per superstep
> > >> >> >>
> > >> >> >> Improve Performance And Scalability
> > >> >> >> - Benchmark on more nodes
> > >> >> >> - Try Asynchronous communication
> > >> >> >> - HAMA-593
> > >> >> >> - Spilling message queue and External Sorting Queue.
> > >> >> >>
> > >> >> >> Apart from this, there could be more graph algorithms
to be
> > >> implemented,
> > >> >> >> ml
> > >> >> >> algorithms to be implemented and Mikalai's work integrated
in
> the
> > >> >> release.
> > >> >> >> When are we planning to have next release?
> > >> >> >>
> > >> >> >> I can take a shot at release management, but I would
definitely
> > need
> > >> >> help
> > >> >> >> for the first time.
> > >> >> >>
> > >> >> >> Thanks,
> > >> >> >> Suraj
> > >> >> >>
> > >> >> >> On Mon, Jul 9, 2012 at 3:22 AM, Thomas Jungblut
> > >> >> >> <thomas.jungblut@gmail.com>**wrote:
> > >> >> >>
> > >> >> >>  So now we released 0.5.0 what are our plans for 0.6.0?
> > >> >> >>> And who is going to do the release managing?
> > >> >> >>>
> > >> >> >>>
> > >> >> > --
> > >> >> > Best Regards, Edward J. Yoon
> > >> >> > @eddieyoon
> > >> >> >
> > >> >> >
> > >> >>
> > >>
> > >>
> > >>
> > >> --
> > >> Best Regards, Edward J. Yoon
> > >> @eddieyoon
> > >>
> >
> >
> >
> > --
> > Best Regards, Edward J. Yoon
> > @eddieyoon
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message