hawq-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Wen Lin <w...@pivotal.io>
Subject Re: [Propose] Create a new HAWQ roadmap page
Date Thu, 14 Jul 2016 07:52:57 GMT
Hi,

Glad to see this roadmap page is created. I am wondering if some
enhancements for Hadoop YARN related can be added into this roadmap.
1. support "YARN label based scheduling" in HAWQ and libyarn,
https://issues.apache.org/jira/browse/HAWQ-742
2. support Fair Scheduler in HAWQ and libyarn;
3. support resource preemption with Capacity Scheduler in HAWQ and libyarn,
http://hortonworks.com/blog/better-slas-via-resource-preemption-in-yarns-capacityscheduler/

It seems no jira is created for number 2 and 3.

Thanks!

On Thu, Jul 14, 2016 at 8:54 AM, Lei Chang <lei_chang@apache.org> wrote:

> Added to the page:
> https://cwiki.apache.org/confluence/display/HAWQ/HAWQ+Roadmap
>
> It can be incrementally updated.
>
> Cheers
> Lei
>
>
>
>
> On Wed, Jun 29, 2016 at 5:23 PM, Lei Chang <lei_chang@apache.org> wrote:
>
> >
> > I classified the items into the following categories. appreciate your
> > comments.
> >
> > Cloud related:
> > [HAWQ-308] - S3 Integration
> > [HAWQ-310] - Snapshot support
> >
> > Data Management Functionality Enhancement
> > [HAWQ-786] - Framework to support pluggable formats and file systems
> > [HAWQ-864] - Support ORC as a native file format
> > [HAWQ-150] - External tables can be designated for both READ and WRITE
> > [HAWQ-304] - Support update and delete on non-heap tables
> > [HAWQ-401] - json type support
> > [HAWQ-319] - REST API for HAWQ
> > [HAWQ-312] - Multiple active master support
> >
> > Performance enhancement
> > [HAWQ-303] - Index support for non-heap tables
> >
> > Languages & Analytics
> > [HAWQ-321] - Support plpython3u
> >
> > Ecosystem:
> > [HAWQ-256] - Integrate Security with Apache Ranger
> > [HAWQ-29] - Refactor HAWQ InputFormat to support Spark/Scala
> >
> > Management & Build
> > [HAWQ-8] - Installing the HAWQ Software thru the Apache Ambari
> > [HAWQ-311] - Data Transfer tool
> > [HAWQ-326] - Support RPM build for HAWQ
> >
> > Cheers
> > Lei
> >
> >
> >
> >
> > On Fri, Jun 24, 2016 at 5:10 PM, Lei Chang <lei_chang@apache.org> wrote:
> >
> >>
> >> Nice, I created a page and we can discuss the items and put them on the
> >> page.
> >>
> >> For the items, I think it makes sense to add at least items in the jira
> >> roadmap panel, here are some major ones I extracted from the panel.
> looks
> >> better to classify them into categories.
> >>
> >> [HAWQ-786] - Framework to support pluggable formats and file systems
> >> [HAWQ-864] - Support ORC as a native file format
> >> [HAWQ-308] - S3 Integration
> >> [HAWQ-256] - Integrate Security with Apache Ranger
> >> [HAWQ-150] - External tables can be designated for both READ and WRITE
> >> [HAWQ-303] - Index support for non-heap tables
> >> [HAWQ-304] - Support update and delete on non-heap tables
> >> [HAWQ-310] - Snapshot support
> >> [HAWQ-312] - Multiple active master support
> >> [HAWQ-319] - REST API for HAWQ
> >> [HAWQ-321] - Support plpython3u
> >> [HAWQ-29] - Refactor HAWQ InputFormat to support Spark/Scala
> >> [HAWQ-311] - Data Transfer tool
> >> [HAWQ-326] - Support RPM build for HAWQ
> >> [HAWQ-8] - Installing the HAWQ Software thru the Apache Ambari
> >> [HAWQ-752] - build pxf compatible with Apache Hadoop
> >> [HAWQ-401] - json type support
> >>
> >> Cheers
> >> Lei
> >>
> >>
> >>
> >> On Thu, Jun 23, 2016 at 11:23 PM, Vineet Goel <vvineet@apache.org>
> wrote:
> >>
> >>> +1 too
> >>>
> >>> I can help start a draft on the wiki based on historical user requests
> >>> and
> >>> trends in the ecosystem. And of course, the roadmap is a living and
> >>> breathing document which will continue to evolve over time based on
> >>> continuous feedback, and more.
> >>>
> >>> -Vineet
> >>>
> >>>
> >>> On Thu, Jun 23, 2016 at 8:18 AM, Kavinder Dhaliwal <
> kdhaliwal@pivotal.io
> >>> >
> >>> wrote:
> >>>
> >>> > +1 I'm in favor of this. The Zeppelin roadmap is very community
> driven
> >>> and
> >>> > having something similar for HAWQ will go a long way to getting more
> >>> > feedback about the overall direction and goals of HAWQ.
> >>> >
> >>> > On Thu, Jun 23, 2016 at 2:02 AM, Lei Chang <lei_chang@apache.org>
> >>> wrote:
> >>> >
> >>> > > Hi Guys,
> >>> > >
> >>> > > I noticed there are a lot of requests about hawq roadmaps coming
> >>> from the
> >>> > > offline hawq activities (meetup et al).
> >>> > >
> >>> > > Although we have the list of backlog JIRAs on our JIRA page
> >>> > > <
> >>> > >
> >>> >
> >>>
> https://issues.apache.org/jira/browse/HAWQ/?selectedTab=com.atlassian.jira.jira-projects-plugin:roadmap-panel
> >>> > > >.
> >>> > > But it does not give a high level description. A good example
from
> >>> other
> >>> > > communities is here:
> >>> > >
> >>> https://cwiki.apache.org/confluence/display/ZEPPELIN/Zeppelin+Roadmap
> >>> > >
> >>> > > So I am proposing we have a similar HAWQ Roadmap page maintained
on
> >>> our
> >>> > > wiki page.
> >>> > >
> >>> > > Thoughts?
> >>> > >
> >>> > > Cheers
> >>> > > Lei
> >>> > >
> >>> >
> >>>
> >>
> >>
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message