hdt-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Rahul Sharma <rahul0...@gmail.com>
Subject Re: Incubator PMC Report
Date Tue, 05 Aug 2014 04:42:54 GMT
Hi Mirko,

Kite addition should help the cause but I am not sure if that is what we
want to do that next. There have been number of things that have been
proposed in the HDT proposal [1] e.g. supporting MRunit, Pig etc, which can
give us next direction.  Bob also shared some thoughts around product
features in the wiki [2]. I think we could take a leaf out of that as well.
IMO we should first try to support existing tooling of Hadoop ecosystem eg
MRUnit, Crunch, Pig, Oozie etc. Debugger could be a great add-on to the
plugin features. Lets throw in our thoughts to determine whats next ?

For new developers I think the best place to start is tacking one's nagging
issues. But maybe we can build a few jiras, provided we do something like
release planning, that way new developers would have some idea of where to
start-off.

regards,
Rahul

[1] http://wiki.apache.org/incubator/HadoopDevelopmentToolsProposal
[2] http://wiki.apache.org/hdt/HDTProductExperience


On Mon, Aug 4, 2014 at 5:23 PM, Mirko Kämpf <mirko.kaempf@gmail.com> wrote:

> Dear Rahul,
>
> thanks for preparation of the report. I think we should collect our ideas
> about the road map of the project.
>
> New developers can not see where to start with. So I think this is big
> barrier. I was thinking about an
> answer, but there is still one question: What is the mission of the
> project?
>
> We want to make software development "against a Hadoop cluster" much
> simpler. But how do we achieve this?
>
> A lot of work is done in the Kite-project. They offer abstractions for
> datasets and several tools.
>
> I suggest we should integrate this kind of functionality into Eclipse via
> the HDT project.
>
> Therefore I work on an example already:
>
> Kite Dataset Browser
> =====================
> One registers a bunch ob Kite-Datasets (no matter if HDFS, Hive, or HBase
> data) in a Kite-Datastore. This is an entry like a database in the eclipse
> plugin. If I click on the store icon, it shows all registered datasets and
> a double click retrieves the 50 first records and shows it in a table,
> together with existing metadata.
>
> Starting here, I think it might make sense to collect such stories in JIRA
> to enable new developers to find a place to start.
>
> I also see, there is a need for a "Morphline" editor and debugger plugin
> for Eclipse.
> And what about an Oozie connector as the high level connector? This would
> allow work with many different cluster, just depending on the Oozie version
> for job submission. But anyway the problem of Hadoop versions (or profiles
> has to be solved first). I think this is the major problem right now. If we
> could switch between different profiles we would be ready for the next step
> towards a TLP.
>
> What are your ideas regarding the Kite development?
>
> Best wishes
> Mirko
>
>
>
>
> 2014-08-04 12:18 GMT+01:00 Rahul Sharma <rahul0208@gmail.com>:
>
> > Hi Devs,
> >
> > We have not get a notification but we are due for report [1] this month.
> I
> > have compiled the following report. Let me know your thoughts.
> >
> > regards
> > Rahul
> >
> > [1] https://wiki.apache.org/incubator/August2014
> >
> >
> >
>  ------------------------------------------------------------------------------------------------------------------------
> >
> > Hadoop Development Tools
> >
> > Eclipse based tools for developing applications on the Hadoop platform
> >
> > Hadoop Development Tools has been incubating since 2012-11-09.
> >
> > Three most important issues to address in the move towards graduation:
> >
> >   1. Build Community
> >   2. Get more activity
> >   3. We have to define and implement an effective approach to manage the
> > "client connections" to multiple clusters.
> >
> > Any issues that the Incubator PMC (IPMC) or ASF Board wish/need to be
> > aware of?
> >
> >  - None
> >
> > How has the community developed since the last report?
> >
> >  - 133 mails exchanged on mailing list
> >
> > How has the project developed since the last report?
> >
> >  - Migrated all the existing functionality from master which enabled us
> fro
> > next release.
> >  - Released 0.0.2 incubating
> >
> > Date of last release:
> >
> >  - July'2014
> >
> > When were the last committers or PMC members elected?
> >
> >  - November 2013
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message