hdt-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mirko Kämpf <mirko.kae...@gmail.com>
Subject Re: Incubator PMC Report
Date Mon, 04 Aug 2014 11:53:30 GMT
Dear Rahul,

thanks for preparation of the report. I think we should collect our ideas
about the road map of the project.

New developers can not see where to start with. So I think this is big
barrier. I was thinking about an
answer, but there is still one question: What is the mission of the project?

We want to make software development "against a Hadoop cluster" much
simpler. But how do we achieve this?

A lot of work is done in the Kite-project. They offer abstractions for
datasets and several tools.

I suggest we should integrate this kind of functionality into Eclipse via
the HDT project.

Therefore I work on an example already:

Kite Dataset Browser
One registers a bunch ob Kite-Datasets (no matter if HDFS, Hive, or HBase
data) in a Kite-Datastore. This is an entry like a database in the eclipse
plugin. If I click on the store icon, it shows all registered datasets and
a double click retrieves the 50 first records and shows it in a table,
together with existing metadata.

Starting here, I think it might make sense to collect such stories in JIRA
to enable new developers to find a place to start.

I also see, there is a need for a "Morphline" editor and debugger plugin
for Eclipse.
And what about an Oozie connector as the high level connector? This would
allow work with many different cluster, just depending on the Oozie version
for job submission. But anyway the problem of Hadoop versions (or profiles
has to be solved first). I think this is the major problem right now. If we
could switch between different profiles we would be ready for the next step
towards a TLP.

What are your ideas regarding the Kite development?

Best wishes

2014-08-04 12:18 GMT+01:00 Rahul Sharma <rahul0208@gmail.com>:

> Hi Devs,
> We have not get a notification but we are due for report [1] this month. I
> have compiled the following report. Let me know your thoughts.
> regards
> Rahul
> [1] https://wiki.apache.org/incubator/August2014
>  ------------------------------------------------------------------------------------------------------------------------
> Hadoop Development Tools
> Eclipse based tools for developing applications on the Hadoop platform
> Hadoop Development Tools has been incubating since 2012-11-09.
> Three most important issues to address in the move towards graduation:
>   1. Build Community
>   2. Get more activity
>   3. We have to define and implement an effective approach to manage the
> "client connections" to multiple clusters.
> Any issues that the Incubator PMC (IPMC) or ASF Board wish/need to be
> aware of?
>  - None
> How has the community developed since the last report?
>  - 133 mails exchanged on mailing list
> How has the project developed since the last report?
>  - Migrated all the existing functionality from master which enabled us fro
> next release.
>  - Released 0.0.2 incubating
> Date of last release:
>  - July'2014
> When were the last committers or PMC members elected?
>  - November 2013

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message