incubator-general mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jean-Baptiste Onofré ...@nanthrax.net>
Subject Re: Hoya Proposal
Date Fri, 10 Jan 2014 09:54:00 GMT
Hi Steve,

can I help during the week end ?

Regards
JB

On 01/10/2014 10:31 AM, Steve Loughran wrote:
> I'm going to do the rename over the w/end, I just need to get a functional
> tests branch in, because merge-over-renames is one of those things you
> don't want to have to do.
>
>
> On 9 January 2014 17:29, Alejandro Abdelnur <tucu@cloudera.com> wrote:
>
>> argh, gmail trying to be extra smart collapsed his comment on that and i
>> missed. sorry
>>
>> thx
>>
>> Alejandro
>> (phone typing)
>>
>>> On Jan 9, 2014, at 9:23, larry mccay <lmccay@apache.org> wrote:
>>>
>>> Hi Alejandro -
>>>
>>> I believe that Steve has already acknowledged that a package rename is
>>> required and that the project is likely not appropriate for Hadoop
>> proper.
>>>
>>> thanks,
>>>
>>> --larry
>>>
>>>
>>> On Thu, Jan 9, 2014 at 11:46 AM, Alejandro Abdelnur <tucu@cloudera.com
>>> wrote:
>>>
>>>> I may have not been clear enough, I was referring to using
>>>> 'org.apache.hadoop' as package prefix for a project other than hadoop.
>>>>
>>>> Thanks
>>>>
>>>>
>>>> On Thu, Jan 9, 2014 at 8:36 AM, Benson Margulies <bimargulies@gmail.com
>>>>> wrote:
>>>>
>>>>> If you can work out a plan to do this directly in Hadoop, there's no
>>>>> need for the incubator. You just build and and contribute it in
>>>>> cahoots with them, and earn commit over there as you go.
>>>>>
>>>>> On Thu, Jan 9, 2014 at 11:14 AM, Alejandro Abdelnur <tucu@cloudera.com
>>>
>>>>> wrote:
>>>>>> Mmmh, if i recall correctly this has come up in the past with other
>>>>> projects and it was decided against it. Could you please check with the
>>>>> hadoop folks about it?
>>>>>>
>>>>>> Thx
>>>>>>
>>>>>>> On Jan 9, 2014, at 1:19 AM, Steve Loughran <stevel@hortonworks.com>
>>>>> wrote:
>>>>>>>
>>>>>>> no its wrong, it should all be under org.apache.hoya.
>>>>>>>
>>>>>>> I had the hadoop prefix so that I could perhaps put it straight
into
>>>> the
>>>>>>> hadoop code as another tools module -no need for incubation.
But as
>>>> the
>>>>>>> actual providers and all tests are related to the deployment
of hbase
>>>>> and
>>>>>>> accumulo, it really comes downstream of those.
>>>>>>>
>>>>>>> so a rename is needed.
>>>>>>>
>>>>>>> but yes, ASF headers everywhere
>>>>>>>
>>>>>>>
>>>>>>>> On 8 January 2014 22:48, Henry Saputra <henry.saputra@gmail.com>
>>>>> wrote:
>>>>>>>>
>>>>>>>> I like how the initial code already put under "
>>>>>>>> org.apache.hadoop.hoya"  with correct ASF header =)
>>>>>>>>
>>>>>>>> - Henry
>>>>>>>>
>>>>>>>> On Wed, Jan 8, 2014 at 7:08 AM, Steve Loughran <
>>>> stevel@hortonworks.com
>>>>>>
>>>>>>>> wrote:
>>>>>>>>> I'm starting to put together the incubation proposal
for Hoya: a
>>>> tool
>>>>> to
>>>>>>>>> dynamically deploy applications such as HBase or Accumulo
on YARN
>>>>>>>>>
>>>>>>>>> https://wiki.apache.org/incubator/HoyaProposal
>>>>>>>>>
>>>>>>>>> It does already work to the extent that it can bring
up either
>>>>>>>> application,
>>>>>>>>> run different clusters of different versions, and remember
where
>>>>>>>> containers
>>>>>>>>> were allocated so that on application restart it can
ask for them
>>>>> back.
>>>>>>>>> That increases data locality and makes a big difference
with HBase.
>>>>>>>>>
>>>>>>>>> It also needs a lot more work -YARN-896 is adding YARN
features
>> that
>>>>>>>> help,
>>>>>>>>> but there's lots of fun to be had in Hoya including
>>>>>>>>> -leading edge work in failure handling, modelling cluster
>>>>> unreliability
>>>>>>>>> and reacting to it. Can we move beyond simple blacklisting
to
>>>>>>>>> "greylisting", accepting unreliable boxes if we have
no altenatives
>>>>>>>>>
>>>>>>>>> Then there's adding more providers, to support different
>> application
>>>>>>>>> installations -I'm starting to write a functional test
framework
>>>> which
>>>>>>>> need
>>>>>>>>> provider-specific workload generations
>>>>>>>>>
>>>>>>>>> Other features: AM should have a web ui that redirects
to the live
>>>>>>>>> endpoints to all the app-specific UIs (e.g. HBase Master
GUI), as
>>>>> well as
>>>>>>>>> displaying cluster state itself, for people and for management
>>>>> tools....
>>>>>>>>>
>>>>>>>>> To summarise: lots of fun to be had
>>>>>>>>>
>>>>>>>>> -steve
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> CONFIDENTIALITY NOTICE
>>>>>>>>> NOTICE: This message is intended for the use of the individual
or
>>>>> entity
>>>>>>>> to
>>>>>>>>> which it is addressed and may contain information that
is
>>>>> confidential,
>>>>>>>>> privileged and exempt from disclosure under applicable
law. If the
>>>>> reader
>>>>>>>>> of this message is not the intended recipient, you are
hereby
>>>> notified
>>>>>>>> that
>>>>>>>>> any printing, copying, dissemination, distribution, disclosure
or
>>>>>>>>> forwarding of this communication is strictly prohibited.
If you
>> have
>>>>>>>>> received this communication in error, please contact
the sender
>>>>>>>> immediately
>>>>>>>>> and delete it from your system. Thank You.
>>>>>>>>
>>>>>>>>
>> ---------------------------------------------------------------------
>>>>>>>> To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
>>>>>>>> For additional commands, e-mail: general-help@incubator.apache.org
>>>>>>>
>>>>>>> --
>>>>>>> CONFIDENTIALITY NOTICE
>>>>>>> NOTICE: This message is intended for the use of the individual
or
>>>>> entity to
>>>>>>> which it is addressed and may contain information that is
>>>> confidential,
>>>>>>> privileged and exempt from disclosure under applicable law. If
the
>>>>> reader
>>>>>>> of this message is not the intended recipient, you are hereby
>> notified
>>>>> that
>>>>>>> any printing, copying, dissemination, distribution, disclosure
or
>>>>>>> forwarding of this communication is strictly prohibited. If you
have
>>>>>>> received this communication in error, please contact the sender
>>>>> immediately
>>>>>>> and delete it from your system. Thank You.
>>>>>>
>>>>>> ---------------------------------------------------------------------
>>>>>> To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
>>>>>> For additional commands, e-mail: general-help@incubator.apache.org
>>>>>
>>>>> ---------------------------------------------------------------------
>>>>> To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
>>>>> For additional commands, e-mail: general-help@incubator.apache.org
>>>>
>>>>
>>>> --
>>>> Alejandro
>>>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
>> For additional commands, e-mail: general-help@incubator.apache.org
>>
>>
>

-- 
Jean-Baptiste Onofré
jbonofre@apache.org
http://blog.nanthrax.net
Talend - http://www.talend.com

---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
For additional commands, e-mail: general-help@incubator.apache.org


Mime
View raw message