hadoop-general mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Nigel Daley <nda...@mac.com>
Subject Re: Hadoop build machine donations?
Date Wed, 03 Aug 2011 05:24:57 GMT
Yes, donating hardware to ASF is tricky for all the reasons you've outlined, not the least
of which you can't donate to a specific project.  That's why I emphasized **access** to hardware
in my request.

Background of current ASF build machines: 
1) Y! officially donated 4 machines to ASF, hosting them in a Y! data center.  ASF allocated
2 to Jenkins which helped Hadoop a little bit.  These are minerva and vesta, aka ubuntu1 and
ubuntu2 slaves.  The other 2 were allocated to a build bot setup which didn't help Hadoop.
2) A year or so later Y! put another 30 machines in the same rack as the 4 donated ASF machines,
but didn't actually donate them to the ASF.   When I asked ASF Infra folks, no one had problems
hooking up non ASF hardware to the ASF Jenkins.  Thus from these 30 machines, hadoop1 thru
hadoop9 are now Jenkins slaves, plus a bunch of other nodes not connected to Jenkins. 

So, donating **access** to hardware for Hadoop builds has been a relatively successful model
over the past couple years.  Again, another open to donating **access** to hardware for Hadoop
builds?

Cheers,
Nige



On Aug 2, 2011, at 12:08 PM, Andrew Bayer wrote:

> As I mentioned before, I believe that ASF Infra explicitly does *not* want
> donated EC2 instances, but again, check with them to be sure and as to what
> the donation process is regardless.
> 
> A.
> 
> On Tue, Aug 2, 2011 at 12:01 PM, <Milind.Bhandarkar@emc.com> wrote:
> 
>> That's good to know, Kos.
>> 
>> Is there any place I can find the actual usage (in terms of utilization /
>> memory-cpu-disk usage) of these build machines ?
>> 
>> Based on the Amazon EC2 pricing (http://aws.amazon.com/ec2/pricing/), I
>> would like to determine which type of instances would be adequate, and how
>> much yearly budget would be needed.
>> 
>> Since ASF is a 501(c)(3) non-profit, charitable organization, these
>> donations would be tax deductible for U.S. Based individuals. (Plus for
>> new AWS members, there is a free tier for 1 year.) Is there build infra in
>> place that will allow these individually donated ec2 instances to be
>> utilized ?
>> 
>> - Milind
>> ---
>> Milind Bhandarkar
>> Greenplum Labs,
>> EMC
>> 
>> 
>> (Disclaimer: Opinions expressed in this email are those of the author, and
>> do
>> not necessarily represent the views of any organization, past or
>> present, the author might be affiliated with.)
>> 
>> 
>> 
>> 
>> 
>> On 8/2/11 11:39 AM, "Konstantin Boudnik" <cos@apache.org> wrote:
>> 
>>> Milind.
>>> 
>>> From about 30 boxes Y! gave to ASF at least (I believe there were more,
>>> but at
>>> least that much) 10 of them were used _exclusively_ as Hadoop build/test
>>> machines (e.g. Hudson slaves) and a number of others (about 5 at least)
>>> were
>>> used for Hadoop deployment testing _exclusively_. So, I think the claim
>>> above
>>> doesn't reflect the real situation with hardware donation or perhaps is
>>> missing some of the details ;).
>>> 
>>> With that in mind I came across this article goo.gl/EVp5S and now I have
>> a
>>> bright idea! Shall the hardware donation be proportional to the proclaimed
>>> market share (or installation base) of a particular commercial
>>> distribution's
>>> vendor?  That'd be only fair I guess... Apache Hadoop is about 65% of the
>>> installation and Y! brought in 30 machines. If a vendor A or C has say 30%
>>> then may be they feel good by donating something like 15 boxes to the
>>> cause?
>>> 
>>> --
>>> Take care,
>>> Konstantin (Cos) Boudnik
>>> 2CAC 8312 4870 D885 8616  6115 220F 6980 1F27 E622
>>> 
>>> Disclaimer: Opinions expressed in this email are those of the author,
>>> and do
>>> not necessarily represent the views of any company the author might be
>>> affiliated with at the moment of writing.
>>> 
>>> 
>>> On Tue, Aug 02, 2011 at 01:38PM, Milind.Bhandarkar@emc.com wrote:
>>>> A couple of questions for Nigel and Andrew:
>>>> 
>>>> 1. What do you mean by "non-single-purpose machines" ? (I guess,
>>>> machines
>>>> they can use for non-Hadoop projects as well ?)
>>>> 
>>>> 2. Do these machines have to be physical boxes or virtual machines be
>>>> okay
>>>> too ? (sudo on virtual machines would be easier to sell than on physical
>>>> boxes.)
>>>> 
>>>> 3. What's the process for donations ?
>>>> 
>>>> - milind
>>>> 
>>>> ---
>>>> Milind Bhandarkar
>>>> Greenplum Labs,
>>>> EMC
>>>> 
>>>> 
>>>> 
>>>> 
>>>> On 8/2/11 8:07 AM, "Andrew Bayer" <andrew.bayer@gmail.com> wrote:
>>>> 
>>>>> FWIW, the ASF Infra guys aren't that keen on more machines they won't
>>>> have
>>>>> physical control of. There may be some wiggle room on that, but they'll
>>>>> definitely need sudo and they would much prefer non-single-purpose
>>>>> machines.
>>>>> Just a heads up from my talks with Infra lately.
>>>>> 
>>>>> A.
>>>>> 
>>>>> On Mon, Aug 1, 2011 at 11:22 PM, Nigel Daley <ndaley@mac.com> wrote:
>>>>> 
>>>>>> Folks,
>>>>>> 
>>>>>> Given the Hadoop build machines have been offline for a few day,
this
>>>>>> would
>>>>>> be a good time to see if other companies can donate **access**  to
>>>> some
>>>>>> build hardware to diversify our donors.  Currently there are roughly
>>>> 30
>>>>>> machines donated by Yahoo!
>>>>>> 
>>>>>> Ideally the hardware is:
>>>>>> * hosted and OS managed by the donor,
>>>>>> * publicly addressable on the internet,
>>>>>> * running Ubuntu or CentOS, and
>>>>>> * sudo access can be given to Apache's Jenkins admins so they can
>>>> create
>>>>>> accounts for committers as needed.
>>>>>> 
>>>>>> Anyone?
>>>>>> 
>>>>>> Thanks,
>>>>>> Nige
>>>>>> 
>>>> 
>>> 
>> 
>> 


Mime
View raw message