Mailing-List: contact general-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: general@hadoop.apache.org
Received-SPF: neutral (athena.apache.org: 69.147.107.21 is neither permitted
 nor denied by domain of mattf@yahoo-inc.com)
From: Matthew Foley <mattf@yahoo-inc.com>
To: "general@hadoop.apache.org" <general@hadoop.apache.org>
CC: "trademarks@apache.org" <trademarks@apache.org>,
        Matthew Foley
	<mattf@yahoo-inc.com>
Date: Wed, 15 Jun 2011 11:00:20 -0700
Subject: Re: [VOTE] Shall we adopt the "Defining Hadoop" page
Thread-Topic: [VOTE] Shall we adopt the "Defining Hadoop" page
Thread-Index: AcwrhhbgxJxKg/kTR0aiVqmVkFsFZw==
Message-ID: <EE59E34D-6618-4A58-8D09-95669FF2EC08@yahoo-inc.com>
References: <A3383C10-E615-434D-A9E1-8C444827596D@apache.org>
 <BANLkTimRX0N7pyrnQrfiwcN=AqHE=FJWzg@mail.gmail.com>
 <090F2993-B9AA-410F-AD99-7AC047FFC974@apache.org>
 <BANLkTi=fkguAtbfi9Wk9CPm8JmFa2Hti3Q@mail.gmail.com>
 <54668CDE-443D-4AF1-B17A-0497D0CDCC8D@yahoo-inc.com>
In-Reply-To: <54668CDE-443D-4AF1-B17A-0497D0CDCC8D@yahoo-inc.com>
Accept-Language: en-US
Content-Language: en-US
acceptlanguage: en-US
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0

Oh, and while I can't officially vote, I think this page is extremely well =
done and
I strongly support it. =20

As an editorial note, however, I would remove the last paragraph in the "Co=
mpatibility"=20
section, referencing the email thread (that I contributed to at length :-) =
). =20
That thread went all over the place, and would be misinforming to the typic=
al
reader.  The distillation on this twiki page IS normative and not confusing=
, and=20
we should leave it at that.

Best,
--Matt

On Jun 15, 2011, at 10:44 AM, Matthew Foley wrote:

Eli, you said:
> Putting a build of Hadoop that has 4 security patches applied into the sa=
me
> category as a product that has entirely re-worked the code and not
> gotten it checked into trunk does a major disservice to the people who
> contribute to and invest in the project.

How would you phrase the distinction, so that it is clear and reasonably un=
ambiguous
for people who are not Hadoop developers?  Do the HTTP and Subversion polic=
ies
draw this distinction, and if so could you please point us at the specific =
text, or=20
copy that text to this thread?

Thanks,
--Matt


On Jun 15, 2011, at 9:40 AM, Eli Collins wrote:

On Tue, Jun 14, 2011 at 7:45 PM, Owen O'Malley <omalley@apache.org> wrote:
>=20
> On Jun 14, 2011, at 5:48 PM, Eli Collins wrote:
>=20
>> Wrt derivative works, it's not clear from the document, but I think we
>> should explicitly adopt the policy of HTTPD and Subversion that
>> backported patches from trunk and security fixes are permitted.
>=20
> Actually, the document is extremely clear that only Apache releases may b=
e called Hadoop.
>=20
> There was a very long thread about why the rapidly expanding Hadoop-ecosy=
stem is leading to at lot of customer confusion about the different "versio=
ns" of Hadoop. We as the Hadoop project don't have the resources or the nec=
essary compatibility test suite to test compatibility between the different=
 sets of cherry picked patches. We also don't have time to ensure that all =
of the 1,000's of patches applied to 0.20.2 in each of the many (10? 15?) d=
ifferent versions have been committed to trunk. Futhermore, under the Apach=
e license, a company Foo could claim that it is a cherry pick version of Ha=
doop without releasing their source code that would enable verification.
>=20
> In summary,
> 1. Hadoop is very successful.
> 2. There are many different commercial products that are trying to use th=
e Hadoop name.
> 3. We can't check or enforce that the cherry pick versions are following =
the rules.
> 4. We don't have a TCK like Java does to validate new versions are compat=
ible.
> 5. By far the most fair way to ensure compatibility and fairness between =
companies is that only Apache Hadoop releases may be called Hadoop.
>=20
> That said, a package that includes a small number (< 3) of security patch=
es that haven't been released yet doesn't seem unreasonable.
>=20

I've spoken with ops teams at many companies,  I am not aware of
anyone who runs an official release (with just 2 security patches). By
this definition many of the most valuable contributors to Hadoop,
including Yahoo!, Cloudera, Facebook, etc are not using Hadoop.  Is
that really the message we want to send? We expect the PMC to enforce
this equally across all parties?

It's a fact of life that companies and ops teams that support Hadoop
need to patch the software before the PMC has time and/or will to vote
on new releases. This is why HTTP and Subversion allow this. Putting a
build of Hadoop that has 4 security patches applied into the same
category as a product that has entirely re-worked the code and not
gotten it checked into trunk does a major disservice to the people who
contribute to and invest in the project.

Thanks,
Eli