hadoop-general mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Todd Lipcon <t...@cloudera.com>
Subject Re: [VOTE] Release Apache Hadoop 0.23.1-rc2
Date Thu, 23 Feb 2012 05:00:33 GMT
On Wed, Feb 22, 2012 at 7:51 PM, Vinod Kumar Vavilapalli
<vinodkv@hortonworks.com> wrote:
> Todd,
> From your analysis at HDFS-2991, looks like this was there in 0.23
> too. Also, seems this happens only at scale, and only (paraphrasing
> you) "when the file is reopened for append on an exact block
> boundary".

Let me clarify: HDFS-2991 basically has two halves:
First half (been present "forever"): when we append() on a block
boundary, we don't log an OP_ADD
Second half (new due to HDFS-2718): if we get an OP_CLOSE for a file
we haven't OP_ADDed, we'll get a ClassCastException on startup.

So even though the first half isn't a regression, the regression in
the second half means that this longstanding bug will now actually
prevent startup.

Also, there's nothing related to scale here. I happened to run into it
doing scale tests, but it turned out to not be relevant. You'll see it
if you run TestDFSIO with standard parameters on trunk or 23.1 (that's
how I discovered it).

> Agree it is a critical fix, but given above, can we proceed along with
> 0.23.1? Anyways, 0.23.1 is still an alpha (albeit of next level), so
> I'd think we can get that in for 0.23.2.

Alright, consider me -0, though it's pretty nasty once you run into
it. The only way I could start my NN again without losing data was to
recompile with the fix in place.


> On Wed, Feb 22, 2012 at 6:43 PM, Todd Lipcon <todd@cloudera.com> wrote:
>> -1, unfortunately. HDFS-2991 is a blocker regression introduced in
>> 0.23.1. See the JIRA for instructions on how to reproduce on the rc2
>> build.
>> -Todd
>> On Fri, Feb 17, 2012 at 11:23 PM, Arun C Murthy <acm@hortonworks.com> wrote:
>>> I've created another release candidate for hadoop-0.23.1 that I would like to
>>> It is available at: http://people.apache.org/~acmurthy/hadoop-0.23.1-rc2/
>>> The hadoop-0.23.1-rc2 svn tag: https://svn.apache.org/repos/asf/hadoop/common/tags/release-0.23.1-rc2
>>> The maven artifacts for hadoop-0.23.1-rc2 are also available at repository.apache.org.
>>> Please try the release and vote; the vote will run for the usual 7 days.
>>> thanks,
>>> Arun
>>> --
>>> Arun C. Murthy
>>> Hortonworks Inc.
>>> http://hortonworks.com/
>> --
>> Todd Lipcon
>> Software Engineer, Cloudera

Todd Lipcon
Software Engineer, Cloudera

View raw message