hadoop-general mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From sanjay Radia <san...@hortonworks.com>
Subject Re: Hoping to merge HDFS-1073 branch soon
Date Tue, 12 Jul 2011 17:38:16 GMT
We can merge 1580  after 1073  is merged in.

Looks like the biggest thing in  your 1073  list  is the Backup NN related changes.
Are you shooting for end of this month?


On Jul 6, 2011, at 8:03 PM, Todd Lipcon wrote:

> Hi all,
> Just an update on this project:
> - The current list of uncommitted patches up for review is:
> 1bea9d3 HDFS-1979. Fix BackupNode and CheckpointNode
> 32db384 Amend HDFS-2011. Fix TestCheckpoint test for double close/abort of
> b6a55a4 HDFS-2101. Update remaining unit tests for new layout
> ca0ace6 HDFS-2133. Address TODOs left in code
> b46825d HDFS-1780. reduce need to rewrite fsimage on statrtup
> 30c858d HDFS-2104. Add flag to SecondaryNameNode to format it during startup
> 942eaef HDFS-2135. Fix regression of HDFS-1955 in branch
> I believe Eli is going to work on reviewing these this week.
> - I've set up a Hudson job for the branch here:
> https://builds.apache.org/job/Hadoop-Hdfs-1073-branch/
> It's currently failing because it's missing some of the patches above. After
> the above patches go in, I expect a pretty clean build, modulo maybe one or
> two things that are environment issues, which I'll tackle later this week.
> - BackupNode and CheckpointNode are working. I've done some basic functional
> testing by pounding edits into the NN while both a 2NN and a BN are
> checkpointing every 2 seconds.
> - I merged with trunk as of this morning, so I think we should be up-to-date
> with trunk patches. Aaron was very helpful and went through all NN-related
> patches in trunk from the last 3 months to make sure we didn't inadvertently
> regress anything - he discovered one bug but everything else looks good.
> Once the above patches are in the branch, I would like to merge. So, if you
> plan on reviewing pre-merge, please do so *this week*. Of course, if you
> don't have time and you find issues post-merge, I absolutely plan on fixing
> them ASAP ;-)
> Thanks
> -Todd
> On Thu, Jun 30, 2011 at 12:11 AM, Todd Lipcon <todd@cloudera.com> wrote:
>> Hey all,
>> Work on the HDFS-1073 branch has been progressing steadily, and I believe
>> we're coming close to the point where it can be merged. To briefly summarize
>> the status:
>> - NameNode and SecondaryNameNode are both fully working and have undergone
>> some stress/fault testing in addition to a over 3000 lines worth of new unit
>> tests.
>> - Most of the existing unit tests have been updated, though a few more need
>> some small tweaks (HDFS-2101)
>> - The BackupNode and CheckpointNode are not currently working, though I am
>> working on it locally and making good progress (HDFS-1979)
>> - There are a few various and sundry small improvements that should
>> probably be done before release, but I think could be done either before or
>> after merge (eg HDFS-2104)
>> Given this, I am expecting that we can merge this into trunk by the end of
>> July if not earlier, as soon as the BN/CN work is complete. If you are
>> hoping to review the code or tests before merge time, this is your early
>> warning! Please do so now!
>> Thanks!
>> -Todd
>> P.S. I will also be giving a short talk about the motivations and current
>> status of this project at Friday's contributor meeting, for those who are
>> able to attend. If we're lucky, maybe even a demo!
>> --
>> Todd Lipcon
>> Software Engineer, Cloudera
> -- 
> Todd Lipcon
> Software Engineer, Cloudera

View raw message