hadoop-general mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Todd Lipcon <t...@cloudera.com>
Subject Re: Hoping to merge HDFS-1073 branch soon
Date Thu, 07 Jul 2011 03:03:28 GMT
Hi all,

Just an update on this project:
- The current list of uncommitted patches up for review is:

1bea9d3 HDFS-1979. Fix BackupNode and CheckpointNode
32db384 Amend HDFS-2011. Fix TestCheckpoint test for double close/abort of
b6a55a4 HDFS-2101. Update remaining unit tests for new layout
ca0ace6 HDFS-2133. Address TODOs left in code
b46825d HDFS-1780. reduce need to rewrite fsimage on statrtup
30c858d HDFS-2104. Add flag to SecondaryNameNode to format it during startup
942eaef HDFS-2135. Fix regression of HDFS-1955 in branch

I believe Eli is going to work on reviewing these this week.

- I've set up a Hudson job for the branch here:
It's currently failing because it's missing some of the patches above. After
the above patches go in, I expect a pretty clean build, modulo maybe one or
two things that are environment issues, which I'll tackle later this week.

- BackupNode and CheckpointNode are working. I've done some basic functional
testing by pounding edits into the NN while both a 2NN and a BN are
checkpointing every 2 seconds.
- I merged with trunk as of this morning, so I think we should be up-to-date
with trunk patches. Aaron was very helpful and went through all NN-related
patches in trunk from the last 3 months to make sure we didn't inadvertently
regress anything - he discovered one bug but everything else looks good.

Once the above patches are in the branch, I would like to merge. So, if you
plan on reviewing pre-merge, please do so *this week*. Of course, if you
don't have time and you find issues post-merge, I absolutely plan on fixing
them ASAP ;-)


On Thu, Jun 30, 2011 at 12:11 AM, Todd Lipcon <todd@cloudera.com> wrote:

> Hey all,
> Work on the HDFS-1073 branch has been progressing steadily, and I believe
> we're coming close to the point where it can be merged. To briefly summarize
> the status:
> - NameNode and SecondaryNameNode are both fully working and have undergone
> some stress/fault testing in addition to a over 3000 lines worth of new unit
> tests.
> - Most of the existing unit tests have been updated, though a few more need
> some small tweaks (HDFS-2101)
> - The BackupNode and CheckpointNode are not currently working, though I am
> working on it locally and making good progress (HDFS-1979)
> - There are a few various and sundry small improvements that should
> probably be done before release, but I think could be done either before or
> after merge (eg HDFS-2104)
> Given this, I am expecting that we can merge this into trunk by the end of
> July if not earlier, as soon as the BN/CN work is complete. If you are
> hoping to review the code or tests before merge time, this is your early
> warning! Please do so now!
> Thanks!
> -Todd
> P.S. I will also be giving a short talk about the motivations and current
> status of this project at Friday's contributor meeting, for those who are
> able to attend. If we're lucky, maybe even a demo!
> --
> Todd Lipcon
> Software Engineer, Cloudera

Todd Lipcon
Software Engineer, Cloudera

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message