hadoop-hdfs-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Todd Lipcon <t...@cloudera.com>
Subject Re: Changes to FSImage/FSEditLog
Date Thu, 18 Nov 2010 00:48:05 GMT
Sure, moving this discussion to the public list. It was only a private
discussion because I find people tend to notice messages in their inbox more
than on JIRA or a list ;-)

-Todd

On Wed, Nov 17, 2010 at 4:41 PM, Jakob Homan <jghoman@gmail.com> wrote:

> Can this message please be re-posted to the hdfs-dev list, so that any
> others who may be planning on work in this code also get a heads up?
> I don't see an advantage to this being a private discussion.
>
> Thanks,
> Jakob
>
> On Wed, Nov 17, 2010 at 4:14 PM, Sanjay Radia <sradia@yahoo-inc.com>
> wrote:
> >
> > On Nov 17, 2010, at 2:08 PM, Hairong Kuang wrote:
> >
> > Hi Todd,
> >
> > Here is the list of fsimage jiras that I am working on:
> >
> > HDFS-1481 // small change
> > HDFS-1458 // small change
> > HDFS-1496  // should be checked into 022, I still have no clue how to fix
> it
> > HDFS-1070  // this one has a simple idea, but turns out that it needs
> major
> > code re-organization. I am still struggling with how to make my code
> change
> > easier to review. But this will definitely collide with yours.
> >
> >  1070 is about changing the generation and processing of an fsimage.
> > It may not conflict with Todd's changes.
> > Todd if your refactoring is similar to the one outlined by Ivan in his
> jira
> > then it is good one for all to read.
> > Jakob has another related Jira.
> > +jakob.
> >
> > Hairong
> >
> > On 11/17/10 11:57 AM, "Todd Lipcon" <todd@cloudera.com> wrote:
> >
> > Hey all,
> >
> > As many of you know, I've been working on HDFS-1073 for a few months, and
> > we're hoping to get it in for 0.22 (OOM has given the OK to merge this
> into
> > branch and Sanjay has helpfully gotten some commitment from Yahoo QA to
> help
> > test it).
> >
> > It's been difficult recently to work on the patch as a lot of changes
> have
> > gone in around FSImage and FSEditLog (eg checksumming, parallel load,
> etc)
> > so the patch falls out of date quickly and is hard to resolve the changes
> > since my patch moves a lot of code. Ivan Kelly over at Yahoo Barcelona
> has
> > also been working on a refactor and I think running into a similar issue.
> >
> > So, I just wanted to ping everyone who has been working on this area of
> the
> > code to find out if there are any more remaining JIRAs that you're hoping
> to
> > commit in the next several weeks that will touch FSImage and FSEditLog.
> If
> > so, we should try to agree on an order in which to commit them so we
> don't
> > end up stomping on each other, and we can help review each other's work
> to
> > move quicker.
> >
> > If you can just reply with a list of any remaining uncommitted JIRAs that
> > touch FSImage/FSEditLog that would be great, then we can work from there
> on
> > how to proceed and minimize the amount of patch rebasing we all have to
> do.
> >
> > Thanks!
> > -Todd
> >
> >
>



-- 
Todd Lipcon
Software Engineer, Cloudera

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message