hadoop-general mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Todd Lipcon <t...@cloudera.com>
Subject Re: Patch testing
Date Thu, 06 Jan 2011 01:42:51 GMT
On Wed, Jan 5, 2011 at 4:39 PM, Nigel Daley <ndaley@mac.com> wrote:

> Thanks for looking into it Todd.  Let's first see if you think it can be
> fixed quickly.  Let me know.
>
>
No problem, it wasn't too bad after all. Patch up on HADOOP-7087 which fixes
this test timeout for me.

-Todd


>  On Jan 5, 2011, at 4:33 PM, Todd Lipcon wrote:
>
> > On Wed, Jan 5, 2011 at 4:19 PM, Nigel Daley <ndaley@mac.com> wrote:
> >
> >> Todd, would love to get
> >> https://issues.apache.org/jira/browse/MAPREDUCE-2121 fixed first since
> >> this is failing every night on trunk.
> >>
> >
> > What if we disable that test, move that issue to 0.22 blocker, and then
> > enable the test-patch? I'll also look into that one today, but if it's
> > something that will take a while to fix, I don't think we should hold off
> > the useful testing for all the other patches.
> >
> > -Todd
> >
> > On Jan 5, 2011, at 2:45 PM, Todd Lipcon wrote:
> >>
> >>> Hi Nigel,
> >>>
> >>> MAPREDUCE-2172 has been fixed for a while. Are there any other
> particular
> >>> JIRAs you think need to be fixed before the MR test-patch queue gets
> >>> enabled? I have a lot of outstanding patches and doing all the
> test-patch
> >>> turnaround manually on 3 different boxes is a real headache.
> >>>
> >>> Thanks
> >>> -Todd
> >>>
> >>> On Tue, Dec 21, 2010 at 1:33 PM, Nigel Daley <ndaley@mac.com> wrote:
> >>>
> >>>> Ok, HDFS is now enabled.  You'll see a stream of updates shortly on
> the
> >> ~30
> >>>> Patch Available HDFS issues.
> >>>>
> >>>> Nige
> >>>>
> >>>> On Dec 20, 2010, at 12:42 PM, Jakob Homan wrote:
> >>>>
> >>>>> I committed HDFS-1511 this morning.  We should be good to go.  I
can
> >>>>> haz snooty robot butler?
> >>>>>
> >>>>> On Fri, Dec 17, 2010 at 8:31 PM, Konstantin Boudnik <cos@apache.org>
> >>>> wrote:
> >>>>>> Thanks Jacob. I am wasted already but I can do it on Sun, I
think,
> >>>>>> unless it is done earlier.
> >>>>>> --
> >>>>>> Take care,
> >>>>>> Konstantin (Cos) Boudnik
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> On Fri, Dec 17, 2010 at 19:41, Jakob Homan <jghoman@gmail.com>
> wrote:
> >>>>>>> Ok.  I'll get a patch out for 1511 tomorrow, unless someone
wants
> to
> >>>>>>> whip one up tonight.
> >>>>>>>
> >>>>>>>
> >>>>>>> On Fri, Dec 17, 2010 at 7:22 PM, Nigel Daley <ndaley@mac.com>
> wrote:
> >>>>>>>> I agree with Cos on fixing HDFS-1511 first. Once that
is done I'll
> >>>> enable hdfs patch testing.
> >>>>>>>>
> >>>>>>>> Cheers,
> >>>>>>>> Nige
> >>>>>>>>
> >>>>>>>> Sent from my iPhone4
> >>>>>>>>
> >>>>>>>> On Dec 17, 2010, at 7:01 PM, Konstantin Boudnik <cos@apache.org>
> >>>> wrote:
> >>>>>>>>
> >>>>>>>>> One more issue needs to be addressed before test-patch
is turned
> on
> >>>> HDFS is
> >>>>>>>>> https://issues.apache.org/jira/browse/HDFS-1511
> >>>>>>>>> --
> >>>>>>>>> Take care,
> >>>>>>>>> Konstantin (Cos) Boudnik
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> On Fri, Dec 17, 2010 at 16:17, Konstantin Boudnik
<
> cos@apache.org>
> >>>> wrote:
> >>>>>>>>>> Considering that because of these 4 faulty cases
every patch
> will
> >> be
> >>>>>>>>>> -1'ed a patch author will still have to look
at it and make a
> >>>> comment
> >>>>>>>>>> why this particular -1 isn't valid. Lesser work,
perhaps, but
> >>>> messier
> >>>>>>>>>> IMO. I'm not blocking it - I just feel like
there's a better
> way.
> >>>>>>>>>>
> >>>>>>>>>> --
> >>>>>>>>>> Take care,
> >>>>>>>>>> Konstantin (Cos) Boudnik
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> On Fri, Dec 17, 2010 at 15:55, Jakob Homan <jghoman@gmail.com>
> >>>> wrote:
> >>>>>>>>>>>> If HDFS is added to the test-patch queue
right now we get
> >>>>>>>>>>>> nothing but dozens of -1'ed patches.
> >>>>>>>>>>> There aren't dozens of patches being submitted
currently.  The
> -1
> >>>>>>>>>>> isn't the important thing, it's the grunt
work of actually
> >> running
> >>>>>>>>>>> (and waiting) for the tests, test-patch,
etc. that Hudson does
> so
> >>>> that
> >>>>>>>>>>> the developer doesn't have to.
> >>>>>>>>>>>
> >>>>>>>>>>> On Fri, Dec 17, 2010 at 3:48 PM, Dhruba
Borthakur <
> >>>> dhruba@gmail.com> wrote:
> >>>>>>>>>>>> +1, thanks for doing this.
> >>>>>>>>>>>>
> >>>>>>>>>>>> On Fri, Dec 17, 2010 at 3:19 PM, Jakob
Homan <
> jghoman@gmail.com
> >>>
> >>>> wrote:
> >>>>>>>>>>>>
> >>>>>>>>>>>>> So, with test-patch updated to show
the failing tests, saving
> >> the
> >>>>>>>>>>>>> developers the need to go and verify
that the failed tests
> are
> >>>> all
> >>>>>>>>>>>>> known, how do people feel about
turning on test-patch again
> for
> >>>> HDFS
> >>>>>>>>>>>>> and mapred?  I think it'll help
prevent any more tests from
> >>>> entering
> >>>>>>>>>>>>> the "yeah, we know" category.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>> jg
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> On Wed, Nov 17, 2010 at 5:08 PM,
Jakob Homan <
> >>>> jhoman@yahoo-inc.com> wrote:
> >>>>>>>>>>>>>> True, each patch would get a
-1 and the failing tests would
> >> need
> >>>> to be
> >>>>>>>>>>>>>> verified as those known bad
(BTW, it would be great if
> Hudson
> >>>> could list
> >>>>>>>>>>>>>> which tests failed in the message
it posts to JIRA).  But
> >> that's
> >>>> still
> >>>>>>>>>>>>> quite
> >>>>>>>>>>>>>> a bit less error-prone work
than if the developer runs the
> >> tests
> >>>> and
> >>>>>>>>>>>>>> test-patch themselves.  Also,
with 22 being cut, there are a
> >> lot
> >>>> of
> >>>>>>>>>>>>> patches
> >>>>>>>>>>>>>> up in the air and several developers
are juggling multiple
> >>>> patches.  The
> >>>>>>>>>>>>>> more automation we can have,
even if it's not perfect, will
> >>>> decrease
> >>>>>>>>>>>>> errors
> >>>>>>>>>>>>>> we may make.
> >>>>>>>>>>>>>> -jg
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Nigel Daley wrote:
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> On Nov 17, 2010, at 3:11
PM, Jakob Homan wrote:
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> It's also ready
to run on MapReduce and HDFS but we won't
> >>>> turn it on
> >>>>>>>>>>>>>>>>> until these projects
build and test cleanly.  Looks like
> >> both
> >>>> these
> >>>>>>>>>>>>> projects
> >>>>>>>>>>>>>>>>> currently have test
failures.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Assuming the projects
are compiling and building, is there
> a
> >>>> reason to
> >>>>>>>>>>>>>>>> not turn it on despite
the test failures? Hudson is
> >> invaluable
> >>>> to
> >>>>>>>>>>>>> developers
> >>>>>>>>>>>>>>>> who then don't have
to run the tests and test-patch
> >>>> themselves.  We
> >>>>>>>>>>>>> didn't
> >>>>>>>>>>>>>>>> turn Hudson off when
it was working previously and there
> >> were
> >>>> known
> >>>>>>>>>>>>>>>> failures.  I think one
of the reasons we have more failing
> >>>> tests now is
> >>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>> higher cost of doing
Hudson's work (not a great excuse I
> >>>> know).  This
> >>>>>>>>>>>>> is
> >>>>>>>>>>>>>>>> particularly true now
because several of the failing tests
> >>>> involve
> >>>>>>>>>>>>> tests
> >>>>>>>>>>>>>>>> timing out, making the
whole testing regime even longer.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Every single patch would
get a -1 and need investigation.
> >>>> Currently,
> >>>>>>>>>>>>> that
> >>>>>>>>>>>>>>> would be about 83 investigations
between MR and HDFS issues
> >>>> that are in
> >>>>>>>>>>>>>>> patch available state. 
Shouldn't we focus on getting these
> >>>> tests fixed
> >>>>>>>>>>>>> or
> >>>>>>>>>>>>>>> removed/?  Also, I need
to get MAPREDUCE-2172 fixed
> (applies
> >> to
> >>>> HDFS as
> >>>>>>>>>>>>>>> well) before I turn this
on.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Cheers,
> >>>>>>>>>>>>>>> Nige
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> --
> >>>>>>>>>>>> Connect to me at http://www.facebook.com/dhruba
> >>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>
> >>>>>>>
> >>>>>>
> >>>>
> >>>>
> >>>
> >>>
> >>> --
> >>> Todd Lipcon
> >>> Software Engineer, Cloudera
> >>
> >>
> >
> >
> > --
> > Todd Lipcon
> > Software Engineer, Cloudera
>
>


-- 
Todd Lipcon
Software Engineer, Cloudera

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message