hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ted Yu <yuzhih...@gmail.com>
Subject Re: hadoop 2.0 dependency upgraded to 2.0.3-alpha
Date Fri, 22 Feb 2013 21:07:37 GMT
bq. to nail down the actual issue will  help making our case.

That is what I am trying to do. Will ping Arun at the meetup this afternoon
about when the next release is coming.

Cheers

On Fri, Feb 22, 2013 at 1:02 PM, Stack <stack@duboce.net> wrote:

> Thanks for trying to fix it Ted.  I think we need to carry our difficulty
> back up to our parent project brethren (Sounds like you have done this
> off-list; lets do it in jira/mail too).  Your digging in to nail down the
> actual issue will  help making our case.  Maybe we can get/help the
> upstream crew make another release (I don't see a note on a new 2.0 release
> up on hadoop -- have I missed it)?
>
> Thanks,
> St.Ack
>
>
> On Fri, Feb 22, 2013 at 11:49 AM, Ted Yu <yuzhihong@gmail.com> wrote:
>
> > Trying out #2, I got:
> >
> >
> >
> testExcludeAllFromMinorCompaction(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat)
> >  Time elapsed: 35.746 sec  <<< FAILURE!
> > java.lang.AssertionError
> >         at org.junit.Assert.fail(Assert.java:86)
> >         at org.junit.Assert.assertTrue(Assert.java:41)
> >         at org.junit.Assert.assertTrue(Assert.java:52)
> >         at
> >
> >
> org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat.runIncrementalPELoad(TestHFileOutputFormat.java:468)
> >         at
> >
> >
> org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat.testExcludeAllFromMinorCompaction(TestHFileOutputFormat.java:713)
> >
> > Arun Murthy got to know this defect. Another 2.0 release is coming soon
> > (next week, hopefully).
> >
> > I plan to wait for the new release.
> >
> > Thanks
> >
> > On Fri, Feb 22, 2013 at 11:39 AM, Ted Yu <yuzhihong@gmail.com> wrote:
> >
> > > The root cause was the removal of capacity-scheduler.xml<
> >
> http://svn.apache.org/viewvc/hadoop/common/branches/branch-2/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/conf/capacity-scheduler.xml
> >
> > from
> > > 2.0.3-alpha maven artifacts.
> > > This config file is needed to start CapacityScheduler which is the
> > default.
> > >
> > > After talking to Sid, I got two suggestions:
> > > 1. embed minimal capacity-scheduler.xml<
> >
> http://svn.apache.org/viewvc/hadoop/common/branches/branch-2/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/conf/capacity-scheduler.xml
> >
> > in
> > > our src/test/resources
> > > 2. use FiFo scheduler instead which doesn't require presence of config
> > file
> > >
> > > I think #2, though temporary fix, is easy to change once a future 2.0
> > > release addresses this issue.
> > >
> > > Once consensus is reached, I will attach another patch to HBASE-7904
> > >
> > > Cheers
> > >
> > >
> > > On Fri, Feb 22, 2013 at 10:31 AM, Andrew Purtell <apurtell@apache.org
> > >wrote:
> > >
> > >> For minicluster unit tests? In my opinion, no.
> > >>
> > >>
> > >> On Fri, Feb 22, 2013 at 10:10 AM, Ted Yu <yuzhihong@gmail.com> wrote:
> > >>
> > >> > I can try to find the proper usage.
> > >> >
> > >> > The question is: do we have to maintain compatibility with
> > 2.0.2-alpha ?
> > >> >
> > >> > Thanks
> > >> >
> > >> > On Fri, Feb 22, 2013 at 10:07 AM, Andrew Purtell <
> apurtell@apache.org
> > >> > >wrote:
> > >> >
> > >> > > Sigh. Every release seems to break our uses of MiniMRCluster
all
> > over
> > >> > > again.
> > >> > >
> > >> > >
> > >> > > On Fri, Feb 22, 2013 at 10:04 AM, Ted Yu <yuzhihong@gmail.com>
> > wrote:
> > >> > >
> > >> > > > I temporarily reverted this change due to:
> > >> > > >
> > >> > > > org.apache.hadoop.yarn.YarnException: Failed to Start
> > >> > > > org.apache.hadoop.mapred.MiniMRCluster
> > >> > > >         at
> > >> > > >
> > >> > > >
> > >> > >
> > >> >
> > >>
> >
> org.apache.hadoop.yarn.service.CompositeService.start(CompositeService.java:78)
> > >> > > >         at
> > >> > > >
> > >> > > >
> > >> > >
> > >> >
> > >>
> >
> org.apache.hadoop.mapred.MiniMRClientClusterFactory.create(MiniMRClientClusterFactory.java:75)
> > >> > > >         at
> > >> > > >
> > >> org.apache.hadoop.mapred.MiniMRCluster.<init>(MiniMRCluster.java:180)
> > >> > > >         at
> > >> > > >
> > >> org.apache.hadoop.mapred.MiniMRCluster.<init>(MiniMRCluster.java:170)
> > >> > > >         at
> > >> > > >
> > >> org.apache.hadoop.mapred.MiniMRCluster.<init>(MiniMRCluster.java:162)
> > >> > > >         at
> > >> > > >
> > >> org.apache.hadoop.mapred.MiniMRCluster.<init>(MiniMRCluster.java:123)
> > >> > > >         at
> > >> > > >
> > >> > > >
> > >> > >
> > >> >
> > >>
> >
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniMapReduceCluster(HBaseTestingUtility.java:1574)
> > >> > > > ...
> > >> > > > Caused by: org.apache.hadoop.yarn.YarnException:
> > >> > > > java.lang.IllegalStateException: Queue configuration missing
> child
> > >> > queue
> > >> > > > names for root
> > >> > > >         at
> > >> > > >
> > >> > > >
> > >> > >
> > >> >
> > >>
> >
> org.apache.hadoop.yarn.server.MiniYARNCluster$ResourceManagerWrapper.start(MiniYARNCluster.java:181)
> > >> > > >         at
> > >> > > >
> > >> > > >
> > >> > >
> > >> >
> > >>
> >
> org.apache.hadoop.yarn.service.CompositeService.start(CompositeService.java:68)
> > >> > > >         ... 38 more
> > >> > > > Caused by: java.lang.IllegalStateException: Queue configuration
> > >> missing
> > >> > > > child queue names for root
> > >> > > >         at
> > >> > > >
> > >> > > >
> > >> > >
> > >> >
> > >>
> >
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.parseQueue(CapacityScheduler.java:328)
> > >> > > >         at
> > >> > > >
> > >> > > >
> > >> > >
> > >> >
> > >>
> >
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.initializeQueues(CapacityScheduler.java:255)
> > >> > > >         at
> > >> > > >
> > >> > > >
> > >> > >
> > >> >
> > >>
> >
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.reinitialize(CapacityScheduler.java:220)
> > >> > > >         at
> > >> > > >
> > >> > > >
> > >> > >
> > >> >
> > >>
> >
> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.init(ResourceManager.java:226)
> > >> > > >         at
> > >> > > >
> > >> > > >
> > >> > >
> > >> >
> > >>
> >
> org.apache.hadoop.yarn.server.MiniYARNCluster$ResourceManagerWrapper.start(MiniYARNCluster.java:161)
> > >> > > >
> > >> > > > Cheers
> > >> > > >
> > >> > > > On Fri, Feb 22, 2013 at 9:55 AM, Andrew Purtell <
> > >> apurtell@apache.org>
> > >> > > > wrote:
> > >> > > >
> > >> > > > > Have you tried running the mapreduce unit tests with
this
> > version
> > >> > Ted?
> > >> > > > Just
> > >> > > > > curious. They have given us grief in the past.
> > >> > > > >
> > >> > > > >
> > >> > > > > On Fri, Feb 22, 2013 at 9:35 AM, Ted Yu <yuzhihong@gmail.com>
> > >> wrote:
> > >> > > > >
> > >> > > > > > Hi,
> > >> > > > > > I have integrated the following JIRA to trunk:
> > >> > > > > > HBASE-7904 Upgrade hadoop 2.0 dependency to 2.0.3-alpha
> > >> > > > > >
> > >> > > > > > Earlier this week, Ramkrishna encountered some
problem where
> > >> HBase
> > >> > > was
> > >> > > > > > built against 2.0.2-alpha but hadoop cluster deployment
was
> > >> using
> > >> > > > > > 2.0.3-alpha
> > >> > > > > >
> > >> > > > > > In case you use HBase against hadoop 2.0, please
note this
> > >> change.
> > >> > > > > >
> > >> > > > > > Cheers
> > >> > > > > >
> > >> > > > >
> > >> > > > >
> > >> > > > >
> > >> > > > > --
> > >> > > > > Best regards,
> > >> > > > >
> > >> > > > >    - Andy
> > >> > > > >
> > >> > > > > Problems worthy of attack prove their worth by hitting
back. -
> > >> Piet
> > >> > > Hein
> > >> > > > > (via Tom White)
> > >> > > > >
> > >> > > >
> > >> > >
> > >> > >
> > >> > >
> > >> > > --
> > >> > > Best regards,
> > >> > >
> > >> > >    - Andy
> > >> > >
> > >> > > Problems worthy of attack prove their worth by hitting back.
-
> Piet
> > >> Hein
> > >> > > (via Tom White)
> > >> > >
> > >> >
> > >>
> > >>
> > >>
> > >> --
> > >> Best regards,
> > >>
> > >>    - Andy
> > >>
> > >> Problems worthy of attack prove their worth by hitting back. - Piet
> Hein
> > >> (via Tom White)
> > >>
> > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message