bigtop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Vaughn E Clinton <Vaughn.E.Clin...@raytheon.com>
Subject Re: Bigtop: Invalid shuffle port number -1 returned
Date Thu, 16 May 2013 18:01:12 GMT

I figured out the problem; I was missing an RPM from the slave node.  For
whatever reason, the hadoop-mapred RPM did not make it to the slave node.
After installing that package, things worked as advertised as a clustered
solution.  I even added one additional node to test things.

 On another note, are there any benchmarking tools that will work with YARN
as of today.  prior to using YARN, I was testing my performance with
Intel's HiBench test suite.  Unfortunately that's based upon MR1.

Anyway, looking to hear from you.  Thanks for the quick response.

Cheers,






From:	Sean Mackrory <mackrorysd@gmail.com>
To:	user@bigtop.apache.org
Date:	05/16/2013 12:53 PM
Subject:	Re: Bigtop: Invalid shuffle port number -1 returned



Hi Vaughn,

The issue you're running into has been reported before (
https://issues.apache.org/jira/browse/BIGTOP-764) but has not been solved
yet - so any additional information you can provide about your setup would
be helpful in tracking down the root cause. When I encountered the problem,
I had to restart the services a couple of times, but after a successful
start up I never saw the problem again on that cluster.



On Thu, May 16, 2013 at 7:01 AM, Vaughn E Clinton <
Vaughn.E.Clinton@raytheon.com> wrote:
  In an attempt to build a cluster solution from my big top .5
  installation, I'm running into the following stack dump every time I
  start the nodemanager of a slave node.
  If I stop the nodemanager, the test completes successfully.

  Anyway, has anyone seem a really detailed document about clustering with
  Bigtop .5 and if so, can you point me to the site.

  One of the attempts stack dump:

  13/05/16 08:54:03 INFO mapreduce.Job: Task Id :
  attempt_1368710898922_0007_m_000008_0, Status : FAILED
  Container launch failed for container_1368710898922_0007_01_000010 :
  java.lang.IllegalStateException: Invalid shuffle port number -1 returned
  for attempt_1368710898922_0007_m_000008_0
          at
  org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl
  $Container.launch(ContainerLauncherImpl.java:168)
          at
  org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl
  $EventProcessor.run(ContainerLauncherImpl.java:390)
          at java.util.concurrent.ThreadPoolExecutor$Worker.runTask
  (ThreadPoolExecutor.java:886)
          at java.util.concurrent.ThreadPoolExecutor$Worker.run
  (ThreadPoolExecutor.java:908)
          at java.lang.Thread.run(Thread.java:662)

  Vaughn





Mime
View raw message